Hi wizards of "->ops->" indirections, I'm trying to use bcgs without preconditioning, for now, and the iterations using "-vec_type cusp -mat_type mpiaijcusp" don't match serial or non-GPU options. I've attached the test program and the 4 outputs (serial/parallel + CPU/GPU). All this is with petsc-dev downloaded now and real scalars. Only the parallel GPU results are different starting from third residual norm seen in results.txt. The other three match one another. Am I doing something wrong? fbcgs (bcgs with -ksp_bcgs_flexible) works fine with all the serial/parallel or CPU/GPU options I've tried. Let me know if you need the matrix, rhs, and initial guess binary files that are read in by the test program. Thanks, Chetan