Hi,

Another crash when using ex2 (src/ksp/ksp/examples/tutorials) in parallel on GPU, when playing with precondtionners:

mpiexec -np 1 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi -sub_pc_type jacobi Norm of error 5.71895e-06 iterations 14 mpiexec -np 1 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi -sub_pc_type jacobi -mat_type aijcusp -vec_type cusp Norm of error 5.71895e-06 iterations 14 mpiexec -np 2 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi -sub_pc_type jacobi Norm of error 5.71895e-06 iterations 14 mpiexec -np 2 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi -sub_pc_type jacobi -mat_type aijcusp -vec_type cusp [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors [0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run [0]PETSC ERROR: to get more information on the crash. [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------

Whereas the last one should work exactly as: mpiexec -np 2 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type jacobi -mat_type aijcusp -vec_type cusp Norm of error 5.71895e-06 iterations 14

Or I miss something ?

this is a known problem with block-jacobi for which Dominic created a pull request here: https://bitbucket.org/petsc/petsc/pull-request/178/make-cusp-and-cusparse-so... I was talking to Dominic yesterday and we plan to get the series of GPU-related pull requests merged into next and master within the next couple of days. If it's urgent, please try that feature branch in the meanwhile. Best regards, Karli