On Fri, Jan 5, 2024 at 4:40 AM Edoardo Centofanti < [email protected]> wrote:
Dear all,
I have a code running both on GPU and CPU. This code has both cuda kernels and calls to PETSc KSP and related PC. In particular, I am trying to perform tests with Hypre BoomerAMG both on CPU and GPU. In order to do that, on CPU I am running the code with -mat_type mpiaij and -vec_type mpi, while on GPU I am using respectively aijcusparse and cuda.
The configuration ran for PETSc (version is 3.20) is ./configure PETSC_ARCH=arch-linux-cuda --with-cuda --download-mumps --download-hypre --with-debugging=0 --download-scalapack --download-parmetis --download-metis --download-fblaslapack=1 --download-mpich --download-make --download-cmake
My problem is that when I try to run my code on GPU it works well, while on CPU with mat_type mpiaij and -vec_type mpi it works regularly until the call to Hypre, then freezes (I have to kill it myself), while with GAMG it works on CPU with the same configuration (and the same code, just PC is changed).
Probably because if hypre is configured with GPU support, even petsc is running on CPU (with -mat_type mpiaij and -vec_type mpi), hypre is still trying to run on GPU, causing the hanging problem. The petsc/hypre interface is not able to support both CPU/GPU with the same build.
On another machine running PETSc version 3.17 everything worked smoothly with the same code and the same configuration, also on Hypre.
I guess with petsc-3.17, you also used a different hypre version (downloaded by petsc).
Do you have any insights on what is happening?
Best regards, Edoardo