November 2021 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] invocation of nvcc
by Satish Balay 04 Nov '21

04 Nov '21

I think we went through this issue before. nvc is the 'c' compiler. And for some reason its giving 'cuda' warnings. I think you might have switched progenv last time [and avoided this compiler] You can try using this compiler manually - on simple code - and then seek help from the admins on how to avoid these warnings.. Satish - create test.c with: #include <stdlib.h> compile (or preporcess): cc -E test.c - You should get: stderr: nvc-Warning-The -gpu option has no effect unless a language-specific option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, -cuda) Now provide this info to the machine admins [or nvidia folk] - and ask how to get rid of this message. Satish On Thu, 4 Nov 2021, Mark Adams wrote: > Correction it is nvc: > > nvc-Warning-The -gpu option has no effect unless a language-specific option > to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, -cuda) > > And I add -mp=gpu to CUDAFLAGS > > > On Thu, Nov 4, 2021 at 1:09 PM Satish Balay <balay(a)mcs.anl.gov> wrote: > > > > > On Thu, 4 Nov 2021, Mark Adams wrote: > > > > > Does anyone know if PETSc calls nvcc and hence can add flags to the > > > invocation? nvcc wants a flag like -mp=gpu but I don't know if we do that > > > or a compiler wrapper in the environment. > > > > $ ./configure --help |grep CUDA > > <snip> > > --CUDAC=<prog> > > Specify the CUDA compiler > > --CUDAFLAGS=<string> > > Specify the CUDA compiler options > > <snip> > > > > Satish > > >

1 0

Re: [petsc-dev] invocation of nvcc
by Mark Adams 04 Nov '21

04 Nov '21

Correction it is nvc: nvc-Warning-The -gpu option has no effect unless a language-specific option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, -cuda) And I add -mp=gpu to CUDAFLAGS On Thu, Nov 4, 2021 at 1:09 PM Satish Balay <balay(a)mcs.anl.gov> wrote: > > On Thu, 4 Nov 2021, Mark Adams wrote: > > > Does anyone know if PETSc calls nvcc and hence can add flags to the > > invocation? nvcc wants a flag like -mp=gpu but I don't know if we do that > > or a compiler wrapper in the environment. > > $ ./configure --help |grep CUDA > <snip> > --CUDAC=<prog> > Specify the CUDA compiler > --CUDAFLAGS=<string> > Specify the CUDA compiler options > <snip> > > Satish >

1 0

Re: [petsc-dev] invocation of nvcc
by Satish Balay 04 Nov '21

04 Nov '21

On Thu, 4 Nov 2021, Mark Adams wrote: > Does anyone know if PETSc calls nvcc and hence can add flags to the > invocation? nvcc wants a flag like -mp=gpu but I don't know if we do that > or a compiler wrapper in the environment. $ ./configure --help |grep CUDA <snip> --CUDAC=<prog> Specify the CUDA compiler --CUDAFLAGS=<string> Specify the CUDA compiler options <snip> Satish

1 0

invocation of nvcc
by Mark Adams 04 Nov '21

04 Nov '21

Does anyone know if PETSc calls nvcc and hence can add flags to the invocation? nvcc wants a flag like -mp=gpu but I don't know if we do that or a compiler wrapper in the environment.

1 0

[DocTip!] #1 : What does the docs build build? What can I delete?
by Patrick Sanan 03 Nov '21

03 Nov '21

This is an experiment to try to disseminate information about the docs. Things here will be less formal and more practical than what might be at petsc.org/release/developers/documentation, and importantly, because this is a mailing list, I can describe things that will go out of date as we improve the docs but is nonetheless useful to current development. On to tip #1 ! What happens when you build the docs locally by following our instructions? What files are generated and can I delete them? The basic instructions are here (Bonus tip: make sure you don't accidentally put petsc-docs-env inside of doc/, as that will greatly confuse Sphinx as it locates its own .rst files!). https://petsc.org/release/developers/documentation/#building-the-html-docs-… 1. doc/makefile calls sphinx-build 2. sphinx-build looks at doc/conf.py and searches the whole doc/ tree for .rst files 3. It then uses the html builder to generate _build/html . 4. You open _build/html/index.html That's not the whole story, though, because we still rely on the "classic" docs system which has a lot of useful logic we haven't made more Sphinx/python native yet. This is controlled in two ways: 1. With a custom extension in doc/ext, relying on the classic "htmlmap" file 2. With some custom hooks defined at the end of conf.py Because of these, when you run sphinx-build, PETSc itself will be configured as minimally as possible to run "make alldoc12", and you'll end up with an arch-classic-docs directory and a doc/_build_classic directory . You'll see (lots of) additional output due to this configuration, and some banners at the end of the build as the generated files are copied and placeholder relative links required by the extension are resolved. That's still not the whole story, though, because we have a special, "add files only", images repository. Thus, in doc/makefile, you'll see very simple logic which does a Git clone to populate doc/images from gitlab.com/petsc/images . So, I end up with these untracked directories: doc/_build Sphinx's output directory. Can be generated quite quickly, so delete whenever you'd like. "make clean" in doc/ deletes this. doc/_build_classic Output for the classic docs build. Delete when desired, but realize that regenerating it can be slow, so only do so if e.g. you've changed the man pages. "make clean" in doc/ also deletes this. doc/images Will not currently be updated in any intelligent way, so if you have images issues, make sure you haven't got any of your own new stuff in here, delete it, and let it be re-cloned. arch-classic-docs Delete as desired, but in general you shouldn't need to.

1 0

petsc-3.16.1 now available
by Satish Balay 02 Nov '21

02 Nov '21

Dear PETSc users, The patch release petsc-3.16.1 is now available for download. https://petsc.org/release/download/ Satish

1 0

Re: [petsc-dev] Running with CUDA and CUDA_VISIBLE_DEVICES=-1
by Jacob Faibussowitsch 01 Nov '21

01 Nov '21

https://gitlab.com/petsc/petsc/-/merge_requests/4512 <https://gitlab.com/petsc/petsc/-/merge_requests/4512> Best regards, Jacob Faibussowitsch (Jacob Fai - booss - oh - vitch) > On Nov 1, 2021, at 11:00, Barry Smith <bsmith(a)petsc.dev> wrote: > > > PETSc code could check for the environmental variable CUDA_VISIBLE_DEVICES=-1 if that makes sense to resolve the situation. > > > >> On Nov 1, 2021, at 11:43 AM, Jacob Faibussowitsch <jacob.fai(a)gmail.com <mailto:[email protected]>> wrote: >> >> Looks like you are tripping up the following: >> >> cerr = cupmGetDeviceCount(&ndev); >> if (PetscUnlikely(cerr == cupmErrorStubLibrary)) { >> … // handle missing driver or stub library >> } else {CHKERRCUPM(cerr);} // your error here >> >> Is it an error if a user configures with cuda (i.e. signals intent to use cuda) but disables all the devices? On the one hand, yes this can be considered an error if the user inadvertently disables the devices via this environment variable without knowing, but on the other hand they should be able to freely set this variable without petsc crashing… Should we warn users? Handle this silently? >> >> Note that petsc does provide '-device_enable none’ option to disable all devices, or if you only want to disable cuda devices '-device_enable_cuda none’ which should achieve the same effect as CUDA_VISIBLE_DEVICES=-1. But maybe it is too obscure to ask users to know about and use these flags instead of setting the cuda env variables. (Btw, can you test that using ‘-device_enable_cuda none’ does not crash when setting CUDA_VISIBLE_DEVICES=-1?) >> >> Best regards, >> >> Jacob Faibussowitsch >> (Jacob Fai - booss - oh - vitch) >> >>> On Nov 1, 2021, at 10:09, Stefano Zampini <stefano.zampini(a)gmail.com <mailto:[email protected]>> wrote: >>> >>> Just found out that if we configure with cuda and then want to run on CPU only using CUDA_VISIBLE_DEVICES=-1 PETSc errors out. Is this intended behavior? I supposed it should work >>> This is with main >>> >>> (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check >>> Running check examples to verify correct installation >>> Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double >>> C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process >>> C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes >>> C/C++ example src/snes/tutorials/ex19 run successfully with cuda >>> Completed test examples >>> >>> (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check CUDA_VISIBLE_DEVICES=1 >>> Running check examples to verify correct installation >>> Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double >>> C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process >>> C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes >>> C/C++ example src/snes/tutorials/ex19 run successfully with cuda >>> Completed test examples >>> >>> (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check CUDA_VISIBLE_DEVICES=-1 >>> Running check examples to verify correct installation >>> Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double >>> Possible error running C/C++ src/snes/tutorials/ex19 with 1 MPI process >>> See http://www.mcs.anl.gov/petsc/documentation/faq.html <http://www.mcs.anl.gov/petsc/documentation/faq.html> >>> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>> [0]PETSC ERROR: GPU error >>> [0]PETSC ERROR: cuda error 100 (cudaErrorNoDevice) : no CUDA-capable device is detected >>> [0]PETSC ERROR: See https://petsc.org/release/faq/ <https://petsc.org/release/faq/> for trouble shooting. >>> [0]PETSC ERROR: Petsc Development GIT revision: v3.16.0-368-g72b201b202 GIT Date: 2021-10-29 14:48:19 +0300 >>> [0]PETSC ERROR: ./ex19 on a arch-ecrcml-cuda-double named qaysar.kaust.edu.sa <http://qaysar.kaust.edu.sa/> by zampins Mon Nov 1 18:06:12 2021 >>> [0]PETSC ERROR: Configure options --with-blaslapack-include=/home/zampins/miniforge/envs/ecrcml-cuda/include --with-blaslapack-lib=/home/zampins/miniforge/envs/ecrcml-cuda/lib/libmkl_rt.so --download-h2opus --with-cuda --with-kblas-dir=/home/zampins/miniforge/envs/ecrcml-cuda --with-magma-dir=/home/zampins/miniforge/envs/ecrcml-cuda --LDFLAGS=/usr/lib/x86_64-linux-gnu/libcuda.so --with-debugging=1 --with-openmp --with-precision=double --with-fc=0 PETSC_ARCH=arch-ecrcml-cuda-double PETSC_DIR=/home/zampins/miniforge/Devel/petsc >>> [0]PETSC ERROR: #1 initialize() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/impls/cupm/cupmdevice.cxx:302 >>> [0]PETSC ERROR: #2 PetscDeviceInitializeTypeFromOptions_Private() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/interface/device.cxx:292 >>> [0]PETSC ERROR: #3 PetscDeviceInitializeFromOptions_Internal() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/interface/device.cxx:417 >>> [0]PETSC ERROR: #4 PetscInitialize_Common() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/pinit.c:956 >>> [0]PETSC ERROR: #5 PetscInitialize() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/pinit.c:1231 >>> -------------------------------------------------------------------------- >>> Primary job terminated normally, but 1 process returned >>> a non-zero exit code. Per user-direction, the job has been aborted. >>> -------------------------------------------------------------------------- >>> -------------------------------------------------------------------------- >>> >>> [ >> >

1 0

Re: [petsc-dev] Running with CUDA and CUDA_VISIBLE_DEVICES=-1
by Barry Smith 01 Nov '21

01 Nov '21

PETSc code could check for the environmental variable CUDA_VISIBLE_DEVICES=-1 if that makes sense to resolve the situation. > On Nov 1, 2021, at 11:43 AM, Jacob Faibussowitsch <jacob.fai(a)gmail.com> wrote: > > Looks like you are tripping up the following: > > cerr = cupmGetDeviceCount(&ndev); > if (PetscUnlikely(cerr == cupmErrorStubLibrary)) { > … // handle missing driver or stub library > } else {CHKERRCUPM(cerr);} // your error here > > Is it an error if a user configures with cuda (i.e. signals intent to use cuda) but disables all the devices? On the one hand, yes this can be considered an error if the user inadvertently disables the devices via this environment variable without knowing, but on the other hand they should be able to freely set this variable without petsc crashing… Should we warn users? Handle this silently? > > Note that petsc does provide '-device_enable none’ option to disable all devices, or if you only want to disable cuda devices '-device_enable_cuda none’ which should achieve the same effect as CUDA_VISIBLE_DEVICES=-1. But maybe it is too obscure to ask users to know about and use these flags instead of setting the cuda env variables. (Btw, can you test that using ‘-device_enable_cuda none’ does not crash when setting CUDA_VISIBLE_DEVICES=-1?) > > Best regards, > > Jacob Faibussowitsch > (Jacob Fai - booss - oh - vitch) > >> On Nov 1, 2021, at 10:09, Stefano Zampini <stefano.zampini(a)gmail.com <mailto:[email protected]>> wrote: >> >> Just found out that if we configure with cuda and then want to run on CPU only using CUDA_VISIBLE_DEVICES=-1 PETSc errors out. Is this intended behavior? I supposed it should work >> This is with main >> >> (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check >> Running check examples to verify correct installation >> Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double >> C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process >> C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes >> C/C++ example src/snes/tutorials/ex19 run successfully with cuda >> Completed test examples >> >> (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check CUDA_VISIBLE_DEVICES=1 >> Running check examples to verify correct installation >> Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double >> C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process >> C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes >> C/C++ example src/snes/tutorials/ex19 run successfully with cuda >> Completed test examples >> >> (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check CUDA_VISIBLE_DEVICES=-1 >> Running check examples to verify correct installation >> Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double >> Possible error running C/C++ src/snes/tutorials/ex19 with 1 MPI process >> See http://www.mcs.anl.gov/petsc/documentation/faq.html <http://www.mcs.anl.gov/petsc/documentation/faq.html> >> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >> [0]PETSC ERROR: GPU error >> [0]PETSC ERROR: cuda error 100 (cudaErrorNoDevice) : no CUDA-capable device is detected >> [0]PETSC ERROR: See https://petsc.org/release/faq/ <https://petsc.org/release/faq/> for trouble shooting. >> [0]PETSC ERROR: Petsc Development GIT revision: v3.16.0-368-g72b201b202 GIT Date: 2021-10-29 14:48:19 +0300 >> [0]PETSC ERROR: ./ex19 on a arch-ecrcml-cuda-double named qaysar.kaust.edu.sa <http://qaysar.kaust.edu.sa/> by zampins Mon Nov 1 18:06:12 2021 >> [0]PETSC ERROR: Configure options --with-blaslapack-include=/home/zampins/miniforge/envs/ecrcml-cuda/include --with-blaslapack-lib=/home/zampins/miniforge/envs/ecrcml-cuda/lib/libmkl_rt.so --download-h2opus --with-cuda --with-kblas-dir=/home/zampins/miniforge/envs/ecrcml-cuda --with-magma-dir=/home/zampins/miniforge/envs/ecrcml-cuda --LDFLAGS=/usr/lib/x86_64-linux-gnu/libcuda.so --with-debugging=1 --with-openmp --with-precision=double --with-fc=0 PETSC_ARCH=arch-ecrcml-cuda-double PETSC_DIR=/home/zampins/miniforge/Devel/petsc >> [0]PETSC ERROR: #1 initialize() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/impls/cupm/cupmdevice.cxx:302 >> [0]PETSC ERROR: #2 PetscDeviceInitializeTypeFromOptions_Private() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/interface/device.cxx:292 >> [0]PETSC ERROR: #3 PetscDeviceInitializeFromOptions_Internal() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/interface/device.cxx:417 >> [0]PETSC ERROR: #4 PetscInitialize_Common() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/pinit.c:956 >> [0]PETSC ERROR: #5 PetscInitialize() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/pinit.c:1231 >> -------------------------------------------------------------------------- >> Primary job terminated normally, but 1 process returned >> a non-zero exit code. Per user-direction, the job has been aborted. >> -------------------------------------------------------------------------- >> -------------------------------------------------------------------------- >> >> [ >

1 0

Re: [petsc-dev] Running with CUDA and CUDA_VISIBLE_DEVICES=-1
by Jacob Faibussowitsch 01 Nov '21

01 Nov '21

Looks like you are tripping up the following: cerr = cupmGetDeviceCount(&ndev); if (PetscUnlikely(cerr == cupmErrorStubLibrary)) { … // handle missing driver or stub library } else {CHKERRCUPM(cerr);} // your error here Is it an error if a user configures with cuda (i.e. signals intent to use cuda) but disables all the devices? On the one hand, yes this can be considered an error if the user inadvertently disables the devices via this environment variable without knowing, but on the other hand they should be able to freely set this variable without petsc crashing… Should we warn users? Handle this silently? Note that petsc does provide '-device_enable none’ option to disable all devices, or if you only want to disable cuda devices '-device_enable_cuda none’ which should achieve the same effect as CUDA_VISIBLE_DEVICES=-1. But maybe it is too obscure to ask users to know about and use these flags instead of setting the cuda env variables. (Btw, can you test that using ‘-device_enable_cuda none’ does not crash when setting CUDA_VISIBLE_DEVICES=-1?) Best regards, Jacob Faibussowitsch (Jacob Fai - booss - oh - vitch) > On Nov 1, 2021, at 10:09, Stefano Zampini <stefano.zampini(a)gmail.com> wrote: > > Just found out that if we configure with cuda and then want to run on CPU only using CUDA_VISIBLE_DEVICES=-1 PETSc errors out. Is this intended behavior? I supposed it should work > This is with main > > (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check > Running check examples to verify correct installation > Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double > C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process > C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes > C/C++ example src/snes/tutorials/ex19 run successfully with cuda > Completed test examples > > (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check CUDA_VISIBLE_DEVICES=1 > Running check examples to verify correct installation > Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double > C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process > C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes > C/C++ example src/snes/tutorials/ex19 run successfully with cuda > Completed test examples > > (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check CUDA_VISIBLE_DEVICES=-1 > Running check examples to verify correct installation > Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double > Possible error running C/C++ src/snes/tutorials/ex19 with 1 MPI process > See http://www.mcs.anl.gov/petsc/documentation/faq.html <http://www.mcs.anl.gov/petsc/documentation/faq.html> > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > [0]PETSC ERROR: GPU error > [0]PETSC ERROR: cuda error 100 (cudaErrorNoDevice) : no CUDA-capable device is detected > [0]PETSC ERROR: See https://petsc.org/release/faq/ <https://petsc.org/release/faq/> for trouble shooting. > [0]PETSC ERROR: Petsc Development GIT revision: v3.16.0-368-g72b201b202 GIT Date: 2021-10-29 14:48:19 +0300 > [0]PETSC ERROR: ./ex19 on a arch-ecrcml-cuda-double named qaysar.kaust.edu.sa <http://qaysar.kaust.edu.sa/> by zampins Mon Nov 1 18:06:12 2021 > [0]PETSC ERROR: Configure options --with-blaslapack-include=/home/zampins/miniforge/envs/ecrcml-cuda/include --with-blaslapack-lib=/home/zampins/miniforge/envs/ecrcml-cuda/lib/libmkl_rt.so --download-h2opus --with-cuda --with-kblas-dir=/home/zampins/miniforge/envs/ecrcml-cuda --with-magma-dir=/home/zampins/miniforge/envs/ecrcml-cuda --LDFLAGS=/usr/lib/x86_64-linux-gnu/libcuda.so --with-debugging=1 --with-openmp --with-precision=double --with-fc=0 PETSC_ARCH=arch-ecrcml-cuda-double PETSC_DIR=/home/zampins/miniforge/Devel/petsc > [0]PETSC ERROR: #1 initialize() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/impls/cupm/cupmdevice.cxx:302 > [0]PETSC ERROR: #2 PetscDeviceInitializeTypeFromOptions_Private() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/interface/device.cxx:292 > [0]PETSC ERROR: #3 PetscDeviceInitializeFromOptions_Internal() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/interface/device.cxx:417 > [0]PETSC ERROR: #4 PetscInitialize_Common() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/pinit.c:956 > [0]PETSC ERROR: #5 PetscInitialize() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/pinit.c:1231 > -------------------------------------------------------------------------- > Primary job terminated normally, but 1 process returned > a non-zero exit code. Per user-direction, the job has been aborted. > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > > [

1 0

Running with CUDA and CUDA_VISIBLE_DEVICES=-1
by Stefano Zampini 01 Nov '21

01 Nov '21

Just found out that if we configure with cuda and then want to run on CPU only using CUDA_VISIBLE_DEVICES=-1 PETSc errors out. Is this intended behavior? I supposed it should work This is with main (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check Running check examples to verify correct installation Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes C/C++ example src/snes/tutorials/ex19 run successfully with cuda Completed test examples (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check CUDA_VISIBLE_DEVICES=1 Running check examples to verify correct installation Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes C/C++ example src/snes/tutorials/ex19 run successfully with cuda Completed test examples (ecrcml-cuda) zampins@qaysar:~/miniforge/Devel/petsc$ make check CUDA_VISIBLE_DEVICES=-1 Running check examples to verify correct installation Using PETSC_DIR=/home/zampins/miniforge/Devel/petsc and PETSC_ARCH=arch-ecrcml-cuda-double Possible error running C/C++ src/snes/tutorials/ex19 with 1 MPI process See http://www.mcs.anl.gov/petsc/documentation/faq.html [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: GPU error [0]PETSC ERROR: cuda error 100 (cudaErrorNoDevice) : no CUDA-capable device is detected [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. [0]PETSC ERROR: Petsc Development GIT revision: v3.16.0-368-g72b201b202 GIT Date: 2021-10-29 14:48:19 +0300 [0]PETSC ERROR: ./ex19 on a arch-ecrcml-cuda-double named qaysar.kaust.edu.sa by zampins Mon Nov 1 18:06:12 2021 [0]PETSC ERROR: Configure options --with-blaslapack-include=/home/zampins/miniforge/envs/ecrcml-cuda/include --with-blaslapack-lib=/home/zampins/miniforge/envs/ecrcml-cuda/lib/libmkl_rt.so --download-h2opus --with-cuda --with-kblas-dir=/home/zampins/miniforge/envs/ecrcml-cuda --with-magma-dir=/home/zampins/miniforge/envs/ecrcml-cuda --LDFLAGS=/usr/lib/x86_64-linux-gnu/libcuda.so --with-debugging=1 --with-openmp --with-precision=double --with-fc=0 PETSC_ARCH=arch-ecrcml-cuda-double PETSC_DIR=/home/zampins/miniforge/Devel/petsc [0]PETSC ERROR: #1 initialize() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/impls/cupm/cupmdevice.cxx:302 [0]PETSC ERROR: #2 PetscDeviceInitializeTypeFromOptions_Private() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/interface/device.cxx:292 [0]PETSC ERROR: #3 PetscDeviceInitializeFromOptions_Internal() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/device/interface/device.cxx:417 [0]PETSC ERROR: #4 PetscInitialize_Common() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/pinit.c:956 [0]PETSC ERROR: #5 PetscInitialize() at /home/zampins/miniforge/Devel/petsc/src/sys/objects/pinit.c:1231 -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- [

1 0