November 2021 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] I am getting this error ...
by Satish Balay 05 Nov '21

05 Nov '21

I guess another way to deal with this is: not use CFLAGS etc.. --with-cc='cc -mp=gpu' --with-cxx='CC -mp=gpu' --with-fc='ftn -mp=gpu' Satish On Fri, 5 Nov 2021, Mark Adams wrote: > Yes, thanks. > I emailed the NERSc person and told him where we are and that we could fix > this manually, but I don't have this issue on Summit ... > He did not understand for the longest time, not sure he does now, that we > do not add -gpu. > And I gave him your reproducer. > With any luck, with the reproducer, they can figure this out at some point. > MPI is not working well for my scaling studies and my app has also given up > on nvhpc, so we are good. > > > On Fri, Nov 5, 2021 at 9:49 AM Satish Balay <balay(a)mcs.anl.gov> wrote: > > > For now - you could manually edit petscvariables and remove -mp=gpu from > > it. > > > > Its primarily required to make configure happy. > > > > Satish > > > > > > > > On Thu, 4 Nov 2021, Barry Smith wrote: > > > > > > > > This comes from the persistent problem with PETSc's make system using > > too many flags for compiling CUDA that have not been tested by configure. > > See below the -mp=gpu is provided probably from the CPPFLAGS or > > CXXPPFLAGS(sp) that is improperly used by the PETSc makefiles to compile > > CUDA code! > > > > > > > > > Using CUDA compile: > > /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc -o .o > > -I/opt/cray/pe/mpich/8.1.10/ofi/nvidia/20.7/include -I/opt/cray/pe/libsci/ > > 21.08.1.2/NVIDIA/20.7/x86_64/include -I/opt/cray/pe/pmi/6.0.14/include > > -I/opt/cray/pe/dsmml/0.2.2/dsmml//include > > -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include -g > > -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 > > -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -std=c++17 -gencode > > arch=compute_80,code=sm_80 -Wno-deprecated-gpu-targets -c > > --compiler-options=-I/global/homes/m/madams/petsc/include > > -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include > > -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include -mp=gpu > > -I/global/homes/m/madams/petsc/include > > -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include > > -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include > > > > > > > > > > > > > On Nov 4, 2021, at 7:31 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > > > > > > > OK, configure is done. > > > > Maybe I have too many -mp=gpu > > > > > > > > CUDAC > > arch-perlmutter-opt-nvidia-cuda/obj/sys/classes/random/impls/curand/curand2.o > > > > gcc: error: unrecognized command line option ‘-mp=gpu’; did you mean > > ‘-mpku’? > > > > > > > > On Thu, Nov 4, 2021 at 5:51 PM Barry Smith <bsmith(a)petsc.dev <mailto: > > bsmith(a)petsc.dev>> wrote: > > > > > > > > Need the same thing for the C++ preprocessor flag > > > > > > > > > > > > > > > >> On Nov 4, 2021, at 5:44 PM, Mark Adams <mfadams(a)lbl.gov <mailto: > > mfadams(a)lbl.gov>> wrote: > > > >> > > > >> It gets a lot further. > > > >> > > > >> On Thu, Nov 4, 2021 at 5:32 PM Mark Adams <mfadams(a)lbl.gov <mailto: > > mfadams(a)lbl.gov>> wrote: > > > >> OK, sorry I missed the CPPFLAGS. It is running now. > > > >> Thanks, > > > >> > > > >> On Thu, Nov 4, 2021 at 4:43 PM Satish Balay <balay(a)mcs.anl.gov > > <mailto:[email protected]>> wrote: > > > >> Multiple e-mail threads on the same issue (: > > > >> > > > >> As suggested in my earlier thread - add -mp=gpu to both CPPFLAGS and > > CFLAGS [or LDFLAGS] > > > >> > > > >> Satish > > > >> > > > >> ------- > > > >> Executing: cc -o /tmp/petsc-Vvs8_T/config.setCompilers/conftest -g > > -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 > > /tmp/petsc-Vvs8_T/config.setCompilers/conftest.o > > > >> Possible ERROR while running linker: > > > >> stderr: > > > >> nvc-Warning-The -gpu option has no effect unless a language-specific > > option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, > > -cuda) > > > >> > > > >> > > > >> On Thu, 4 Nov 2021, Mark Adams wrote: > > > >> > > > >> > It is CPPFLAGS. I seem to get the same behavior. > > > >> > > > > >> > FWIW, I did get this response from NERSc but I don't know how to > > interpret > > > >> > it. > > > >> > > > > >> > He seems to be saying that I don't need -mp=gpu for the device > > compiler > > > >> > (nvcc). He seems to think that I am adding -gpu. > > > >> > > > > >> > nvcc -- The device compiler does not need any of those flags > > because it > > > >> > already knows that it's being fed cuda code. The warning you're > > seeing is > > > >> > coming from nvc (which is the host / CPU side compiler) if you're > > in the > > > >> > PrgEnv-nvidia environment. You should not need to add -mp=gpu and > > -cuda, > > > >> > please just add the -cuda flag (to your host code) not to the > > device code. > > > >> > > > > >> > I will try to talk with this guy again. > > > >> > > > > >> > Thanks, > > > >> > > > > >> > > > > >> > On Thu, Nov 4, 2021 at 4:11 PM Barry Smith <bsmith(a)petsc.dev > > <mailto:[email protected]>> wrote: > > > >> > > > > >> > > > > > >> > > Yes, you need to use the CPPFLAGS which maybe called CPPCFLAGS > > I am not > > > >> > > sure > > > >> > > > > > >> > > > > > >> > > On Nov 4, 2021, at 3:23 PM, Mark Adams <mfadams(a)lbl.gov <mailto: > > mfadams(a)lbl.gov>> wrote: > > > >> > > > > > >> > > Ah, CCFLAGS does not seem to work. > > > >> > > > > > >> > > On Thu, Nov 4, 2021 at 3:07 PM Barry Smith <bsmith(a)petsc.dev > > <mailto:[email protected]>> wrote: > > > >> > > > > > >> > >> > > > >> > >> You have to pass in the flag to turn off the bitching about > > -gpu to the > > > >> > >> C preprocessor, not the C compiler. > > > >> > >> > > > >> > >> > > > >> > >> stderr: > > > >> > >> nvc-Warning-The -gpu option has no effect unless a > > language-specific > > > >> > >> option to enable GPU code generation is used (e.g.: -acc, > > -mp=gpu, -stdpar, > > > >> > >> -cuda) > > > >> > >> Source: > > > >> > >> #include "confdefs.h" > > > >> > >> #include "conffix.h" > > > >> > >> #include <stdlib.h> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > On Nov 4, 2021, at 2:49 PM, Mark Adams <mfadams(a)lbl.gov > > <mailto:[email protected]>> wrote: > > > >> > >> > > > > >> > >> > on Perlmutter with nvhpc: > > > >> > >> > > > > >> > >> > Defined make macro "CPP" to "cc --use cpp32" > > > >> > >> > Preprocessing source: > > > >> > >> > #include "confdefs.h" > > > >> > >> > #include "conffix.h" > > > >> > >> > #include <stdlib.h> > > > >> > >> > > > > >> > >> > Executing: cc --use cpp32 > > -I/tmp/petsc-jV9U1b/config.setCompilers > > > >> > >> /tmp/petsc-jV9U1b/config.setCompilers/conftest.c > > > >> > >> > Possible ERROR while running preprocessor: exit code 1 > > > >> > >> > stderr: > > > >> > >> > nvc-Error-Unknown switch: --use > > > >> > >> > Source: > > > >> > >> > <configure.log> > > > >> > >> > > > >> > >> <configure.log> > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> <configure.log> > > > > > > > > <make.log><configure.log> > > > > > > > > >

1 0

Re: [petsc-dev] I am getting this error ...
by Mark Adams 05 Nov '21

05 Nov '21

Yes, thanks. I emailed the NERSc person and told him where we are and that we could fix this manually, but I don't have this issue on Summit ... He did not understand for the longest time, not sure he does now, that we do not add -gpu. And I gave him your reproducer. With any luck, with the reproducer, they can figure this out at some point. MPI is not working well for my scaling studies and my app has also given up on nvhpc, so we are good. On Fri, Nov 5, 2021 at 9:49 AM Satish Balay <balay(a)mcs.anl.gov> wrote: > For now - you could manually edit petscvariables and remove -mp=gpu from > it. > > Its primarily required to make configure happy. > > Satish > > > > On Thu, 4 Nov 2021, Barry Smith wrote: > > > > > This comes from the persistent problem with PETSc's make system using > too many flags for compiling CUDA that have not been tested by configure. > See below the -mp=gpu is provided probably from the CPPFLAGS or > CXXPPFLAGS(sp) that is improperly used by the PETSc makefiles to compile > CUDA code! > > > > > > Using CUDA compile: > /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc -o .o > -I/opt/cray/pe/mpich/8.1.10/ofi/nvidia/20.7/include -I/opt/cray/pe/libsci/ > 21.08.1.2/NVIDIA/20.7/x86_64/include -I/opt/cray/pe/pmi/6.0.14/include > -I/opt/cray/pe/dsmml/0.2.2/dsmml//include > -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include -g > -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 > -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -std=c++17 -gencode > arch=compute_80,code=sm_80 -Wno-deprecated-gpu-targets -c > --compiler-options=-I/global/homes/m/madams/petsc/include > -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include > -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include -mp=gpu > -I/global/homes/m/madams/petsc/include > -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include > -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include > > > > > > > > > On Nov 4, 2021, at 7:31 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > > > > > OK, configure is done. > > > Maybe I have too many -mp=gpu > > > > > > CUDAC > arch-perlmutter-opt-nvidia-cuda/obj/sys/classes/random/impls/curand/curand2.o > > > gcc: error: unrecognized command line option ‘-mp=gpu’; did you mean > ‘-mpku’? > > > > > > On Thu, Nov 4, 2021 at 5:51 PM Barry Smith <bsmith(a)petsc.dev <mailto: > bsmith(a)petsc.dev>> wrote: > > > > > > Need the same thing for the C++ preprocessor flag > > > > > > > > > > > >> On Nov 4, 2021, at 5:44 PM, Mark Adams <mfadams(a)lbl.gov <mailto: > mfadams(a)lbl.gov>> wrote: > > >> > > >> It gets a lot further. > > >> > > >> On Thu, Nov 4, 2021 at 5:32 PM Mark Adams <mfadams(a)lbl.gov <mailto: > mfadams(a)lbl.gov>> wrote: > > >> OK, sorry I missed the CPPFLAGS. It is running now. > > >> Thanks, > > >> > > >> On Thu, Nov 4, 2021 at 4:43 PM Satish Balay <balay(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > >> Multiple e-mail threads on the same issue (: > > >> > > >> As suggested in my earlier thread - add -mp=gpu to both CPPFLAGS and > CFLAGS [or LDFLAGS] > > >> > > >> Satish > > >> > > >> ------- > > >> Executing: cc -o /tmp/petsc-Vvs8_T/config.setCompilers/conftest -g > -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 > /tmp/petsc-Vvs8_T/config.setCompilers/conftest.o > > >> Possible ERROR while running linker: > > >> stderr: > > >> nvc-Warning-The -gpu option has no effect unless a language-specific > option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, > -cuda) > > >> > > >> > > >> On Thu, 4 Nov 2021, Mark Adams wrote: > > >> > > >> > It is CPPFLAGS. I seem to get the same behavior. > > >> > > > >> > FWIW, I did get this response from NERSc but I don't know how to > interpret > > >> > it. > > >> > > > >> > He seems to be saying that I don't need -mp=gpu for the device > compiler > > >> > (nvcc). He seems to think that I am adding -gpu. > > >> > > > >> > nvcc -- The device compiler does not need any of those flags > because it > > >> > already knows that it's being fed cuda code. The warning you're > seeing is > > >> > coming from nvc (which is the host / CPU side compiler) if you're > in the > > >> > PrgEnv-nvidia environment. You should not need to add -mp=gpu and > -cuda, > > >> > please just add the -cuda flag (to your host code) not to the > device code. > > >> > > > >> > I will try to talk with this guy again. > > >> > > > >> > Thanks, > > >> > > > >> > > > >> > On Thu, Nov 4, 2021 at 4:11 PM Barry Smith <bsmith(a)petsc.dev > <mailto:[email protected]>> wrote: > > >> > > > >> > > > > >> > > Yes, you need to use the CPPFLAGS which maybe called CPPCFLAGS > I am not > > >> > > sure > > >> > > > > >> > > > > >> > > On Nov 4, 2021, at 3:23 PM, Mark Adams <mfadams(a)lbl.gov <mailto: > mfadams(a)lbl.gov>> wrote: > > >> > > > > >> > > Ah, CCFLAGS does not seem to work. > > >> > > > > >> > > On Thu, Nov 4, 2021 at 3:07 PM Barry Smith <bsmith(a)petsc.dev > <mailto:[email protected]>> wrote: > > >> > > > > >> > >> > > >> > >> You have to pass in the flag to turn off the bitching about > -gpu to the > > >> > >> C preprocessor, not the C compiler. > > >> > >> > > >> > >> > > >> > >> stderr: > > >> > >> nvc-Warning-The -gpu option has no effect unless a > language-specific > > >> > >> option to enable GPU code generation is used (e.g.: -acc, > -mp=gpu, -stdpar, > > >> > >> -cuda) > > >> > >> Source: > > >> > >> #include "confdefs.h" > > >> > >> #include "conffix.h" > > >> > >> #include <stdlib.h> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > On Nov 4, 2021, at 2:49 PM, Mark Adams <mfadams(a)lbl.gov > <mailto:[email protected]>> wrote: > > >> > >> > > > >> > >> > on Perlmutter with nvhpc: > > >> > >> > > > >> > >> > Defined make macro "CPP" to "cc --use cpp32" > > >> > >> > Preprocessing source: > > >> > >> > #include "confdefs.h" > > >> > >> > #include "conffix.h" > > >> > >> > #include <stdlib.h> > > >> > >> > > > >> > >> > Executing: cc --use cpp32 > -I/tmp/petsc-jV9U1b/config.setCompilers > > >> > >> /tmp/petsc-jV9U1b/config.setCompilers/conftest.c > > >> > >> > Possible ERROR while running preprocessor: exit code 1 > > >> > >> > stderr: > > >> > >> > nvc-Error-Unknown switch: --use > > >> > >> > Source: > > >> > >> > <configure.log> > > >> > >> > > >> > >> <configure.log> > > >> > > > > >> > > > > >> > > > > >> > > > >> > > >> <configure.log> > > > > > > <make.log><configure.log> > > > > >

1 0

Re: [petsc-dev] [DocTip!] #2: Aiming for self-updating docs
by Patrick Sanan 05 Nov '21

05 Nov '21

Good question. We don't have any that start at specific line numbers, currently, as it is indeed too brittle - I suspect I removed some that were there at one point. The inclusions essentially all use :start-at: to specify a line to match. Ideally you'd also use :end-at: to specify where to stop, but some short snippets do instead specify a number of lines. Note that a practically handy option is :append:, as in this example where you are trying to excerpt the function PetscError(), and you can match on the PetscFunctionReturn(0) and then re-add the closing brace. .. literalinclude:: /../src/sys/error/err.c :start-at: PetscErrorCode <https://petsc.org/release/docs/manualpages/Sys/PetscErrorCode.html#PetscErr…> PetscError <https://petsc.org/release/docs/manualpages/Sys/PetscError.html#PetscError>( :end-at: PetscFunctionReturn <https://petsc.org/release/docs/manualpages/Sys/PetscFunctionReturn.html#Pet…>(0) :append: } Am Fr., 5. Nov. 2021 um 15:00 Uhr schrieb Matthew Knepley <knepley(a)gmail.com >: > Question about this. Do we have any literalincludes that are indexed by > line number? I thought I have done this, and your discussion > below makes me realize that this is fragile. There is still some fragility > with function names, but that ostensibly can be caught by the > mechanism itself. > > Thanks, > > Matt > > On Fri, Nov 5, 2021 at 4:01 AM Patrick Sanan <patrick.sanan(a)gmail.com> > wrote: > >> A big problem with documentation is that it goes out of date. This >> quickly takes something that was useful to something which is actually >> harmful. Showing the user things that don't work exactly as described is >> worse than showing them nothing. >> >> So, it's good to aspire to writing docs in as future-proof a way as >> possible, especially given the broad scope and rapid rate of change in >> PETSc. >> >> One step in this direction that we use currently is to avoid including >> explicit code blocks and output [2], but rather excerpt from the src/ tree >> for both code and the same output used for the test suite. >> >> An example of where this is done is in the classic "hand-on" tutorials >> [1]: these used to be part of an HTML page with the output inline, but now >> the source uses syntax like this: >> >> .. literalinclude:: /../src/ksp/ksp/tutorials/output/ex50_tut_3.out >> :language: none >> >> This isn't a bulletproof approach yet - the output can still be out of >> date, but good enough to pass the test suite (though that can at least be >> bulk-updated with a single command), and the input command is still >> literally included. Still, it reduces the maintenance burden of >> documentation and reduces the chance of confusing the reader with stale >> information. >> >> [1]: https://petsc.org/release/tutorials/handson/ >> [2]: >> https://petsc.org/release/developers/documentation/#sphinx-documentation-gu… >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> >

1 0

Re: [petsc-dev] [DocTip!] #2: Aiming for self-updating docs
by Matthew Knepley 05 Nov '21

05 Nov '21

Question about this. Do we have any literalincludes that are indexed by line number? I thought I have done this, and your discussion below makes me realize that this is fragile. There is still some fragility with function names, but that ostensibly can be caught by the mechanism itself. Thanks, Matt On Fri, Nov 5, 2021 at 4:01 AM Patrick Sanan <patrick.sanan(a)gmail.com> wrote: > A big problem with documentation is that it goes out of date. This quickly > takes something that was useful to something which is actually harmful. > Showing the user things that don't work exactly as described is worse than > showing them nothing. > > So, it's good to aspire to writing docs in as future-proof a way as > possible, especially given the broad scope and rapid rate of change in > PETSc. > > One step in this direction that we use currently is to avoid including > explicit code blocks and output [2], but rather excerpt from the src/ tree > for both code and the same output used for the test suite. > > An example of where this is done is in the classic "hand-on" tutorials > [1]: these used to be part of an HTML page with the output inline, but now > the source uses syntax like this: > > .. literalinclude:: /../src/ksp/ksp/tutorials/output/ex50_tut_3.out > :language: none > > This isn't a bulletproof approach yet - the output can still be out of > date, but good enough to pass the test suite (though that can at least be > bulk-updated with a single command), and the input command is still > literally included. Still, it reduces the maintenance burden of > documentation and reduces the chance of confusing the reader with stale > information. > > [1]: https://petsc.org/release/tutorials/handson/ > [2]: > https://petsc.org/release/developers/documentation/#sphinx-documentation-gu… > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] I am getting this error ...
by Satish Balay 05 Nov '21

05 Nov '21

For now - you could manually edit petscvariables and remove -mp=gpu from it. Its primarily required to make configure happy. Satish On Thu, 4 Nov 2021, Barry Smith wrote: > > This comes from the persistent problem with PETSc's make system using too many flags for compiling CUDA that have not been tested by configure. See below the -mp=gpu is provided probably from the CPPFLAGS or CXXPPFLAGS(sp) that is improperly used by the PETSc makefiles to compile CUDA code! > > > Using CUDA compile: /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc -o .o -I/opt/cray/pe/mpich/8.1.10/ofi/nvidia/20.7/include -I/opt/cray/pe/libsci/21.08.1.2/NVIDIA/20.7/x86_64/include -I/opt/cray/pe/pmi/6.0.14/include -I/opt/cray/pe/dsmml/0.2.2/dsmml//include -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include -g -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -std=c++17 -gencode arch=compute_80,code=sm_80 -Wno-deprecated-gpu-targets -c --compiler-options=-I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include -mp=gpu -I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include > > > > > On Nov 4, 2021, at 7:31 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > > > OK, configure is done. > > Maybe I have too many -mp=gpu > > > > CUDAC arch-perlmutter-opt-nvidia-cuda/obj/sys/classes/random/impls/curand/curand2.o > > gcc: error: unrecognized command line option ‘-mp=gpu’; did you mean ‘-mpku’? > > > > On Thu, Nov 4, 2021 at 5:51 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: > > > > Need the same thing for the C++ preprocessor flag > > > > > > > >> On Nov 4, 2021, at 5:44 PM, Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: > >> > >> It gets a lot further. > >> > >> On Thu, Nov 4, 2021 at 5:32 PM Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: > >> OK, sorry I missed the CPPFLAGS. It is running now. > >> Thanks, > >> > >> On Thu, Nov 4, 2021 at 4:43 PM Satish Balay <balay(a)mcs.anl.gov <mailto:[email protected]>> wrote: > >> Multiple e-mail threads on the same issue (: > >> > >> As suggested in my earlier thread - add -mp=gpu to both CPPFLAGS and CFLAGS [or LDFLAGS] > >> > >> Satish > >> > >> ------- > >> Executing: cc -o /tmp/petsc-Vvs8_T/config.setCompilers/conftest -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 /tmp/petsc-Vvs8_T/config.setCompilers/conftest.o > >> Possible ERROR while running linker: > >> stderr: > >> nvc-Warning-The -gpu option has no effect unless a language-specific option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, -cuda) > >> > >> > >> On Thu, 4 Nov 2021, Mark Adams wrote: > >> > >> > It is CPPFLAGS. I seem to get the same behavior. > >> > > >> > FWIW, I did get this response from NERSc but I don't know how to interpret > >> > it. > >> > > >> > He seems to be saying that I don't need -mp=gpu for the device compiler > >> > (nvcc). He seems to think that I am adding -gpu. > >> > > >> > nvcc -- The device compiler does not need any of those flags because it > >> > already knows that it's being fed cuda code. The warning you're seeing is > >> > coming from nvc (which is the host / CPU side compiler) if you're in the > >> > PrgEnv-nvidia environment. You should not need to add -mp=gpu and -cuda, > >> > please just add the -cuda flag (to your host code) not to the device code. > >> > > >> > I will try to talk with this guy again. > >> > > >> > Thanks, > >> > > >> > > >> > On Thu, Nov 4, 2021 at 4:11 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: > >> > > >> > > > >> > > Yes, you need to use the CPPFLAGS which maybe called CPPCFLAGS I am not > >> > > sure > >> > > > >> > > > >> > > On Nov 4, 2021, at 3:23 PM, Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: > >> > > > >> > > Ah, CCFLAGS does not seem to work. > >> > > > >> > > On Thu, Nov 4, 2021 at 3:07 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: > >> > > > >> > >> > >> > >> You have to pass in the flag to turn off the bitching about -gpu to the > >> > >> C preprocessor, not the C compiler. > >> > >> > >> > >> > >> > >> stderr: > >> > >> nvc-Warning-The -gpu option has no effect unless a language-specific > >> > >> option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, > >> > >> -cuda) > >> > >> Source: > >> > >> #include "confdefs.h" > >> > >> #include "conffix.h" > >> > >> #include <stdlib.h> > >> > >> > >> > >> > >> > >> > >> > >> > On Nov 4, 2021, at 2:49 PM, Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: > >> > >> > > >> > >> > on Perlmutter with nvhpc: > >> > >> > > >> > >> > Defined make macro "CPP" to "cc --use cpp32" > >> > >> > Preprocessing source: > >> > >> > #include "confdefs.h" > >> > >> > #include "conffix.h" > >> > >> > #include <stdlib.h> > >> > >> > > >> > >> > Executing: cc --use cpp32 -I/tmp/petsc-jV9U1b/config.setCompilers > >> > >> /tmp/petsc-jV9U1b/config.setCompilers/conftest.c > >> > >> > Possible ERROR while running preprocessor: exit code 1 > >> > >> > stderr: > >> > >> > nvc-Error-Unknown switch: --use > >> > >> > Source: > >> > >> > <configure.log> > >> > >> > >> > >> <configure.log> > >> > > > >> > > > >> > > > >> > > >> > >> <configure.log> > > > > <make.log><configure.log> > >

1 0

[DocTip!] #2: Aiming for self-updating docs
by Patrick Sanan 05 Nov '21

05 Nov '21

A big problem with documentation is that it goes out of date. This quickly takes something that was useful to something which is actually harmful. Showing the user things that don't work exactly as described is worse than showing them nothing. So, it's good to aspire to writing docs in as future-proof a way as possible, especially given the broad scope and rapid rate of change in PETSc. One step in this direction that we use currently is to avoid including explicit code blocks and output [2], but rather excerpt from the src/ tree for both code and the same output used for the test suite. An example of where this is done is in the classic "hand-on" tutorials [1]: these used to be part of an HTML page with the output inline, but now the source uses syntax like this: .. literalinclude:: /../src/ksp/ksp/tutorials/output/ex50_tut_3.out :language: none This isn't a bulletproof approach yet - the output can still be out of date, but good enough to pass the test suite (though that can at least be bulk-updated with a single command), and the input command is still literally included. Still, it reduces the maintenance burden of documentation and reduces the chance of confusing the reader with stale information. [1]: https://petsc.org/release/tutorials/handson/ [2]: https://petsc.org/release/developers/documentation/#sphinx-documentation-gu…

1 0

Re: [petsc-dev] I am getting this error ...
by Mark Adams 04 Nov '21

04 Nov '21

I see. crazy graph of compiler flags. OK, nothing critical. GNU works fine and the app that I support has given up on nvhpc also (don't know why). On Thu, Nov 4, 2021 at 9:08 PM Barry Smith <bsmith(a)petsc.dev> wrote: > > This comes from the persistent problem with PETSc's make system using > too many flags for compiling CUDA that have not been tested by configure. > See below the -mp=gpu is provided probably from the CPPFLAGS or > CXXPPFLAGS(sp) that is improperly used by the PETSc makefiles to compile > CUDA code! > > > Using CUDA compile: > /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc -o .o > -I/opt/cray/pe/mpich/8.1.10/ofi/nvidia/20.7/include -I/opt/cray/pe/libsci/ > 21.08.1.2/NVIDIA/20.7/x86_64/include -I/opt/cray/pe/pmi/6.0.14/include > -I/opt/cray/pe/dsmml/0.2.2/dsmml//include > -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include -g > -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 > -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -std=c++17 -gencode > arch=compute_80,code=sm_80 -Wno-deprecated-gpu-targets -c > --compiler-options=-I/global/homes/m/madams/petsc/include > -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include > -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include -mp=gpu > -I/global/homes/m/madams/petsc/include > -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include > -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include > > > > On Nov 4, 2021, at 7:31 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > OK, configure is done. > Maybe I have too many -mp=gpu > > CUDAC > arch-perlmutter-opt-nvidia-cuda/obj/sys/classes/random/impls/curand/curand2.o > gcc: error: unrecognized command line option ‘-mp=gpu’; did you mean > ‘-mpku’? > > On Thu, Nov 4, 2021 at 5:51 PM Barry Smith <bsmith(a)petsc.dev> wrote: > >> >> Need the same thing for the C++ preprocessor flag >> >> >> >> On Nov 4, 2021, at 5:44 PM, Mark Adams <mfadams(a)lbl.gov> wrote: >> >> It gets a lot further. >> >> On Thu, Nov 4, 2021 at 5:32 PM Mark Adams <mfadams(a)lbl.gov> wrote: >> >>> OK, sorry I missed the CPPFLAGS. It is running now. >>> Thanks, >>> >>> On Thu, Nov 4, 2021 at 4:43 PM Satish Balay <balay(a)mcs.anl.gov> wrote: >>> >>>> Multiple e-mail threads on the same issue (: >>>> >>>> As suggested in my earlier thread - add -mp=gpu to both CPPFLAGS and >>>> CFLAGS [or LDFLAGS] >>>> >>>> Satish >>>> >>>> ------- >>>> Executing: cc -o /tmp/petsc-Vvs8_T/config.setCompilers/conftest -g >>>> -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 >>>> /tmp/petsc-Vvs8_T/config.setCompilers/conftest.o >>>> Possible ERROR while running linker: >>>> stderr: >>>> nvc-Warning-The -gpu option has no effect unless a language-specific >>>> option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, >>>> -cuda) >>>> >>>> >>>> On Thu, 4 Nov 2021, Mark Adams wrote: >>>> >>>> > It is CPPFLAGS. I seem to get the same behavior. >>>> > >>>> > FWIW, I did get this response from NERSc but I don't know how to >>>> interpret >>>> > it. >>>> > >>>> > He seems to be saying that I don't need -mp=gpu for the device >>>> compiler >>>> > (nvcc). He seems to think that I am adding -gpu. >>>> > >>>> > nvcc -- The device compiler does not need any of those flags because >>>> it >>>> > already knows that it's being fed cuda code. The warning you're >>>> seeing is >>>> > coming from nvc (which is the host / CPU side compiler) if you're in >>>> the >>>> > PrgEnv-nvidia environment. You should not need to add -mp=gpu and >>>> -cuda, >>>> > please just add the -cuda flag (to your host code) not to the device >>>> code. >>>> > >>>> > I will try to talk with this guy again. >>>> > >>>> > Thanks, >>>> > >>>> > >>>> > On Thu, Nov 4, 2021 at 4:11 PM Barry Smith <bsmith(a)petsc.dev> wrote: >>>> > >>>> > > >>>> > > Yes, you need to use the CPPFLAGS which maybe called CPPCFLAGS I >>>> am not >>>> > > sure >>>> > > >>>> > > >>>> > > On Nov 4, 2021, at 3:23 PM, Mark Adams <mfadams(a)lbl.gov> wrote: >>>> > > >>>> > > Ah, CCFLAGS does not seem to work. >>>> > > >>>> > > On Thu, Nov 4, 2021 at 3:07 PM Barry Smith <bsmith(a)petsc.dev> >>>> wrote: >>>> > > >>>> > >> >>>> > >> You have to pass in the flag to turn off the bitching about -gpu >>>> to the >>>> > >> C preprocessor, not the C compiler. >>>> > >> >>>> > >> >>>> > >> stderr: >>>> > >> nvc-Warning-The -gpu option has no effect unless a >>>> language-specific >>>> > >> option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, >>>> -stdpar, >>>> > >> -cuda) >>>> > >> Source: >>>> > >> #include "confdefs.h" >>>> > >> #include "conffix.h" >>>> > >> #include <stdlib.h> >>>> > >> >>>> > >> >>>> > >> >>>> > >> > On Nov 4, 2021, at 2:49 PM, Mark Adams <mfadams(a)lbl.gov> wrote: >>>> > >> > >>>> > >> > on Perlmutter with nvhpc: >>>> > >> > >>>> > >> > Defined make macro "CPP" to "cc --use cpp32" >>>> > >> > Preprocessing source: >>>> > >> > #include "confdefs.h" >>>> > >> > #include "conffix.h" >>>> > >> > #include <stdlib.h> >>>> > >> > >>>> > >> > Executing: cc --use cpp32 >>>> -I/tmp/petsc-jV9U1b/config.setCompilers >>>> > >> /tmp/petsc-jV9U1b/config.setCompilers/conftest.c >>>> > >> > Possible ERROR while running preprocessor: exit code 1 >>>> > >> > stderr: >>>> > >> > nvc-Error-Unknown switch: --use >>>> > >> > Source: >>>> > >> > <configure.log> >>>> > >> >>>> > >> <configure.log> >>>> > > >>>> > > >>>> > > >>>> > >>>> >>>> <configure.log> >> >> >> <make.log><configure.log> > > >

1 0

Re: [petsc-dev] I am getting this error ...
by Barry Smith 04 Nov '21

04 Nov '21

This comes from the persistent problem with PETSc's make system using too many flags for compiling CUDA that have not been tested by configure. See below the -mp=gpu is provided probably from the CPPFLAGS or CXXPPFLAGS(sp) that is improperly used by the PETSc makefiles to compile CUDA code! Using CUDA compile: /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc -o .o -I/opt/cray/pe/mpich/8.1.10/ofi/nvidia/20.7/include -I/opt/cray/pe/libsci/21.08.1.2/NVIDIA/20.7/x86_64/include -I/opt/cray/pe/pmi/6.0.14/include -I/opt/cray/pe/dsmml/0.2.2/dsmml//include -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include -g -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -std=c++17 -gencode arch=compute_80,code=sm_80 -Wno-deprecated-gpu-targets -c --compiler-options=-I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include -mp=gpu -I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include > On Nov 4, 2021, at 7:31 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > OK, configure is done. > Maybe I have too many -mp=gpu > > CUDAC arch-perlmutter-opt-nvidia-cuda/obj/sys/classes/random/impls/curand/curand2.o > gcc: error: unrecognized command line option ‘-mp=gpu’; did you mean ‘-mpku’? > > On Thu, Nov 4, 2021 at 5:51 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: > > Need the same thing for the C++ preprocessor flag > > > >> On Nov 4, 2021, at 5:44 PM, Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: >> >> It gets a lot further. >> >> On Thu, Nov 4, 2021 at 5:32 PM Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: >> OK, sorry I missed the CPPFLAGS. It is running now. >> Thanks, >> >> On Thu, Nov 4, 2021 at 4:43 PM Satish Balay <balay(a)mcs.anl.gov <mailto:[email protected]>> wrote: >> Multiple e-mail threads on the same issue (: >> >> As suggested in my earlier thread - add -mp=gpu to both CPPFLAGS and CFLAGS [or LDFLAGS] >> >> Satish >> >> ------- >> Executing: cc -o /tmp/petsc-Vvs8_T/config.setCompilers/conftest -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 /tmp/petsc-Vvs8_T/config.setCompilers/conftest.o >> Possible ERROR while running linker: >> stderr: >> nvc-Warning-The -gpu option has no effect unless a language-specific option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, -cuda) >> >> >> On Thu, 4 Nov 2021, Mark Adams wrote: >> >> > It is CPPFLAGS. I seem to get the same behavior. >> > >> > FWIW, I did get this response from NERSc but I don't know how to interpret >> > it. >> > >> > He seems to be saying that I don't need -mp=gpu for the device compiler >> > (nvcc). He seems to think that I am adding -gpu. >> > >> > nvcc -- The device compiler does not need any of those flags because it >> > already knows that it's being fed cuda code. The warning you're seeing is >> > coming from nvc (which is the host / CPU side compiler) if you're in the >> > PrgEnv-nvidia environment. You should not need to add -mp=gpu and -cuda, >> > please just add the -cuda flag (to your host code) not to the device code. >> > >> > I will try to talk with this guy again. >> > >> > Thanks, >> > >> > >> > On Thu, Nov 4, 2021 at 4:11 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: >> > >> > > >> > > Yes, you need to use the CPPFLAGS which maybe called CPPCFLAGS I am not >> > > sure >> > > >> > > >> > > On Nov 4, 2021, at 3:23 PM, Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: >> > > >> > > Ah, CCFLAGS does not seem to work. >> > > >> > > On Thu, Nov 4, 2021 at 3:07 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: >> > > >> > >> >> > >> You have to pass in the flag to turn off the bitching about -gpu to the >> > >> C preprocessor, not the C compiler. >> > >> >> > >> >> > >> stderr: >> > >> nvc-Warning-The -gpu option has no effect unless a language-specific >> > >> option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, >> > >> -cuda) >> > >> Source: >> > >> #include "confdefs.h" >> > >> #include "conffix.h" >> > >> #include <stdlib.h> >> > >> >> > >> >> > >> >> > >> > On Nov 4, 2021, at 2:49 PM, Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: >> > >> > >> > >> > on Perlmutter with nvhpc: >> > >> > >> > >> > Defined make macro "CPP" to "cc --use cpp32" >> > >> > Preprocessing source: >> > >> > #include "confdefs.h" >> > >> > #include "conffix.h" >> > >> > #include <stdlib.h> >> > >> > >> > >> > Executing: cc --use cpp32 -I/tmp/petsc-jV9U1b/config.setCompilers >> > >> /tmp/petsc-jV9U1b/config.setCompilers/conftest.c >> > >> > Possible ERROR while running preprocessor: exit code 1 >> > >> > stderr: >> > >> > nvc-Error-Unknown switch: --use >> > >> > Source: >> > >> > <configure.log> >> > >> >> > >> <configure.log> >> > > >> > > >> > > >> > >> >> <configure.log> > > <make.log><configure.log>

1 0

Re: [petsc-dev] I am getting this error ...
by Mark Adams 04 Nov '21

04 Nov '21

OK, configure is done. Maybe I have too many -mp=gpu CUDAC arch-perlmutter-opt-nvidia-cuda/obj/sys/classes/random/impls/curand/curand2.o gcc: error: unrecognized command line option ‘-mp=gpu’; did you mean ‘-mpku’? On Thu, Nov 4, 2021 at 5:51 PM Barry Smith <bsmith(a)petsc.dev> wrote: > > Need the same thing for the C++ preprocessor flag > > > > On Nov 4, 2021, at 5:44 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > It gets a lot further. > > On Thu, Nov 4, 2021 at 5:32 PM Mark Adams <mfadams(a)lbl.gov> wrote: > >> OK, sorry I missed the CPPFLAGS. It is running now. >> Thanks, >> >> On Thu, Nov 4, 2021 at 4:43 PM Satish Balay <balay(a)mcs.anl.gov> wrote: >> >>> Multiple e-mail threads on the same issue (: >>> >>> As suggested in my earlier thread - add -mp=gpu to both CPPFLAGS and >>> CFLAGS [or LDFLAGS] >>> >>> Satish >>> >>> ------- >>> Executing: cc -o /tmp/petsc-Vvs8_T/config.setCompilers/conftest -g >>> -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 >>> /tmp/petsc-Vvs8_T/config.setCompilers/conftest.o >>> Possible ERROR while running linker: >>> stderr: >>> nvc-Warning-The -gpu option has no effect unless a language-specific >>> option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, >>> -cuda) >>> >>> >>> On Thu, 4 Nov 2021, Mark Adams wrote: >>> >>> > It is CPPFLAGS. I seem to get the same behavior. >>> > >>> > FWIW, I did get this response from NERSc but I don't know how to >>> interpret >>> > it. >>> > >>> > He seems to be saying that I don't need -mp=gpu for the device compiler >>> > (nvcc). He seems to think that I am adding -gpu. >>> > >>> > nvcc -- The device compiler does not need any of those flags because it >>> > already knows that it's being fed cuda code. The warning you're seeing >>> is >>> > coming from nvc (which is the host / CPU side compiler) if you're in >>> the >>> > PrgEnv-nvidia environment. You should not need to add -mp=gpu and >>> -cuda, >>> > please just add the -cuda flag (to your host code) not to the device >>> code. >>> > >>> > I will try to talk with this guy again. >>> > >>> > Thanks, >>> > >>> > >>> > On Thu, Nov 4, 2021 at 4:11 PM Barry Smith <bsmith(a)petsc.dev> wrote: >>> > >>> > > >>> > > Yes, you need to use the CPPFLAGS which maybe called CPPCFLAGS I >>> am not >>> > > sure >>> > > >>> > > >>> > > On Nov 4, 2021, at 3:23 PM, Mark Adams <mfadams(a)lbl.gov> wrote: >>> > > >>> > > Ah, CCFLAGS does not seem to work. >>> > > >>> > > On Thu, Nov 4, 2021 at 3:07 PM Barry Smith <bsmith(a)petsc.dev> wrote: >>> > > >>> > >> >>> > >> You have to pass in the flag to turn off the bitching about -gpu >>> to the >>> > >> C preprocessor, not the C compiler. >>> > >> >>> > >> >>> > >> stderr: >>> > >> nvc-Warning-The -gpu option has no effect unless a language-specific >>> > >> option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, >>> -stdpar, >>> > >> -cuda) >>> > >> Source: >>> > >> #include "confdefs.h" >>> > >> #include "conffix.h" >>> > >> #include <stdlib.h> >>> > >> >>> > >> >>> > >> >>> > >> > On Nov 4, 2021, at 2:49 PM, Mark Adams <mfadams(a)lbl.gov> wrote: >>> > >> > >>> > >> > on Perlmutter with nvhpc: >>> > >> > >>> > >> > Defined make macro "CPP" to "cc --use cpp32" >>> > >> > Preprocessing source: >>> > >> > #include "confdefs.h" >>> > >> > #include "conffix.h" >>> > >> > #include <stdlib.h> >>> > >> > >>> > >> > Executing: cc --use cpp32 -I/tmp/petsc-jV9U1b/config.setCompilers >>> > >> /tmp/petsc-jV9U1b/config.setCompilers/conftest.c >>> > >> > Possible ERROR while running preprocessor: exit code 1 >>> > >> > stderr: >>> > >> > nvc-Error-Unknown switch: --use >>> > >> > Source: >>> > >> > <configure.log> >>> > >> >>> > >> <configure.log> >>> > > >>> > > >>> > > >>> > >>> >>> <configure.log> > > >

1 0

Re: [petsc-dev] I am getting this error ...
by Barry Smith 04 Nov '21

04 Nov '21

Need the same thing for the C++ preprocessor flag > On Nov 4, 2021, at 5:44 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > It gets a lot further. > > On Thu, Nov 4, 2021 at 5:32 PM Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: > OK, sorry I missed the CPPFLAGS. It is running now. > Thanks, > > On Thu, Nov 4, 2021 at 4:43 PM Satish Balay <balay(a)mcs.anl.gov <mailto:[email protected]>> wrote: > Multiple e-mail threads on the same issue (: > > As suggested in my earlier thread - add -mp=gpu to both CPPFLAGS and CFLAGS [or LDFLAGS] > > Satish > > ------- > Executing: cc -o /tmp/petsc-Vvs8_T/config.setCompilers/conftest -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 /tmp/petsc-Vvs8_T/config.setCompilers/conftest.o > Possible ERROR while running linker: > stderr: > nvc-Warning-The -gpu option has no effect unless a language-specific option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, -cuda) > > > On Thu, 4 Nov 2021, Mark Adams wrote: > > > It is CPPFLAGS. I seem to get the same behavior. > > > > FWIW, I did get this response from NERSc but I don't know how to interpret > > it. > > > > He seems to be saying that I don't need -mp=gpu for the device compiler > > (nvcc). He seems to think that I am adding -gpu. > > > > nvcc -- The device compiler does not need any of those flags because it > > already knows that it's being fed cuda code. The warning you're seeing is > > coming from nvc (which is the host / CPU side compiler) if you're in the > > PrgEnv-nvidia environment. You should not need to add -mp=gpu and -cuda, > > please just add the -cuda flag (to your host code) not to the device code. > > > > I will try to talk with this guy again. > > > > Thanks, > > > > > > On Thu, Nov 4, 2021 at 4:11 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: > > > > > > > > Yes, you need to use the CPPFLAGS which maybe called CPPCFLAGS I am not > > > sure > > > > > > > > > On Nov 4, 2021, at 3:23 PM, Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: > > > > > > Ah, CCFLAGS does not seem to work. > > > > > > On Thu, Nov 4, 2021 at 3:07 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: > > > > > >> > > >> You have to pass in the flag to turn off the bitching about -gpu to the > > >> C preprocessor, not the C compiler. > > >> > > >> > > >> stderr: > > >> nvc-Warning-The -gpu option has no effect unless a language-specific > > >> option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, > > >> -cuda) > > >> Source: > > >> #include "confdefs.h" > > >> #include "conffix.h" > > >> #include <stdlib.h> > > >> > > >> > > >> > > >> > On Nov 4, 2021, at 2:49 PM, Mark Adams <mfadams(a)lbl.gov <mailto:[email protected]>> wrote: > > >> > > > >> > on Perlmutter with nvhpc: > > >> > > > >> > Defined make macro "CPP" to "cc --use cpp32" > > >> > Preprocessing source: > > >> > #include "confdefs.h" > > >> > #include "conffix.h" > > >> > #include <stdlib.h> > > >> > > > >> > Executing: cc --use cpp32 -I/tmp/petsc-jV9U1b/config.setCompilers > > >> /tmp/petsc-jV9U1b/config.setCompilers/conftest.c > > >> > Possible ERROR while running preprocessor: exit code 1 > > >> > stderr: > > >> > nvc-Error-Unknown switch: --use > > >> > Source: > > >> > <configure.log> > > >> > > >> <configure.log> > > > > > > > > > > > > > <configure.log>

1 0