November 2021 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] PTScotch problem on Mac
by Matthew Knepley 23 Nov '21

23 Nov '21

On Tue, Nov 23, 2021 at 11:17 AM Barry Smith <bsmith(a)petsc.dev> wrote: > > I would normally say make a pull request but they have their own git > server that requires you ask them for an account so I suggest posting an > issue with your patch. > Also need an account to create an issue :) Matt > Barry > > > On Nov 23, 2021, at 10:55 AM, Matthew Knepley <knepley(a)gmail.com> wrote: > > This is the same flex problem as I had for c2html, but I was more > determined tracking it down this time. The first problem is that we were > not renaming in the parser, > > main *$:/PETSc3/petsc/petsc-dev$ git diff > diff --git a/config/BuildSystem/config/packages/PTScotch.py > b/config/BuildSystem/config/packages/PTScotch.py > index d1c277b6e9f..e046804c17f 100644 > --- a/config/BuildSystem/config/packages/PTScotch.py > +++ b/config/BuildSystem/config/packages/PTScotch.py > @@ -70,7 +70,7 @@ class Configure(config.package.Package): > if self.libraries.add('-lrt','timer_create'): ldflags += ' -lrt' > self.cflags = self.cflags + ' -DCOMMON_RANDOM_FIXED_SEED' > # do not use -DSCOTCH_PTHREAD because requires MPI built for threads. > - self.cflags = self.cflags + ' -DSCOTCH_RENAME > -Drestrict="'+self.compilers.cRestrict+'"' > + self.cflags = self.cflags + ' -DSCOTCH_RENAME -DSCOTCH_RENAME_PARSER > -Drestrict="'+self.compilers.cRestrict+'"' > # this is needed on the Mac, because common2.c includes common.h > which DOES NOT include mpi.h because > # SCOTCH_PTSCOTCH is NOT defined above Mac does not know what > clock_gettime() is! > if self.setCompilers.isDarwin(self.log): > > Second, they were not treating this case completely correctly: > > (93454e8...):/PETSc3/petsc/petsc-dev/arch-master-debug/externalpackages/git.ptscotch/src/libscotch$ > git diff HEAD~1 > diff --git a/src/libscotch/parser_yy.h b/src/libscotch/parser_yy.h > index 931315d..95b8160 100644 > --- a/src/libscotch/parser_yy.h > +++ b/src/libscotch/parser_yy.h > @@ -62,6 +62,9 @@ > > #if ((defined SCOTCH_RENAME_PARSER) || (defined yylex)) /* If prefix > renaming */ > #define scotchyyparse stratParserParse2 /* Parser function > name */ > +#if !defined(yylex) > +#define yylex scotchyylex > +#endif > #ifndef yylval > #define yylval SCOTCH_NAME_MACRO3 (scotchyy, > SCOTCH_NAME_SUFFIXC, lval) /* It should be Yacc/Bison's job to redefine it! > */ > #endif /* yylval */ > > How should we go about getting this fix in? Do you need to have our own > branch of PTScotch? > > Thanks, > > Matt > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] PTScotch problem on Mac
by Barry Smith 23 Nov '21

23 Nov '21

I would normally say make a pull request but they have their own git server that requires you ask them for an account so I suggest posting an issue with your patch. Barry > On Nov 23, 2021, at 10:55 AM, Matthew Knepley <knepley(a)gmail.com> wrote: > > This is the same flex problem as I had for c2html, but I was more determined tracking it down this time. The first problem is that we were not renaming in the parser, > > main *$:/PETSc3/petsc/petsc-dev$ git diff > diff --git a/config/BuildSystem/config/packages/PTScotch.py b/config/BuildSystem/config/packages/PTScotch.py > index d1c277b6e9f..e046804c17f 100644 > --- a/config/BuildSystem/config/packages/PTScotch.py > +++ b/config/BuildSystem/config/packages/PTScotch.py > @@ -70,7 +70,7 @@ class Configure(config.package.Package): > if self.libraries.add('-lrt','timer_create'): ldflags += ' -lrt' > self.cflags = self.cflags + ' -DCOMMON_RANDOM_FIXED_SEED' > # do not use -DSCOTCH_PTHREAD because requires MPI built for threads. > - self.cflags = self.cflags + ' -DSCOTCH_RENAME -Drestrict="'+self.compilers.cRestrict+'"' > + self.cflags = self.cflags + ' -DSCOTCH_RENAME -DSCOTCH_RENAME_PARSER -Drestrict="'+self.compilers.cRestrict+'"' > # this is needed on the Mac, because common2.c includes common.h which DOES NOT include mpi.h because > # SCOTCH_PTSCOTCH is NOT defined above Mac does not know what clock_gettime() is! > if self.setCompilers.isDarwin(self.log): > > Second, they were not treating this case completely correctly: > > (93454e8...):/PETSc3/petsc/petsc-dev/arch-master-debug/externalpackages/git.ptscotch/src/libscotch$ git diff HEAD~1 > diff --git a/src/libscotch/parser_yy.h b/src/libscotch/parser_yy.h > index 931315d..95b8160 100644 > --- a/src/libscotch/parser_yy.h > +++ b/src/libscotch/parser_yy.h > @@ -62,6 +62,9 @@ > > #if ((defined SCOTCH_RENAME_PARSER) || (defined yylex)) /* If prefix renaming */ > #define scotchyyparse stratParserParse2 /* Parser function name */ > +#if !defined(yylex) > +#define yylex scotchyylex > +#endif > #ifndef yylval > #define yylval SCOTCH_NAME_MACRO3 (scotchyy, SCOTCH_NAME_SUFFIXC, lval) /* It should be Yacc/Bison's job to redefine it! > */ > #endif /* yylval */ > > How should we go about getting this fix in? Do you need to have our own branch of PTScotch? > > Thanks, > > Matt > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

PTScotch problem on Mac
by Matthew Knepley 23 Nov '21

23 Nov '21

This is the same flex problem as I had for c2html, but I was more determined tracking it down this time. The first problem is that we were not renaming in the parser, main *$:/PETSc3/petsc/petsc-dev$ git diff diff --git a/config/BuildSystem/config/packages/PTScotch.py b/config/BuildSystem/config/packages/PTScotch.py index d1c277b6e9f..e046804c17f 100644 --- a/config/BuildSystem/config/packages/PTScotch.py +++ b/config/BuildSystem/config/packages/PTScotch.py @@ -70,7 +70,7 @@ class Configure(config.package.Package): if self.libraries.add('-lrt','timer_create'): ldflags += ' -lrt' self.cflags = self.cflags + ' -DCOMMON_RANDOM_FIXED_SEED' # do not use -DSCOTCH_PTHREAD because requires MPI built for threads. - self.cflags = self.cflags + ' -DSCOTCH_RENAME -Drestrict="'+self.compilers.cRestrict+'"' + self.cflags = self.cflags + ' -DSCOTCH_RENAME -DSCOTCH_RENAME_PARSER -Drestrict="'+self.compilers.cRestrict+'"' # this is needed on the Mac, because common2.c includes common.h which DOES NOT include mpi.h because # SCOTCH_PTSCOTCH is NOT defined above Mac does not know what clock_gettime() is! if self.setCompilers.isDarwin(self.log): Second, they were not treating this case completely correctly: (93454e8...):/PETSc3/petsc/petsc-dev/arch-master-debug/externalpackages/git.ptscotch/src/libscotch$ git diff HEAD~1 diff --git a/src/libscotch/parser_yy.h b/src/libscotch/parser_yy.h index 931315d..95b8160 100644 --- a/src/libscotch/parser_yy.h +++ b/src/libscotch/parser_yy.h @@ -62,6 +62,9 @@ #if ((defined SCOTCH_RENAME_PARSER) || (defined yylex)) /* If prefix renaming */ #define scotchyyparse stratParserParse2 /* Parser function name */ +#if !defined(yylex) +#define yylex scotchyylex +#endif #ifndef yylval #define yylval SCOTCH_NAME_MACRO3 (scotchyy, SCOTCH_NAME_SUFFIXC, lval) /* It should be Yacc/Bison's job to redefine it! */ #endif /* yylval */ How should we go about getting this fix in? Do you need to have our own branch of PTScotch? Thanks, Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] DMPlexCreateNaturalVector
by Matthew Knepley 22 Nov '21

22 Nov '21

On Mon, Nov 22, 2021 at 7:24 PM Matthew Knepley <knepley(a)gmail.com> wrote: > On Mon, Nov 22, 2021 at 7:21 PM Alexis Marboeuf < > alexis.marboeuf(a)hotmail.fr> wrote: > >> Dear PETSc Team, >> >> I am writing a DMPlexCreateNaturalVector routine in >> src/dm/impls/plex/plexnatural.c. What is the way to properly interface the >> call such as for DMCreateGlobalVector? In particular, do you want to unify >> the call with its DMDA counterpart DMDACreateNaturalVector of >> src/dm/impls/da/dadist.c? >> > > This exists currently. However, we are debating how it should behave. > Currently the natural vector has a global order that matches the > original global order, but has the current partition. It could be that we > want a vector that matches the original partition, so we are discussing > this online. > I forgot to point you to the discussion: https://gitlab.com/petsc/petsc/-/merge_requests/4547 Thanks, Matt > Thanks, > > Matt > > >> Thanks, >> Alexis >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] DMPlexCreateNaturalVector
by Matthew Knepley 22 Nov '21

22 Nov '21

On Mon, Nov 22, 2021 at 7:21 PM Alexis Marboeuf <alexis.marboeuf(a)hotmail.fr> wrote: > Dear PETSc Team, > > I am writing a DMPlexCreateNaturalVector routine in > src/dm/impls/plex/plexnatural.c. What is the way to properly interface the > call such as for DMCreateGlobalVector? In particular, do you want to unify > the call with its DMDA counterpart DMDACreateNaturalVector of > src/dm/impls/da/dadist.c? > This exists currently. However, we are debating how it should behave. Currently the natural vector has a global order that matches the original global order, but has the current partition. It could be that we want a vector that matches the original partition, so we are discussing this online. Thanks, Matt > Thanks, > Alexis > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

DMPlexCreateNaturalVector
by Alexis Marboeuf 22 Nov '21

22 Nov '21

Dear PETSc Team, I am writing a DMPlexCreateNaturalVector routine in src/dm/impls/plex/plexnatural.c. What is the way to properly interface the call such as for DMCreateGlobalVector? In particular, do you want to unify the call with its DMDA counterpart DMDACreateNaturalVector of src/dm/impls/da/dadist.c? Thanks, Alexis

1 0

Re: [petsc-dev] kokkos error
by Junchao Zhang 21 Nov '21

21 Nov '21

You can add -v to the offending command line to see what happened, i.e., how nvcc_wrapper passed options to g++. --Junchao Zhang On Sun, Nov 21, 2021 at 12:05 PM Mark Adams <mfadams(a)lbl.gov> wrote: > Any idea what is going wrong with this? > > Using PETSC_DIR=/global/homes/m/madams/petsc and > PETSC_ARCH=arch-perlmutter-opt-gcc-kokkos-cuda > C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process > C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes > C/C++ example src/snes/tutorials/ex19 run successfully with cuda > gmake[3]: [/global/homes/m/madams/petsc/lib/petsc/conf/rules:301: > ex3k.PETSc] Error 2 (ignored) > *******************Error detected during compile or > link!******************* > See http://www.mcs.anl.gov/petsc/documentation/faq.html > /global/homes/m/madams/petsc/src/snes/tutorials ex3k > > ********************************************************************************* > > PATH=/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib:/global/common/software/nersc/shasta2105/arm-forge/21.0.2-linux-x86_64/bin:/global/common/software/nersc/cos1.3/cuda/11.3.0/bin:/opt/cray/pe/perftools/21.10.0/bin:/opt/cray/pe/papi/ > 6.0.0.10/bin:/opt/cray/pe/gcc/9.3.0/bin:/opt/cray/pe/craype/2.7.11/bin:/global/common/software/nersc/shasta2105/python/3.8-anaconda-2021.05/bin:/global/homes/m/madams/.local/perlmutter/3.8-anaconda-2021.05/bin:/global/common/software/nersc/shasta2105/cmake/git-master/bin:/global/common/software/nersc/bin:/opt/cray/libfabric/1.11.0.4.79/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/opt/cray/pe/bin:`dirname > <http://6.0.0.10/bin:/opt/cray/pe/gcc/9.3.0/bin:/opt/cray/pe/craype/2.7.11/b…> > /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc` > NVCC_WRAPPER_DEFAULT_COMPILER=CC > /global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/bin/nvcc_wrapper > --expt-extended-lambda -g -Xcompiler -rdynamic -DLANDAU_DIM=2 > -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -ccbin CC > -std=c++17 -gencode arch=compute_80,code=sm_80 -Wno-deprecated-gpu-targets > -I/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/include -I/opt/cray/pe/libsci/ > 21.08.1.2/GNU/9.1/x86_64/include -I/opt/cray/pe/pmi/6.0.14/include > -I/opt/cray/pe/dsmml/0.2.2/dsmml//include > -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include > -I/global/homes/m/madams/petsc/include > -I/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/include > -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include -g > -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -fPIC -O3 -g > -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -O3 > -L/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib > -L/opt/cray/pe/mpich/8.1.10/gtl/lib -L/opt/cray/pe/libsci/ > 21.08.1.2/GNU/9.1/x86_64/lib -L/opt/cray/pe/pmi/6.0.14/lib > -L/opt/cray/pe/dsmml/0.2.2/dsmml//lib > -L/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 > -Wl,--as-needed,-lmpi_gnu_91,--no-as-needed > -Wl,--as-needed,-lsci_gnu_82_mpi,--no-as-needed > -Wl,--as-needed,-lsci_gnu_82,--no-as-needed > -Wl,--as-needed,-ldsmml,--no-as-needed -lpmi -lpmi2 -lmpi_gtl_cuda -ldl > -lxpmem ex3k.kokkos.cxx > -Wl,-rpath,/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib > -L/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib > -Wl,-rpath,/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib > -L/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib > -Wl,-rpath,/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64 > -L/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64 > -L/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64/stubs > -Wl,-rpath,/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib > -L/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib > -Wl,-rpath,/opt/cray/pe/mpich/8.1.10/gtl/lib > -L/opt/cray/pe/mpich/8.1.10/gtl/lib -Wl,-rpath,/opt/cray/pe/libsci/ > 21.08.1.2/GNU/9.1/x86_64/lib -L/opt/cray/pe/libsci/ > 21.08.1.2/GNU/9.1/x86_64/lib -Wl,-rpath,/opt/cray/pe/pmi/6.0.14/lib > -L/opt/cray/pe/pmi/6.0.14/lib -Wl,-rpath,/opt/cray/pe/dsmml/0.2.2/dsmml/lib > -L/opt/cray/pe/dsmml/0.2.2/dsmml/lib > -Wl,-rpath,/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 > -L/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 > -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib/gcc/x86_64-suse-linux/9.3.0 > -L/opt/cray/pe/gcc/9.3.0/snos/lib/gcc/x86_64-suse-linux/9.3.0 > -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib64 > -L/opt/cray/pe/gcc/9.3.0/snos/lib64 > -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib > -L/opt/cray/pe/gcc/9.3.0/snos/lib -lpetsc -lkokkoskernels > -lkokkoscontainers -lkokkoscore -lp4est -lsc -lparmetis -lmetis -lz > -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -lcuda -lX11 > -lstdc++ -ldl -lpmi -lpmi2 -lmpi_gtl_cuda -lxpmem -lgfortran -lm > -lmpifort_gnu_91 -lmpi_gnu_91 -lsci_gnu_82_mpi -lsci_gnu_82 -ldsmml > -lgfortran -lquadmath -lpthread -lm -lgcc_s -lquadmath -lstdc++ -ldl -o ex3k > nvcc_wrapper - *warning* you have set multiple optimization flags (-O*), > only the last is used because nvcc can only accept a single optimization > setting. > > *g++: error: unrecognized command line option '-rpath'*gmake[4]: *** > [/global/homes/m/madams/petsc/lib/petsc/conf/test:32: ex3k] Error 1 > gmake[3]: [makefile:115: runex3k_kokkos] Error 2 (ignored) > 1,25c1,4 > < atol=1e-50, rtol=1e-08, stol=1e-08, maxit=50, maxf=10000 > < Vec Object: Exact Solution 2 MPI processes > < type: mpikokkos >

1 0

kokkos error
by Mark Adams 21 Nov '21

21 Nov '21

Any idea what is going wrong with this? Using PETSC_DIR=/global/homes/m/madams/petsc and PETSC_ARCH=arch-perlmutter-opt-gcc-kokkos-cuda C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes C/C++ example src/snes/tutorials/ex19 run successfully with cuda gmake[3]: [/global/homes/m/madams/petsc/lib/petsc/conf/rules:301: ex3k.PETSc] Error 2 (ignored) *******************Error detected during compile or link!******************* See http://www.mcs.anl.gov/petsc/documentation/faq.html /global/homes/m/madams/petsc/src/snes/tutorials ex3k ********************************************************************************* PATH=/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib:/global/common/software/nersc/shasta2105/arm-forge/21.0.2-linux-x86_64/bin:/global/common/software/nersc/cos1.3/cuda/11.3.0/bin:/opt/cray/pe/perftools/21.10.0/bin:/opt/cray/pe/papi/ 6.0.0.10/bin:/opt/cray/pe/gcc/9.3.0/bin:/opt/cray/pe/craype/2.7.11/bin:/global/common/software/nersc/shasta2105/python/3.8-anaconda-2021.05/bin:/global/homes/m/madams/.local/perlmutter/3.8-anaconda-2021.05/bin:/global/common/software/nersc/shasta2105/cmake/git-master/bin:/global/common/software/nersc/bin:/opt/cray/libfabric/1.11.0.4.79/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/opt/cray/pe/bin:`dirname /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=CC /global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/bin/nvcc_wrapper --expt-extended-lambda -g -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -ccbin CC -std=c++17 -gencode arch=compute_80,code=sm_80 -Wno-deprecated-gpu-targets -I/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/include -I/opt/cray/pe/libsci/ 21.08.1.2/GNU/9.1/x86_64/include -I/opt/cray/pe/pmi/6.0.14/include -I/opt/cray/pe/dsmml/0.2.2/dsmml//include -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include -I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/include -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -fPIC -O3 -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -O3 -L/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib -L/opt/cray/pe/mpich/8.1.10/gtl/lib -L/opt/cray/pe/libsci/ 21.08.1.2/GNU/9.1/x86_64/lib -L/opt/cray/pe/pmi/6.0.14/lib -L/opt/cray/pe/dsmml/0.2.2/dsmml//lib -L/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 -Wl,--as-needed,-lmpi_gnu_91,--no-as-needed -Wl,--as-needed,-lsci_gnu_82_mpi,--no-as-needed -Wl,--as-needed,-lsci_gnu_82,--no-as-needed -Wl,--as-needed,-ldsmml,--no-as-needed -lpmi -lpmi2 -lmpi_gtl_cuda -ldl -lxpmem ex3k.kokkos.cxx -Wl,-rpath,/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib -L/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib -Wl,-rpath,/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib -L/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib -Wl,-rpath,/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64 -L/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64 -L/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64/stubs -Wl,-rpath,/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib -L/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib -Wl,-rpath,/opt/cray/pe/mpich/8.1.10/gtl/lib -L/opt/cray/pe/mpich/8.1.10/gtl/lib -Wl,-rpath,/opt/cray/pe/libsci/ 21.08.1.2/GNU/9.1/x86_64/lib -L/opt/cray/pe/libsci/ 21.08.1.2/GNU/9.1/x86_64/lib -Wl,-rpath,/opt/cray/pe/pmi/6.0.14/lib -L/opt/cray/pe/pmi/6.0.14/lib -Wl,-rpath,/opt/cray/pe/dsmml/0.2.2/dsmml/lib -L/opt/cray/pe/dsmml/0.2.2/dsmml/lib -Wl,-rpath,/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 -L/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib/gcc/x86_64-suse-linux/9.3.0 -L/opt/cray/pe/gcc/9.3.0/snos/lib/gcc/x86_64-suse-linux/9.3.0 -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib64 -L/opt/cray/pe/gcc/9.3.0/snos/lib64 -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib -L/opt/cray/pe/gcc/9.3.0/snos/lib -lpetsc -lkokkoskernels -lkokkoscontainers -lkokkoscore -lp4est -lsc -lparmetis -lmetis -lz -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -lcuda -lX11 -lstdc++ -ldl -lpmi -lpmi2 -lmpi_gtl_cuda -lxpmem -lgfortran -lm -lmpifort_gnu_91 -lmpi_gnu_91 -lsci_gnu_82_mpi -lsci_gnu_82 -ldsmml -lgfortran -lquadmath -lpthread -lm -lgcc_s -lquadmath -lstdc++ -ldl -o ex3k nvcc_wrapper - *warning* you have set multiple optimization flags (-O*), only the last is used because nvcc can only accept a single optimization setting. *g++: error: unrecognized command line option '-rpath'*gmake[4]: *** [/global/homes/m/madams/petsc/lib/petsc/conf/test:32: ex3k] Error 1 gmake[3]: [makefile:115: runex3k_kokkos] Error 2 (ignored) 1,25c1,4 < atol=1e-50, rtol=1e-08, stol=1e-08, maxit=50, maxf=10000 < Vec Object: Exact Solution 2 MPI processes < type: mpikokkos

1 0

Re: [petsc-dev] testset
by Mark Adams 20 Nov '21

20 Nov '21

Yep, misaligned indentation. On Sat, Nov 20, 2021 at 11:52 AM Matthew Knepley <knepley(a)gmail.com> wrote: > On Sat, Nov 20, 2021 at 9:14 AM Mark Adams <mfadams(a)lbl.gov> wrote: > >> I am trying to add a test set and I am getting this error with the >> appended syntax. >> I do not see the problem. Any ideas? >> > > Maybe not enough indenting for "test:" ? It is clear that one of those > lines is the problem. > > Thanks, > > Matt > > >> Thanks, >> Mark >> >> 06:03 2 nid003756 main *= perlmutter:~/petsc$ *make >> PETSC_DIR=/global/homes/m/madams/petsc PETSC_ARCH=arch-perlmutter-opt-gcc >> -f gmakefile test search='ts_utils_dmplexlandau_tutorials-ex2_0'* >> /usr/bin/python /global/homes/m/madams/petsc/config/gmakegentest.py >> --petsc-dir=/global/homes/m/madams/petsc >> --petsc-arch=arch-perlmutter-opt-gcc >> --testdir=./arch-perlmutter-opt-gcc/tests >> Traceback (most recent call last): >> File "/global/homes/m/madams/petsc/config/gmakegentest.py", line 1093, >> in <module> >> testdir=opts.testdir, check=opts.check_output) >> File "/global/homes/m/madams/petsc/config/gmakegentest.py", line 1060, >> in main >> dataDict=pEx.walktree(os.path.join(pEx.srcdir)) >> File "/global/homes/m/madams/petsc/config/gmakegentest.py", line 924, >> in walktree >> self.genPetscTests(root,dirs,files,dataDict) >> File "/global/homes/m/madams/petsc/config/gmakegentest.py", line 897, >> in genPetscTests >> dataDict[root].update(testparse.parseTestFile(fullex,0)) >> File "/global/homes/m/madams/petsc/config/testparse.py", line 542, in >> parseTestFile >> testDict[basename]=parseTests(testString,srcfile,fileNums,verbosity) >> File "/global/homes/m/madams/petsc/config/testparse.py", line 498, in >> parseTests >> testnames,subdicts=parseTest(test,srcfile,verbosity) >> File "/global/homes/m/madams/petsc/config/testparse.py", line 426, in >> parseTest >> if not var in acceptedkeys: raise Exception("Keyword: "+var+" from: >> "+line+" is not valid") >> Exception: Keyword: est from: est: is not valid >> make: *** [gmakefile.test:97: arch-perlmutter-opt-gcc/tests/testfiles] >> Error 1 >> 06:08 2 nid003756 main *= perlmutter:~/petsc$ >> >> /*TEST >> >> testset: >> requires: p4est !complex double >> output_file: output/ex2_0.out >> args: -dm_landau_num_species_grid 1,1 -dm_landau_Ez 0 >> -petscspace_degree 3 -petscspace_poly_tensor 1 -dm_landau_type p4est -info >> :dm,tsadapt -dm_landau_ion_masses 2 -dm_landau_ion_charges 1 >> -dm_landau_thermal_temps 5,5 -dm_landau_n 2,2 -dm_landau_n_0 5e19 >> -ts_monitor -snes_rtol 1.e-10 -snes_stol 1.e-14 -snes_monitor >> -snes_converged_reason -snes_max_it 10 -ts_type arkimex -ts_arkimex_type >> 1bee -ts_max_snes_failures -1 -ts_rtol 1e-3 -ts_dt 1.e-1 -ts_max_time 1 >> -ts_adapt_clip .5,1.25 -ts_max_steps 2 -ts_adapt_scale_solve_failed 0.75 >> -ts_adapt_time_step_increase_delay 5 -pc_type lu -ksp_type preonly >> -dm_landau_amr_levels_max 2,2 -ex2_impurity_source_type pulse >> -ex2_pulse_start_time 1e-1 -ex2_pulse_width_time 10 -ex2_pulse_rate 1e-2 >> -ex2_t_cold .05 -ex2_plot_dt 1e-1 -dm_refine 1 -dm_landau_gpu_assembly true >> -dm_landau_batch_size 2 >> test: >> suffix: 0 >> requires: !kokkos_kernels !cuda >> args: -dm_landau_device_type cpu >> test: >> suffix: kokkos >> requires: kokkos_kernels >> args: -dm_landau_device_type kokkos -dm_mat_type aijkokkos >> -dm_vec_type kokkos >> test: >> suffix: cuda >> requires: cuda >> args: -dm_landau_device_type cuda -dm_mat_type aijcusparse >> -dm_vec_type cuda -mat_cusparse_use_cpu_solve >> >> TEST*/ >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> >

1 0

Re: [petsc-dev] testset
by Matthew Knepley 20 Nov '21

20 Nov '21

On Sat, Nov 20, 2021 at 9:14 AM Mark Adams <mfadams(a)lbl.gov> wrote: > I am trying to add a test set and I am getting this error with the > appended syntax. > I do not see the problem. Any ideas? > Maybe not enough indenting for "test:" ? It is clear that one of those lines is the problem. Thanks, Matt > Thanks, > Mark > > 06:03 2 nid003756 main *= perlmutter:~/petsc$ *make > PETSC_DIR=/global/homes/m/madams/petsc PETSC_ARCH=arch-perlmutter-opt-gcc > -f gmakefile test search='ts_utils_dmplexlandau_tutorials-ex2_0'* > /usr/bin/python /global/homes/m/madams/petsc/config/gmakegentest.py > --petsc-dir=/global/homes/m/madams/petsc > --petsc-arch=arch-perlmutter-opt-gcc > --testdir=./arch-perlmutter-opt-gcc/tests > Traceback (most recent call last): > File "/global/homes/m/madams/petsc/config/gmakegentest.py", line 1093, > in <module> > testdir=opts.testdir, check=opts.check_output) > File "/global/homes/m/madams/petsc/config/gmakegentest.py", line 1060, > in main > dataDict=pEx.walktree(os.path.join(pEx.srcdir)) > File "/global/homes/m/madams/petsc/config/gmakegentest.py", line 924, in > walktree > self.genPetscTests(root,dirs,files,dataDict) > File "/global/homes/m/madams/petsc/config/gmakegentest.py", line 897, in > genPetscTests > dataDict[root].update(testparse.parseTestFile(fullex,0)) > File "/global/homes/m/madams/petsc/config/testparse.py", line 542, in > parseTestFile > testDict[basename]=parseTests(testString,srcfile,fileNums,verbosity) > File "/global/homes/m/madams/petsc/config/testparse.py", line 498, in > parseTests > testnames,subdicts=parseTest(test,srcfile,verbosity) > File "/global/homes/m/madams/petsc/config/testparse.py", line 426, in > parseTest > if not var in acceptedkeys: raise Exception("Keyword: "+var+" from: > "+line+" is not valid") > Exception: Keyword: est from: est: is not valid > make: *** [gmakefile.test:97: arch-perlmutter-opt-gcc/tests/testfiles] > Error 1 > 06:08 2 nid003756 main *= perlmutter:~/petsc$ > > /*TEST > > testset: > requires: p4est !complex double > output_file: output/ex2_0.out > args: -dm_landau_num_species_grid 1,1 -dm_landau_Ez 0 > -petscspace_degree 3 -petscspace_poly_tensor 1 -dm_landau_type p4est -info > :dm,tsadapt -dm_landau_ion_masses 2 -dm_landau_ion_charges 1 > -dm_landau_thermal_temps 5,5 -dm_landau_n 2,2 -dm_landau_n_0 5e19 > -ts_monitor -snes_rtol 1.e-10 -snes_stol 1.e-14 -snes_monitor > -snes_converged_reason -snes_max_it 10 -ts_type arkimex -ts_arkimex_type > 1bee -ts_max_snes_failures -1 -ts_rtol 1e-3 -ts_dt 1.e-1 -ts_max_time 1 > -ts_adapt_clip .5,1.25 -ts_max_steps 2 -ts_adapt_scale_solve_failed 0.75 > -ts_adapt_time_step_increase_delay 5 -pc_type lu -ksp_type preonly > -dm_landau_amr_levels_max 2,2 -ex2_impurity_source_type pulse > -ex2_pulse_start_time 1e-1 -ex2_pulse_width_time 10 -ex2_pulse_rate 1e-2 > -ex2_t_cold .05 -ex2_plot_dt 1e-1 -dm_refine 1 -dm_landau_gpu_assembly true > -dm_landau_batch_size 2 > test: > suffix: 0 > requires: !kokkos_kernels !cuda > args: -dm_landau_device_type cpu > test: > suffix: kokkos > requires: kokkos_kernels > args: -dm_landau_device_type kokkos -dm_mat_type aijkokkos > -dm_vec_type kokkos > test: > suffix: cuda > requires: cuda > args: -dm_landau_device_type cuda -dm_mat_type aijcusparse > -dm_vec_type cuda -mat_cusparse_use_cpu_solve > > TEST*/ > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0