October 2018 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Mark Adams 31 Oct '18

31 Oct '18

On Wed, Oct 31, 2018 at 6:59 AM Karl Rupp <rupp(a)iue.tuwien.ac.at> wrote: > Hi Mark, > > ah, I was confused by the Python information at the beginning of > configure.log. So it is picking up the correct compiler. > > Have you tried uncommenting the check for GNU? > I'm getting a make error. Thanks,

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Karl Rupp 31 Oct '18

31 Oct '18

Hi Mark, ah, I was confused by the Python information at the beginning of configure.log. So it is picking up the correct compiler. Have you tried uncommenting the check for GNU? Best regards, Karli On 10/31/18 11:40 AM, Mark Adams wrote: > It looks like configure is not finding the correct cc. It does not seem > hard to find. > > 06:37 master= /lustre/atlas/proj-shared/geo127/petsc$ cc --version > gcc (GCC) 6.3.0 20161221 (Cray Inc.) > Copyright (C) 2016 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > 06:37 master= /lustre/atlas/proj-shared/geo127/petsc$ which cc > /opt/cray/craype/2.5.13/bin/cc > 06:38 master= /lustre/atlas/proj-shared/geo127/petsc$ which gcc > /opt/gcc/6.3.0/bin/gcc > > > On Wed, Oct 31, 2018 at 6:34 AM Mark Adams <mfadams(a)lbl.gov > <mailto:[email protected]>> wrote: > > > > On Wed, Oct 31, 2018 at 5:05 AM Karl Rupp <rupp(a)iue.tuwien.ac.at > <mailto:[email protected]>> wrote: > > Hi Mark, > > please comment or remove lines 83 and 84 in > config/BuildSystem/config/packages/cuda.py > > Is there a compiler newer than GCC 4.3 available? > > > You mean 6.3? > > 06:33 ~$ module avail gcc > > ----------------------------------------------------- > /opt/modulefiles ----------------------------------------------------- > gcc/4.8.1 gcc/4.9.3 gcc/6.1.0 > gcc/6.3.0(default) gcc/7.2.0 > gcc/4.8.2 gcc/5.3.0 gcc/6.2.0 gcc/7.1.0 > gcc/7.3.0 > > > Best regards, > Karli > > > > On 10/31/18 8:15 AM, Mark Adams via petsc-dev wrote: > > After loading a cuda module ... > > > > On Wed, Oct 31, 2018 at 2:58 AM Mark Adams <mfadams(a)lbl.gov > <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > I get an error with --with-cuda=1 > > > > On Tue, Oct 30, 2018 at 4:44 PM Smith, Barry F. > <bsmith(a)mcs.anl.gov <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> > wrote: > > > > --with-cudac=1 should be --with-cuda=1 > > > > > > > > > On Oct 30, 2018, at 12:35 PM, Smith, Barry F. via > petsc-dev > > <petsc-dev(a)mcs.anl.gov <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>> > wrote: > > > > > > > > > > > >> On Oct 29, 2018, at 8:09 PM, Mark Adams > <mfadams(a)lbl.gov <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > >> > > >> And a debug build seems to work: > > > > > > Well ok. > > > > > > Are there newer versions of the Gnu compiler > for this > > system? Are there any other compilers on the system > that would > > likely be less buggy? IBM compilers? If this simple code > > generates a gross error with optimization who's to > say how many > > more subtle bugs may be induced in the library by the > buggy > > optimizer (there may be none but IMHO probability > says there > > will be others). > > > > > > Is there any chance that valgrind runs on this > machine; > > you could run the optimized version through it and > see what it says. > > > > > > Barry > > > > > >> > > >> 21:04 1 master= > /lustre/atlas/proj-shared/geo127/petsc$ make > > > PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda > > PETSC_ARCH="" test > > >> Running test examples to verify correct installation > > >> Using > > > PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda > > and PETSC_ARCH= > > >> *******************Error detected during compile or > > link!******************* > > >> See > http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> > > > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > > ex19 > > >> > > > ********************************************************************************* > > >> cc -o ex19.o -c -g > > > -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include `pwd`/ex19.c > > >> cc -g -o ex19 ex19.o > > > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > > -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl > > >> > > > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): > > In function `PetscDLOpen': > > >> > > > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: > > Using 'dlopen' in statically linked applications > requires at > > runtime the shared libraries from the glibc version > used for linking > > >> > > > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): > > In function `PetscOpenSocket': > > >> > > > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: > > warning: Using 'gethostbyname' in statically linked > applications > > requires at runtime the shared libraries from the > glibc version > > used for linking > > >> true ex19 > > >> rm ex19.o > > >> Possible error running C/C++ > > src/snes/examples/tutorials/ex19 with 1 MPI process > > >> See > http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > > >> Number of SNES iterations = 2 > > >> Application 19081049 resources: utime ~1s, stime > ~1s, Rss > > ~17112, inblocks ~36504, outblocks ~111043 > > >> Possible error running C/C++ > > src/snes/examples/tutorials/ex19 with 2 MPI processes > > >> See > http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > > >> Number of SNES iterations = 2 > > >> Application 19081050 resources: utime ~1s, stime > ~1s, Rss > > ~19816, inblocks ~36527, outblocks ~111043 > > >> 5a6 > > >>> Application 19081051 resources: utime ~1s, stime > ~0s, Rss > > ~13864, inblocks ~36527, outblocks ~111043 > > >> > > > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > > >> Possible problem with ex19_hypre, diffs above > > >> ========================================= > > >> *******************Error detected during compile or > > link!******************* > > >> See > http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> > > > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > > ex5f > > >> > ********************************************************* > > >> ftn -c -g > > > -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include > > -o ex5f.o ex5f.F90 > > >> ftn -g -o ex5f ex5f.o > > > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > > -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl > > >> > > > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): > > In function `PetscDLOpen': > > >> > > > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: > > Using 'dlopen' in statically linked applications > requires at > > runtime the shared libraries from the glibc version > used for linking > > >> > > > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): > > In function `PetscOpenSocket': > > >> > > > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: > > warning: Using 'gethostbyname' in statically linked > applications > > requires at runtime the shared libraries from the > glibc version > > used for linking > > >> rm ex5f.o > > >> Possible error running Fortran example > > src/snes/examples/tutorials/ex5f with 1 MPI process > > >> See > http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> Number of SNES iterations = 4 > > >> Application 19081055 resources: utime ~1s, stime > ~0s, Rss > > ~12760, inblocks ~36800, outblocks ~111983 > > >> Completed test examples > > >> 21:06 master= /lustre/atlas/proj-shared/geo127/petsc$ > > > > > >

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Mark Adams 31 Oct '18

31 Oct '18

It looks like configure is not finding the correct cc. It does not seem hard to find. 06:37 master= /lustre/atlas/proj-shared/geo127/petsc$ cc --version gcc (GCC) 6.3.0 20161221 (Cray Inc.) Copyright (C) 2016 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 06:37 master= /lustre/atlas/proj-shared/geo127/petsc$ which cc /opt/cray/craype/2.5.13/bin/cc 06:38 master= /lustre/atlas/proj-shared/geo127/petsc$ which gcc /opt/gcc/6.3.0/bin/gcc On Wed, Oct 31, 2018 at 6:34 AM Mark Adams <mfadams(a)lbl.gov> wrote: > > > On Wed, Oct 31, 2018 at 5:05 AM Karl Rupp <rupp(a)iue.tuwien.ac.at> wrote: > >> Hi Mark, >> >> please comment or remove lines 83 and 84 in >> config/BuildSystem/config/packages/cuda.py >> >> Is there a compiler newer than GCC 4.3 available? >> > > You mean 6.3? > > 06:33 ~$ module avail gcc > > ----------------------------------------------------- /opt/modulefiles > ----------------------------------------------------- > gcc/4.8.1 gcc/4.9.3 gcc/6.1.0 > gcc/6.3.0(default) gcc/7.2.0 > gcc/4.8.2 gcc/5.3.0 gcc/6.2.0 gcc/7.1.0 > gcc/7.3.0 > > > >> >> Best regards, >> Karli >> >> >> >> On 10/31/18 8:15 AM, Mark Adams via petsc-dev wrote: >> > After loading a cuda module ... >> > >> > On Wed, Oct 31, 2018 at 2:58 AM Mark Adams <mfadams(a)lbl.gov >> > <mailto:[email protected]>> wrote: >> > >> > I get an error with --with-cuda=1 >> > >> > On Tue, Oct 30, 2018 at 4:44 PM Smith, Barry F. <bsmith(a)mcs.anl.gov >> > <mailto:[email protected]>> wrote: >> > >> > --with-cudac=1 should be --with-cuda=1 >> > >> > >> > >> > > On Oct 30, 2018, at 12:35 PM, Smith, Barry F. via petsc-dev >> > <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>> wrote: >> > > >> > > >> > > >> > >> On Oct 29, 2018, at 8:09 PM, Mark Adams <mfadams(a)lbl.gov >> > <mailto:[email protected]>> wrote: >> > >> >> > >> And a debug build seems to work: >> > > >> > > Well ok. >> > > >> > > Are there newer versions of the Gnu compiler for this >> > system? Are there any other compilers on the system that would >> > likely be less buggy? IBM compilers? If this simple code >> > generates a gross error with optimization who's to say how many >> > more subtle bugs may be induced in the library by the buggy >> > optimizer (there may be none but IMHO probability says there >> > will be others). >> > > >> > > Is there any chance that valgrind runs on this machine; >> > you could run the optimized version through it and see what it >> says. >> > > >> > > Barry >> > > >> > >> >> > >> 21:04 1 master= /lustre/atlas/proj-shared/geo127/petsc$ make >> > >> PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda >> > PETSC_ARCH="" test >> > >> Running test examples to verify correct installation >> > >> Using >> > >> PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda >> > and PETSC_ARCH= >> > >> *******************Error detected during compile or >> > link!******************* >> > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> > >> >> > >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials >> > ex19 >> > >> >> > >> ********************************************************************************* >> > >> cc -o ex19.o -c -g >> > >> -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include >> `pwd`/ex19.c >> > >> cc -g -o ex19 ex19.o >> > >> -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> > >> -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> > >> -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl >> > >> >> > >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): >> > In function `PetscDLOpen': >> > >> >> > >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: >> > Using 'dlopen' in statically linked applications requires at >> > runtime the shared libraries from the glibc version used for >> linking >> > >> >> > >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): >> > In function `PetscOpenSocket': >> > >> >> > >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: >> > warning: Using 'gethostbyname' in statically linked applications >> > requires at runtime the shared libraries from the glibc version >> > used for linking >> > >> true ex19 >> > >> rm ex19.o >> > >> Possible error running C/C++ >> > src/snes/examples/tutorials/ex19 with 1 MPI process >> > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. >> > >> Number of SNES iterations = 2 >> > >> Application 19081049 resources: utime ~1s, stime ~1s, Rss >> > ~17112, inblocks ~36504, outblocks ~111043 >> > >> Possible error running C/C++ >> > src/snes/examples/tutorials/ex19 with 2 MPI processes >> > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. >> > >> Number of SNES iterations = 2 >> > >> Application 19081050 resources: utime ~1s, stime ~1s, Rss >> > ~19816, inblocks ~36527, outblocks ~111043 >> > >> 5a6 >> > >>> Application 19081051 resources: utime ~1s, stime ~0s, Rss >> > ~13864, inblocks ~36527, outblocks ~111043 >> > >> >> > >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials >> > >> Possible problem with ex19_hypre, diffs above >> > >> ========================================= >> > >> *******************Error detected during compile or >> > link!******************* >> > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> > >> >> > >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials >> > ex5f >> > >> ********************************************************* >> > >> ftn -c -g >> > >> -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include >> > -o ex5f.o ex5f.F90 >> > >> ftn -g -o ex5f ex5f.o >> > >> -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> > >> -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> > >> -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl >> > >> >> > >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): >> > In function `PetscDLOpen': >> > >> >> > >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: >> > Using 'dlopen' in statically linked applications requires at >> > runtime the shared libraries from the glibc version used for >> linking >> > >> >> > >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): >> > In function `PetscOpenSocket': >> > >> >> > >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: >> > warning: Using 'gethostbyname' in statically linked applications >> > requires at runtime the shared libraries from the glibc version >> > used for linking >> > >> rm ex5f.o >> > >> Possible error running Fortran example >> > src/snes/examples/tutorials/ex5f with 1 MPI process >> > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> > >> Number of SNES iterations = 4 >> > >> Application 19081055 resources: utime ~1s, stime ~0s, Rss >> > ~12760, inblocks ~36800, outblocks ~111983 >> > >> Completed test examples >> > >> 21:06 master= /lustre/atlas/proj-shared/geo127/petsc$ >> > > >> > >> >

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Mark Adams 31 Oct '18

31 Oct '18

On Wed, Oct 31, 2018 at 5:05 AM Karl Rupp <rupp(a)iue.tuwien.ac.at> wrote: > Hi Mark, > > please comment or remove lines 83 and 84 in > config/BuildSystem/config/packages/cuda.py > > Is there a compiler newer than GCC 4.3 available? > You mean 6.3? 06:33 ~$ module avail gcc ----------------------------------------------------- /opt/modulefiles ----------------------------------------------------- gcc/4.8.1 gcc/4.9.3 gcc/6.1.0 gcc/6.3.0(default) gcc/7.2.0 gcc/4.8.2 gcc/5.3.0 gcc/6.2.0 gcc/7.1.0 gcc/7.3.0 > > Best regards, > Karli > > > > On 10/31/18 8:15 AM, Mark Adams via petsc-dev wrote: > > After loading a cuda module ... > > > > On Wed, Oct 31, 2018 at 2:58 AM Mark Adams <mfadams(a)lbl.gov > > <mailto:[email protected]>> wrote: > > > > I get an error with --with-cuda=1 > > > > On Tue, Oct 30, 2018 at 4:44 PM Smith, Barry F. <bsmith(a)mcs.anl.gov > > <mailto:[email protected]>> wrote: > > > > --with-cudac=1 should be --with-cuda=1 > > > > > > > > > On Oct 30, 2018, at 12:35 PM, Smith, Barry F. via petsc-dev > > <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>> wrote: > > > > > > > > > > > >> On Oct 29, 2018, at 8:09 PM, Mark Adams <mfadams(a)lbl.gov > > <mailto:[email protected]>> wrote: > > >> > > >> And a debug build seems to work: > > > > > > Well ok. > > > > > > Are there newer versions of the Gnu compiler for this > > system? Are there any other compilers on the system that would > > likely be less buggy? IBM compilers? If this simple code > > generates a gross error with optimization who's to say how many > > more subtle bugs may be induced in the library by the buggy > > optimizer (there may be none but IMHO probability says there > > will be others). > > > > > > Is there any chance that valgrind runs on this machine; > > you could run the optimized version through it and see what it > says. > > > > > > Barry > > > > > >> > > >> 21:04 1 master= /lustre/atlas/proj-shared/geo127/petsc$ make > > > PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda > > PETSC_ARCH="" test > > >> Running test examples to verify correct installation > > >> Using > > > PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda > > and PETSC_ARCH= > > >> *******************Error detected during compile or > > link!******************* > > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> > > > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > > ex19 > > >> > > > ********************************************************************************* > > >> cc -o ex19.o -c -g > > > -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include > `pwd`/ex19.c > > >> cc -g -o ex19 ex19.o > > > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > > -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl > > >> > > > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): > > In function `PetscDLOpen': > > >> > > > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: > > Using 'dlopen' in statically linked applications requires at > > runtime the shared libraries from the glibc version used for > linking > > >> > > > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): > > In function `PetscOpenSocket': > > >> > > > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: > > warning: Using 'gethostbyname' in statically linked applications > > requires at runtime the shared libraries from the glibc version > > used for linking > > >> true ex19 > > >> rm ex19.o > > >> Possible error running C/C++ > > src/snes/examples/tutorials/ex19 with 1 MPI process > > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > > >> Number of SNES iterations = 2 > > >> Application 19081049 resources: utime ~1s, stime ~1s, Rss > > ~17112, inblocks ~36504, outblocks ~111043 > > >> Possible error running C/C++ > > src/snes/examples/tutorials/ex19 with 2 MPI processes > > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > > >> Number of SNES iterations = 2 > > >> Application 19081050 resources: utime ~1s, stime ~1s, Rss > > ~19816, inblocks ~36527, outblocks ~111043 > > >> 5a6 > > >>> Application 19081051 resources: utime ~1s, stime ~0s, Rss > > ~13864, inblocks ~36527, outblocks ~111043 > > >> > > > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > > >> Possible problem with ex19_hypre, diffs above > > >> ========================================= > > >> *******************Error detected during compile or > > link!******************* > > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> > > > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > > ex5f > > >> ********************************************************* > > >> ftn -c -g > > > -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include > > -o ex5f.o ex5f.F90 > > >> ftn -g -o ex5f ex5f.o > > > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > > -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl > > >> > > > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): > > In function `PetscDLOpen': > > >> > > > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: > > Using 'dlopen' in statically linked applications requires at > > runtime the shared libraries from the glibc version used for > linking > > >> > > > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): > > In function `PetscOpenSocket': > > >> > > > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: > > warning: Using 'gethostbyname' in statically linked applications > > requires at runtime the shared libraries from the glibc version > > used for linking > > >> rm ex5f.o > > >> Possible error running Fortran example > > src/snes/examples/tutorials/ex5f with 1 MPI process > > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > > >> Number of SNES iterations = 4 > > >> Application 19081055 resources: utime ~1s, stime ~0s, Rss > > ~12760, inblocks ~36800, outblocks ~111983 > > >> Completed test examples > > >> 21:06 master= /lustre/atlas/proj-shared/geo127/petsc$ > > > > > >

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Karl Rupp 31 Oct '18

31 Oct '18

Hi Mark, please comment or remove lines 83 and 84 in config/BuildSystem/config/packages/cuda.py Is there a compiler newer than GCC 4.3 available? Best regards, Karli On 10/31/18 8:15 AM, Mark Adams via petsc-dev wrote: > After loading a cuda module ... > > On Wed, Oct 31, 2018 at 2:58 AM Mark Adams <mfadams(a)lbl.gov > <mailto:[email protected]>> wrote: > > I get an error with --with-cuda=1 > > On Tue, Oct 30, 2018 at 4:44 PM Smith, Barry F. <bsmith(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > --with-cudac=1 should be --with-cuda=1 > > > > > On Oct 30, 2018, at 12:35 PM, Smith, Barry F. via petsc-dev > <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>> wrote: > > > > > > > >> On Oct 29, 2018, at 8:09 PM, Mark Adams <mfadams(a)lbl.gov > <mailto:[email protected]>> wrote: > >> > >> And a debug build seems to work: > > > > Well ok. > > > > Are there newer versions of the Gnu compiler for this > system? Are there any other compilers on the system that would > likely be less buggy? IBM compilers? If this simple code > generates a gross error with optimization who's to say how many > more subtle bugs may be induced in the library by the buggy > optimizer (there may be none but IMHO probability says there > will be others). > > > > Is there any chance that valgrind runs on this machine; > you could run the optimized version through it and see what it says. > > > > Barry > > > >> > >> 21:04 1 master= /lustre/atlas/proj-shared/geo127/petsc$ make > PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda > PETSC_ARCH="" test > >> Running test examples to verify correct installation > >> Using > PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda > and PETSC_ARCH= > >> *******************Error detected during compile or > link!******************* > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > ex19 > >> > ********************************************************************************* > >> cc -o ex19.o -c -g > -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include `pwd`/ex19.c > >> cc -g -o ex19 ex19.o > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl > >> > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): > In function `PetscDLOpen': > >> > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: > Using 'dlopen' in statically linked applications requires at > runtime the shared libraries from the glibc version used for linking > >> > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): > In function `PetscOpenSocket': > >> > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: > warning: Using 'gethostbyname' in statically linked applications > requires at runtime the shared libraries from the glibc version > used for linking > >> true ex19 > >> rm ex19.o > >> Possible error running C/C++ > src/snes/examples/tutorials/ex19 with 1 MPI process > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > >> Number of SNES iterations = 2 > >> Application 19081049 resources: utime ~1s, stime ~1s, Rss > ~17112, inblocks ~36504, outblocks ~111043 > >> Possible error running C/C++ > src/snes/examples/tutorials/ex19 with 2 MPI processes > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > >> Number of SNES iterations = 2 > >> Application 19081050 resources: utime ~1s, stime ~1s, Rss > ~19816, inblocks ~36527, outblocks ~111043 > >> 5a6 > >>> Application 19081051 resources: utime ~1s, stime ~0s, Rss > ~13864, inblocks ~36527, outblocks ~111043 > >> > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > >> Possible problem with ex19_hypre, diffs above > >> ========================================= > >> *******************Error detected during compile or > link!******************* > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> > /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > ex5f > >> ********************************************************* > >> ftn -c -g > -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include > -o ex5f.o ex5f.F90 > >> ftn -g -o ex5f ex5f.o > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl > >> > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): > In function `PetscDLOpen': > >> > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: > Using 'dlopen' in statically linked applications requires at > runtime the shared libraries from the glibc version used for linking > >> > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): > In function `PetscOpenSocket': > >> > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: > warning: Using 'gethostbyname' in statically linked applications > requires at runtime the shared libraries from the glibc version > used for linking > >> rm ex5f.o > >> Possible error running Fortran example > src/snes/examples/tutorials/ex5f with 1 MPI process > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> Number of SNES iterations = 4 > >> Application 19081055 resources: utime ~1s, stime ~0s, Rss > ~12760, inblocks ~36800, outblocks ~111983 > >> Completed test examples > >> 21:06 master= /lustre/atlas/proj-shared/geo127/petsc$ > > >

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Mark Adams 31 Oct '18

31 Oct '18

After loading a cuda module ... On Wed, Oct 31, 2018 at 2:58 AM Mark Adams <mfadams(a)lbl.gov> wrote: > I get an error with --with-cuda=1 > > On Tue, Oct 30, 2018 at 4:44 PM Smith, Barry F. <bsmith(a)mcs.anl.gov> > wrote: > >> --with-cudac=1 should be --with-cuda=1 >> >> >> >> > On Oct 30, 2018, at 12:35 PM, Smith, Barry F. via petsc-dev < >> petsc-dev(a)mcs.anl.gov> wrote: >> > >> > >> > >> >> On Oct 29, 2018, at 8:09 PM, Mark Adams <mfadams(a)lbl.gov> wrote: >> >> >> >> And a debug build seems to work: >> > >> > Well ok. >> > >> > Are there newer versions of the Gnu compiler for this system? Are >> there any other compilers on the system that would likely be less buggy? >> IBM compilers? If this simple code generates a gross error with >> optimization who's to say how many more subtle bugs may be induced in the >> library by the buggy optimizer (there may be none but IMHO probability says >> there will be others). >> > >> > Is there any chance that valgrind runs on this machine; you could >> run the optimized version through it and see what it says. >> > >> > Barry >> > >> >> >> >> 21:04 1 master= /lustre/atlas/proj-shared/geo127/petsc$ make >> PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda >> PETSC_ARCH="" test >> >> Running test examples to verify correct installation >> >> Using >> PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda >> and PETSC_ARCH= >> >> *******************Error detected during compile or >> link!******************* >> >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials ex19 >> >> >> ********************************************************************************* >> >> cc -o ex19.o -c -g >> -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include >> `pwd`/ex19.c >> >> cc -g -o ex19 ex19.o >> -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl >> >> >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): >> In function `PetscDLOpen': >> >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: >> warning: Using 'dlopen' in statically linked applications requires at >> runtime the shared libraries from the glibc version used for linking >> >> >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): >> In function `PetscOpenSocket': >> >> >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: >> warning: Using 'gethostbyname' in statically linked applications requires >> at runtime the shared libraries from the glibc version used for linking >> >> true ex19 >> >> rm ex19.o >> >> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 >> MPI process >> >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. >> >> Number of SNES iterations = 2 >> >> Application 19081049 resources: utime ~1s, stime ~1s, Rss ~17112, >> inblocks ~36504, outblocks ~111043 >> >> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 >> MPI processes >> >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. >> >> Number of SNES iterations = 2 >> >> Application 19081050 resources: utime ~1s, stime ~1s, Rss ~19816, >> inblocks ~36527, outblocks ~111043 >> >> 5a6 >> >>> Application 19081051 resources: utime ~1s, stime ~0s, Rss ~13864, >> inblocks ~36527, outblocks ~111043 >> >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials >> >> Possible problem with ex19_hypre, diffs above >> >> ========================================= >> >> *******************Error detected during compile or >> link!******************* >> >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials ex5f >> >> ********************************************************* >> >> ftn -c -g >> -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include >> -o ex5f.o ex5f.F90 >> >> ftn -g -o ex5f ex5f.o >> -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib >> -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl >> >> >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): >> In function `PetscDLOpen': >> >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: >> warning: Using 'dlopen' in statically linked applications requires at >> runtime the shared libraries from the glibc version used for linking >> >> >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): >> In function `PetscOpenSocket': >> >> >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: >> warning: Using 'gethostbyname' in statically linked applications requires >> at runtime the shared libraries from the glibc version used for linking >> >> rm ex5f.o >> >> Possible error running Fortran example >> src/snes/examples/tutorials/ex5f with 1 MPI process >> >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> >> Number of SNES iterations = 4 >> >> Application 19081055 resources: utime ~1s, stime ~0s, Rss ~12760, >> inblocks ~36800, outblocks ~111983 >> >> Completed test examples >> >> 21:06 master= /lustre/atlas/proj-shared/geo127/petsc$ >> > >> >>

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Mark Adams 31 Oct '18

31 Oct '18

I get an error with --with-cuda=1 On Tue, Oct 30, 2018 at 4:44 PM Smith, Barry F. <bsmith(a)mcs.anl.gov> wrote: > --with-cudac=1 should be --with-cuda=1 > > > > > On Oct 30, 2018, at 12:35 PM, Smith, Barry F. via petsc-dev < > petsc-dev(a)mcs.anl.gov> wrote: > > > > > > > >> On Oct 29, 2018, at 8:09 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > >> > >> And a debug build seems to work: > > > > Well ok. > > > > Are there newer versions of the Gnu compiler for this system? Are > there any other compilers on the system that would likely be less buggy? > IBM compilers? If this simple code generates a gross error with > optimization who's to say how many more subtle bugs may be induced in the > library by the buggy optimizer (there may be none but IMHO probability says > there will be others). > > > > Is there any chance that valgrind runs on this machine; you could run > the optimized version through it and see what it says. > > > > Barry > > > >> > >> 21:04 1 master= /lustre/atlas/proj-shared/geo127/petsc$ make > PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda > PETSC_ARCH="" test > >> Running test examples to verify correct installation > >> Using > PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda > and PETSC_ARCH= > >> *******************Error detected during compile or > link!******************* > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials ex19 > >> > ********************************************************************************* > >> cc -o ex19.o -c -g > -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include > `pwd`/ex19.c > >> cc -g -o ex19 ex19.o > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl > >> > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): > In function `PetscDLOpen': > >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: > warning: Using 'dlopen' in statically linked applications requires at > runtime the shared libraries from the glibc version used for linking > >> > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): > In function `PetscOpenSocket': > >> > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: > warning: Using 'gethostbyname' in statically linked applications requires > at runtime the shared libraries from the glibc version used for linking > >> true ex19 > >> rm ex19.o > >> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 > MPI process > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > >> Number of SNES iterations = 2 > >> Application 19081049 resources: utime ~1s, stime ~1s, Rss ~17112, > inblocks ~36504, outblocks ~111043 > >> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 > MPI processes > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. > >> Number of SNES iterations = 2 > >> Application 19081050 resources: utime ~1s, stime ~1s, Rss ~19816, > inblocks ~36527, outblocks ~111043 > >> 5a6 > >>> Application 19081051 resources: utime ~1s, stime ~0s, Rss ~13864, > inblocks ~36527, outblocks ~111043 > >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials > >> Possible problem with ex19_hypre, diffs above > >> ========================================= > >> *******************Error detected during compile or > link!******************* > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials ex5f > >> ********************************************************* > >> ftn -c -g > -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include > -o ex5f.o ex5f.F90 > >> ftn -g -o ex5f ex5f.o > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib > -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl > >> > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): > In function `PetscDLOpen': > >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: > warning: Using 'dlopen' in statically linked applications requires at > runtime the shared libraries from the glibc version used for linking > >> > /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): > In function `PetscOpenSocket': > >> > /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: > warning: Using 'gethostbyname' in statically linked applications requires > at runtime the shared libraries from the glibc version used for linking > >> rm ex5f.o > >> Possible error running Fortran example src/snes/examples/tutorials/ex5f > with 1 MPI process > >> See http://www.mcs.anl.gov/petsc/documentation/faq.html > >> Number of SNES iterations = 4 > >> Application 19081055 resources: utime ~1s, stime ~0s, Rss ~12760, > inblocks ~36800, outblocks ~111983 > >> Completed test examples > >> 21:06 master= /lustre/atlas/proj-shared/geo127/petsc$ > > > >

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Smith, Barry F. 30 Oct '18

30 Oct '18

--with-cudac=1 should be --with-cuda=1 > On Oct 30, 2018, at 12:35 PM, Smith, Barry F. via petsc-dev <petsc-dev(a)mcs.anl.gov> wrote: > > > >> On Oct 29, 2018, at 8:09 PM, Mark Adams <mfadams(a)lbl.gov> wrote: >> >> And a debug build seems to work: > > Well ok. > > Are there newer versions of the Gnu compiler for this system? Are there any other compilers on the system that would likely be less buggy? IBM compilers? If this simple code generates a gross error with optimization who's to say how many more subtle bugs may be induced in the library by the buggy optimizer (there may be none but IMHO probability says there will be others). > > Is there any chance that valgrind runs on this machine; you could run the optimized version through it and see what it says. > > Barry > >> >> 21:04 1 master= /lustre/atlas/proj-shared/geo127/petsc$ make PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda PETSC_ARCH="" test >> Running test examples to verify correct installation >> Using PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda and PETSC_ARCH= >> *******************Error detected during compile or link!******************* >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials ex19 >> ********************************************************************************* >> cc -o ex19.o -c -g -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include `pwd`/ex19.c >> cc -g -o ex19 ex19.o -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): In function `PetscDLOpen': >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): In function `PetscOpenSocket': >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking >> true ex19 >> rm ex19.o >> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. >> Number of SNES iterations = 2 >> Application 19081049 resources: utime ~1s, stime ~1s, Rss ~17112, inblocks ~36504, outblocks ~111043 >> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 MPI processes >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> lid velocity = 0.0016, prandtl # = 1., grashof # = 1. >> Number of SNES iterations = 2 >> Application 19081050 resources: utime ~1s, stime ~1s, Rss ~19816, inblocks ~36527, outblocks ~111043 >> 5a6 >>> Application 19081051 resources: utime ~1s, stime ~0s, Rss ~13864, inblocks ~36527, outblocks ~111043 >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials >> Possible problem with ex19_hypre, diffs above >> ========================================= >> *******************Error detected during compile or link!******************* >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials ex5f >> ********************************************************* >> ftn -c -g -I/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/include -o ex5f.o ex5f.F90 >> ftn -g -o ex5f ex5f.o -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib -Wl,-rpath,/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib -L/lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib -lpetsc -lHYPRE -lflapack -lfblas -lparmetis -lmetis -ldl >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(dlimpl.o): In function `PetscDLOpen': >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/dll/dlimpl.c:108: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking >> /lustre/atlas/proj-shared/geo127/petsc_titan_dbg64idx_gnu_cuda/lib/libpetsc.a(send.o): In function `PetscOpenSocket': >> /lustre/atlas1/geo127/proj-shared/petsc/src/sys/classes/viewer/impls/socket/send.c:108: warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking >> rm ex5f.o >> Possible error running Fortran example src/snes/examples/tutorials/ex5f with 1 MPI process >> See http://www.mcs.anl.gov/petsc/documentation/faq.html >> Number of SNES iterations = 4 >> Application 19081055 resources: utime ~1s, stime ~0s, Rss ~12760, inblocks ~36800, outblocks ~111983 >> Completed test examples >> 21:06 master= /lustre/atlas/proj-shared/geo127/petsc$ >

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Mark Adams 30 Oct '18

30 Oct '18

And I do not seem to have CUDA working: 15:38 134 master= /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials$ aprun -n 1 ./ex19 -da_refine 3 -snes_monitor -dm_mat_type seqaijcusparse -dm_vec_type seqcuda -pc_type gamg -ksp_monitor [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing package: http://www.mcs.anl.gov/petsc/documentation/installation.html#external [0]PETSC ERROR: Unknown vector type: seqcuda [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. lid velocity = 0.0016, prandtl # = 1., grashof # = 1. [0]PETSC ERROR: Petsc Development GIT revision: v3.10.2-461-g0ed19bb123 GIT Date: 2018-10-29 13:43:53 +0100 [0]PETSC ERROR: ./ex19 on a named nid09363 by adams Tue Oct 30 15:38:44 2018 [0]PETSC ERROR: Configure options --with-cudac=1 --with-batch=0 --prefix=/lustre/atlas/proj-shared/geo127/petsc_titan_opt64idx_gnu_cuda --download-hypre --download-metis --download-parmetis --download-fblaslapack --with-cc=cc --with-clib-autodetect=0 --with-cxx=CC --with-cxxlib-autodetect=0 --with-fc=ftn --with-fortranlib-autodetect=0 --with-shared-libraries=0 --known-mpi-shared-libraries=1 --with-mpiexec=aprun --with-x=0 --with-64-bit-indices --with-debugging=0 PETSC_ARCH=arch-titan-opt64idx-gnu-cuda PETSC_DIR=/lustre/atlas/proj-shared/geo127/petsc [0]PETSC ERROR: #1 VecSetType() line 42 in /lustre/atlas1/geo127/proj-shared/petsc/src/vec/vec/interface/vecreg.c [0]PETSC ERROR: #2 DMCreateGlobalVector_DA() line 39 in /lustre/atlas1/geo127/proj-shared/petsc/src/dm/impls/da/dadist.c [0]PETSC ERROR: #3 DMCreateGlobalVector() line 928 in /lustre/atlas1/geo127/proj-shared/petsc/src/dm/interface/dm.c [0]PETSC ERROR: #4 main() line 158 in /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials/ex19.c [0]PETSC ERROR: PETSc Option Table entries: [0]PETSC ERROR: -da_refine 3 [0]PETSC ERROR: -dm_mat_type seqaijcusparse [0]PETSC ERROR: -dm_vec_type seqcuda [0]PETSC ERROR: -ksp_monitor [0]PETSC ERROR: -pc_type gamg [0]PETSC ERROR: -snes_monitor [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint(a)mcs.anl.gov---------- Rank 0 [Tue Oct 30 15:38:44 2018] [c24-2c0s6n3] application called MPI_Abort(MPI_COMM_WORLD, 86) - process 0 _pmiu_daemon(SIGCHLD): [NID 09363] [c24-2c0s6n3] [Tue Oct 30 15:38:44 2018] PE RANK 0 exit signal Aborted Application 19084507 exit codes: 134 Application 19084507 resources: utime ~0s, stime ~1s, Rss ~36228, inblocks ~19404, outblocks ~51053 15:38 134 master= /lustre/atlas/proj-shared/geo127/petsc/src/snes/examples/tutorials$ >

1 0

Re: [petsc-dev] Error running on Titan with GPUs & GNU
by Mark Adams 30 Oct '18

30 Oct '18

> > > > Are there newer versions of the Gnu compiler for this system? Yes: -------------------------------------------------------------------------------------------------- /opt/modulefiles -------------------------------------------------------------------------------------------------- gcc/4.8.1 gcc/4.8.2 gcc/4.9.3 gcc/5.3.0 gcc/6.1.0 gcc/6.2.0 gcc/6.3.0(default) gcc/7.1.0 gcc/7.2.0 gcc/7.3.0 > Are there any other compilers on the system that would likely be less > buggy? IBM compilers? If this simple code generates a gross error with > optimization who's to say how many more subtle bugs may be induced in the > library by the buggy optimizer (there may be none but IMHO probability says > there will be others). > Let me ask them what they recommend to use with cuda codes. > > Is there any chance that valgrind runs on this machine; you could run > the optimized version through it and see what it says. > > Valgrind works but tons of output and I could not see anything interesting in there. And, this test does work with 1 processor! I think this is only a problem when GMRES is uses as the eigen estimator in Cheby. GMRES solvers work and Cheby works with -mg_levels_esteig_ksp_type cg.

1 0