March 2020 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Globsearch fails for me when running tests
by Scott Kruger 25 Mar '20

25 Mar '20

Ugh -- this is ugly. Can't we just tell users to either use the '%' syntax or recompile their linux kernel? Just kidding. I'll take a look. Scott On 3/25/20 3:48 PM, Jed Brown wrote: > Scott, you can't pass '$(alltesttargets)' on the command line like this. > > TESTTARGETS := $(shell $(PYTHON) -c"import sys,fnmatch,itertools; m=[fnmatch.filter(sys.argv[2].split(),p) for p in sys.argv[1].split()]; print(' '.join(list(itertools.chain.from_iterable(m))))" '$(globsearch)' '$(alltesttargets)') > > For this feature, probably put them in an argsfile > > $(file >$(TESTDIR)/globsearch.args,$(alltesttargets)) > > and make your Python read from that file. I don't know a way to pass it > on stdin. > > Scott Kruger <kruger(a)txcorp.com> writes: > >> What platform? >> >> On 3/25/20 3:20 PM, Stefano Zampini wrote: >>> This was working before.. >>> >>> [szampini@localhost petsc]$ make -f gmakefile.test test globsearch='dm*' >>> make: execvp: /usr/bin/sh: Argument list too long >>> Using MAKEFLAGS: -- globsearch=dm* >>> # No tests run >>> # No tests run >>> # No tests run >>> >>> [szampini@localhost petsc]$ git branch >>> * knepley/feature-dm-remove-hybrid >>> >>> [szampini@localhost petsc]$ make -v >>> GNU Make 4.2.1 >>> Built for x86_64-redhat-linux-gnu >>> Copyright (C) 1988-2016 Free Software Foundation, Inc. >>> License GPLv3+: GNU GPL version 3 or later >>> <http://gnu.org/licenses/gpl.html> >>> This is free software: you are free to change and redistribute it. >>> There is NO WARRANTY, to the extent permitted by law. >>> >> >> -- >> Tech-X Corporation kruger(a)txcorp.com >> 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841 >> Boulder, CO 80303 Fax: (303) 448-7756 -- Tech-X Corporation kruger(a)txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841 Boulder, CO 80303 Fax: (303) 448-7756

1 0

Re: [petsc-dev] Globsearch fails for me when running tests
by Jed Brown 25 Mar '20

25 Mar '20

Scott, you can't pass '$(alltesttargets)' on the command line like this. TESTTARGETS := $(shell $(PYTHON) -c"import sys,fnmatch,itertools; m=[fnmatch.filter(sys.argv[2].split(),p) for p in sys.argv[1].split()]; print(' '.join(list(itertools.chain.from_iterable(m))))" '$(globsearch)' '$(alltesttargets)') For this feature, probably put them in an argsfile $(file >$(TESTDIR)/globsearch.args,$(alltesttargets)) and make your Python read from that file. I don't know a way to pass it on stdin. Scott Kruger <kruger(a)txcorp.com> writes: > What platform? > > On 3/25/20 3:20 PM, Stefano Zampini wrote: >> This was working before.. >> >> [szampini@localhost petsc]$ make -f gmakefile.test test globsearch='dm*' >> make: execvp: /usr/bin/sh: Argument list too long >> Using MAKEFLAGS: -- globsearch=dm* >> # No tests run >> # No tests run >> # No tests run >> >> [szampini@localhost petsc]$ git branch >> * knepley/feature-dm-remove-hybrid >> >> [szampini@localhost petsc]$ make -v >> GNU Make 4.2.1 >> Built for x86_64-redhat-linux-gnu >> Copyright (C) 1988-2016 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later >> <http://gnu.org/licenses/gpl.html> >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. >> > > -- > Tech-X Corporation kruger(a)txcorp.com > 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841 > Boulder, CO 80303 Fax: (303) 448-7756

1 0

Re: [petsc-dev] Globsearch fails for me when running tests
by Stefano Zampini 25 Mar '20

25 Mar '20

> > > What do you have for > > getconf ARG_MAX > > [szampini@localhost petsc]$ getconf ARG_MAX 2097152 > make -f gmakefile.test test search=dm% > > works. This is on Fedora 30 > > make: execvp: /usr/bin/sh: Argument list too long > > Using MAKEFLAGS: -- globsearch=dm* > > # No tests run > > # No tests run > > # No tests run > > > > [szampini@localhost petsc]$ git branch > > * knepley/feature-dm-remove-hybrid > > > > [szampini@localhost petsc]$ make -v > > GNU Make 4.2.1 > > Built for x86_64-redhat-linux-gnu > > Copyright (C) 1988-2016 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later < > http://gnu.org/licenses/gpl.html> > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. > -- Stefano

1 0

Re: [petsc-dev] Globsearch fails for me when running tests
by Scott Kruger 25 Mar '20

25 Mar '20

What platform? On 3/25/20 3:20 PM, Stefano Zampini wrote: > This was working before.. > > [szampini@localhost petsc]$ make -f gmakefile.test test globsearch='dm*' > make: execvp: /usr/bin/sh: Argument list too long > Using MAKEFLAGS: -- globsearch=dm* > # No tests run > # No tests run > # No tests run > > [szampini@localhost petsc]$ git branch > * knepley/feature-dm-remove-hybrid > > [szampini@localhost petsc]$ make -v > GNU Make 4.2.1 > Built for x86_64-redhat-linux-gnu > Copyright (C) 1988-2016 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > -- Tech-X Corporation kruger(a)txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 974-1841 Boulder, CO 80303 Fax: (303) 448-7756

1 0

Re: [petsc-dev] Globsearch fails for me when running tests
by Jed Brown 25 Mar '20

25 Mar '20

Stefano Zampini <stefano.zampini(a)gmail.com> writes: > This was working before.. > > [szampini@localhost petsc]$ make -f gmakefile.test test globsearch='dm*' What do you have for getconf ARG_MAX make -f gmakefile.test test search=dm% > make: execvp: /usr/bin/sh: Argument list too long > Using MAKEFLAGS: -- globsearch=dm* > # No tests run > # No tests run > # No tests run > > [szampini@localhost petsc]$ git branch > * knepley/feature-dm-remove-hybrid > > [szampini@localhost petsc]$ make -v > GNU Make 4.2.1 > Built for x86_64-redhat-linux-gnu > Copyright (C) 1988-2016 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law.

1 0

Globsearch fails for me when running tests
by Stefano Zampini 25 Mar '20

25 Mar '20

This was working before.. [szampini@localhost petsc]$ make -f gmakefile.test test globsearch='dm*' make: execvp: /usr/bin/sh: Argument list too long Using MAKEFLAGS: -- globsearch=dm* # No tests run # No tests run # No tests run [szampini@localhost petsc]$ git branch * knepley/feature-dm-remove-hybrid [szampini@localhost petsc]$ make -v GNU Make 4.2.1 Built for x86_64-redhat-linux-gnu Copyright (C) 1988-2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

1 0

Fwd: Suggestions for MatProductCreate()
by hong＠aspiritech.org 24 Mar '20

24 Mar '20

Lisandro, > As I said in previous emails, I'm not complaining about the API, I said it > was a welcome adition. All I'm asking is: > > 1) Fix the handling of reference counting. This is an implementation > detail, not really an API change. > I'm creating a branch from master and patch it with your suggestions on reference counting. > > 2) Add a simple new routine to the API, MatProductReset(D) [or maybe name > it MatProductClear(D) ], that should throw away all of the D->product stuff > to release memory related to the product algorithm implementation. After > calling MatProductReset(D), you cannot call anymore MatProductNumeric(). > Do you mean adding a public function MatProductClear(D) to clear internal data structure 'product' attached to D? For example, after destroy A and B, user should call MatProductClear(D), i.e., MatDestroy(&A) ; MatDestroy(&B) ; MatProductClear(D); // throw away all of the D->product stuff, thus D becomes a standard matrix I understand that you are concerned about the dangling pointers. However, most users compute D =A*B, then destroy A, B and D separately in random order. Do we require user to call MatProductClear(D) after A or B is destroyed? > >> 3) For MatProduct, I consider >> MatProductSetFromOptions() is logically equivalent to >> MatSetFromOptions(), >> MatProductSymbolic() is logically equivalent to MatSetUp(), >> and >> MatProductNumeric() assembles the matproduct D repeatedly when A or B >> are updated. >> >> > OK, so... your point it that what looks like "setup" code > in MatProductSetFromOptions() is just some minimal stuff required to > actually dispatch the SetFromOptions_typeA_typeB routine, right? In that > case, you are right, no changes/refactors needed here. > Yes, MatProductSetFromOptions() only dispatches SetFromOptions_typeA_typeB_producttype() which handles local options and polymorphism. The organization of new API is described at the top of petsc/src/mat/interface/matproduct.c. Hong

1 0

Re: [petsc-dev] Configure --download-eigen hangs
by Jacob Faibussowitsch 24 Mar '20

24 Mar '20

> the configure.log sent didn't list the build command used for eigen. Yeah it never got there. The log is the “complete” log from configure. After a while I always just ctrl-c the configure. I will try building from conf.eigen and see if it goes through. Best regards, Jacob Faibussowitsch (Jacob Fai - booss - oh - vitch) Cell: (312) 694-3391 > On Mar 24, 2020, at 1:27 PM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > the configure.log sent didn't list the build command used for eigen. > >>>> > Have to rebuild EIGEN, /Users/jacobfaibussowitsch/NoSync/petsc/64-arch-darwin-c-debug/externalpackages/git.eigen/eigen.petscconf != /Users/jacobfaibussowitsch/NoSync/petsc/64-arch-darwin-c-debug/lib/petsc/conf/pkg.conf.eigen > <<<< > > You can check the contents of /Users/jacobfaibussowitsch/NoSync/petsc/64-arch-darwin-c-debug/lib/petsc/conf/pkg.conf.eigen - use it to manually build eigen and see if the problem persists. > > In my build - I see: [invoked from /Users/balay/petsc/64-arch-darwin-c-debug/externalpackages/git.eigen/petsc-build] > > Executing: /usr/local/bin/cmake .. -DCMAKE_INSTALL_PREFIX=/Users/balay/petsc/64-arch-darwin-c-debug -DCMAKE_VERBOSE_MAKEFILE=1 -DCMAKE_C_COMPILER="/Users/balay/petsc/64-arch-darwin-c-debug/bin/mpicc" -DCMAKE_AR=/usr/bin/ar -DCMAKE_RANLIB=/usr/bin/ranlib -DCMAKE_C_FLAGS:STRING="-fstack-protector -fno-stack-check -Qunused-arguments -g3" -DCMAKE_C_FLAGS_DEBUG:STRING="-fstack-protector -fno-stack-check -Qunused-arguments -g3" -DCMAKE_C_FLAGS_RELEASE:STRING="-fstack-protector -fno-stack-check -Qunused-arguments -g3" -DCMAKE_CXX_COMPILER="/Users/balay/petsc/64-arch-darwin-c-debug/bin/mpicxx" -DCMAKE_CXX_FLAGS:STRING="-fstack-protector -fno-stack-check -g" -DCMAKE_CXX_FLAGS_DEBUG:STRING="-fstack-protector -fno-stack-check -g" -DCMAKE_CXX_FLAGS_RELEASE:STRING="-fstack-protector -fno-stack-check -g" -DCMAKE_Fortran_COMPILER="/Users/balay/petsc/64-arch-darwin-c-debug/bin/mpif90" -DCMAKE_Fortran_FLAGS:STRING="-ffree-line-length-0 -g" -DCMAKE_Fortran_FLAGS_DEBUG:STRING="-ffree-line-lengt > h-0 -g" -DCMAKE_Fortran_FLAGS_RELEASE:STRING="-ffree-line-length-0 -g" -DBUILD_SHARED_LIBS=on -DENABLE_OPENMP=OFF > > Satish > > On Tue, 24 Mar 2020, Pierre Jolivet wrote: > >> FWIW, I’ve had the same issue as Jacob for quite a while and I’ve since been using brew install eigen to bypass this. >> I’ve just checked again, and look at one of the process spawned during ./configure. That doesn’t look alright to me. >> >> Thanks, >> Pierre >> >> >> >>> On 24 Mar 2020, at 6:40 PM, Satish Balay via petsc-dev <petsc-dev(a)mcs.anl.gov> wrote: >>> >>> I just attempted a build on osx - and didn't see any such hang. I'm not sure why your build is hanging.. >>> >>> balay@kpro petsc % ./configure --download-eigen --download-pragmatic --download-metis --download-mpich >>> >>> Satish >>> >>> On Tue, 24 Mar 2020, Jacob Faibussowitsch wrote: >>> >>>> Hello all, >>>> >>>> Trying to install pragmatic using petsc, which requires Eigen package. I am installing these packages using arch/lib/petsc/conf/reconfigure —download-eigen —download-pragmatic. This is as far as it gets: >>>> >>>> /reconfigure-64-arch-darwin-c-debug.py --download-eigen >>>> =============================================================================== >>>> Configuring PETSc to compile on your system >>>> =============================================================================== >>>> =============================================================================== >>>> Trying to download git://https://gitlab.com/libeigen/eigen for EIGEN =============================================================================== >>>> =============================================================================== >>>> Configuring EIGEN with cmake, this may take several minutes =============================================================================== >>>> >>>> I have let this run for ~30 minute periods 3 times and it never progresses. I think that it does finish configure, but never progresses to the make stage. >>>> >>>> I have checked htop (brew version of top) and the only relevant processes running are bash with ~5% CPU. This is less robust but on my setup the window bar (where the close, minimize, etc buttons are) of the terminal window also reports the currently running process and this also only shows bash. For reference when configure is running normally, the top bar switches between clang, ld, expr, cmake, g++, etc. System resources used usually shoots up to ~40-90% as it gets into the different configures and makes for each package. >>>> >>>> I have tried this on 2 different petsc configurations; --with-64-bit-indices=0/1, since this originally didn’t work with 64 bit indices so I figured that may be the issue. I have also cleared my ccache completely using ccache -c -C and rerun both configures. >>>> >>>> I have attached both configure.logs, note configure.log —> —with-64-bit-indices=1, configure.log.bkp —> —with-64-bit-indices=0 >>>> >>>> >>>> Best regards, >>>> >>>> Jacob Faibussowitsch >>>> (Jacob Fai - booss - oh - vitch) >>>> Cell: (312) 694-3391 >>>> >> >>

1 0

Re: [petsc-dev] Configure --download-eigen hangs
by Satish Balay 24 Mar '20

24 Mar '20

the configure.log sent didn't list the build command used for eigen. >>> Have to rebuild EIGEN, /Users/jacobfaibussowitsch/NoSync/petsc/64-arch-darwin-c-debug/externalpackages/git.eigen/eigen.petscconf != /Users/jacobfaibussowitsch/NoSync/petsc/64-arch-darwin-c-debug/lib/petsc/conf/pkg.conf.eigen <<<< You can check the contents of /Users/jacobfaibussowitsch/NoSync/petsc/64-arch-darwin-c-debug/lib/petsc/conf/pkg.conf.eigen - use it to manually build eigen and see if the problem persists. In my build - I see: [invoked from /Users/balay/petsc/64-arch-darwin-c-debug/externalpackages/git.eigen/petsc-build] Executing: /usr/local/bin/cmake .. -DCMAKE_INSTALL_PREFIX=/Users/balay/petsc/64-arch-darwin-c-debug -DCMAKE_VERBOSE_MAKEFILE=1 -DCMAKE_C_COMPILER="/Users/balay/petsc/64-arch-darwin-c-debug/bin/mpicc" -DCMAKE_AR=/usr/bin/ar -DCMAKE_RANLIB=/usr/bin/ranlib -DCMAKE_C_FLAGS:STRING="-fstack-protector -fno-stack-check -Qunused-arguments -g3" -DCMAKE_C_FLAGS_DEBUG:STRING="-fstack-protector -fno-stack-check -Qunused-arguments -g3" -DCMAKE_C_FLAGS_RELEASE:STRING="-fstack-protector -fno-stack-check -Qunused-arguments -g3" -DCMAKE_CXX_COMPILER="/Users/balay/petsc/64-arch-darwin-c-debug/bin/mpicxx" -DCMAKE_CXX_FLAGS:STRING="-fstack-protector -fno-stack-check -g" -DCMAKE_CXX_FLAGS_DEBUG:STRING="-fstack-protector -fno-stack-check -g" -DCMAKE_CXX_FLAGS_RELEASE:STRING="-fstack-protector -fno-stack-check -g" -DCMAKE_Fortran_COMPILER="/Users/balay/petsc/64-arch-darwin-c-debug/bin/mpif90" -DCMAKE_Fortran_FLAGS:STRING="-ffree-line-length-0 -g" -DCMAKE_Fortran_FLAGS_DEBUG:STRING="-ffree-line-lengt h-0 -g" -DCMAKE_Fortran_FLAGS_RELEASE:STRING="-ffree-line-length-0 -g" -DBUILD_SHARED_LIBS=on -DENABLE_OPENMP=OFF Satish On Tue, 24 Mar 2020, Pierre Jolivet wrote: > FWIW, I’ve had the same issue as Jacob for quite a while and I’ve since been using brew install eigen to bypass this. > I’ve just checked again, and look at one of the process spawned during ./configure. That doesn’t look alright to me. > > Thanks, > Pierre > > > > > On 24 Mar 2020, at 6:40 PM, Satish Balay via petsc-dev <petsc-dev(a)mcs.anl.gov> wrote: > > > > I just attempted a build on osx - and didn't see any such hang. I'm not sure why your build is hanging.. > > > > balay@kpro petsc % ./configure --download-eigen --download-pragmatic --download-metis --download-mpich > > > > Satish > > > > On Tue, 24 Mar 2020, Jacob Faibussowitsch wrote: > > > >> Hello all, > >> > >> Trying to install pragmatic using petsc, which requires Eigen package. I am installing these packages using arch/lib/petsc/conf/reconfigure —download-eigen —download-pragmatic. This is as far as it gets: > >> > >> /reconfigure-64-arch-darwin-c-debug.py --download-eigen > >> =============================================================================== > >> Configuring PETSc to compile on your system > >> =============================================================================== > >> =============================================================================== > >> Trying to download git://https://gitlab.com/libeigen/eigen for EIGEN =============================================================================== > >> =============================================================================== > >> Configuring EIGEN with cmake, this may take several minutes =============================================================================== > >> > >> I have let this run for ~30 minute periods 3 times and it never progresses. I think that it does finish configure, but never progresses to the make stage. > >> > >> I have checked htop (brew version of top) and the only relevant processes running are bash with ~5% CPU. This is less robust but on my setup the window bar (where the close, minimize, etc buttons are) of the terminal window also reports the currently running process and this also only shows bash. For reference when configure is running normally, the top bar switches between clang, ld, expr, cmake, g++, etc. System resources used usually shoots up to ~40-90% as it gets into the different configures and makes for each package. > >> > >> I have tried this on 2 different petsc configurations; --with-64-bit-indices=0/1, since this originally didn’t work with 64 bit indices so I figured that may be the issue. I have also cleared my ccache completely using ccache -c -C and rerun both configures. > >> > >> I have attached both configure.logs, note configure.log —> —with-64-bit-indices=1, configure.log.bkp —> —with-64-bit-indices=0 > >> > >> > >> Best regards, > >> > >> Jacob Faibussowitsch > >> (Jacob Fai - booss - oh - vitch) > >> Cell: (312) 694-3391 > >> > >

1 0

Re: [petsc-dev] Configure --download-eigen hangs
by Pierre Jolivet 24 Mar '20

24 Mar '20

FWIW, I’ve had the same issue as Jacob for quite a while and I’ve since been using brew install eigen to bypass this. I’ve just checked again, and look at one of the process spawned during ./configure. That doesn’t look alright to me. Thanks, Pierre > On 24 Mar 2020, at 6:40 PM, Satish Balay via petsc-dev <petsc-dev(a)mcs.anl.gov> wrote: > > I just attempted a build on osx - and didn't see any such hang. I'm not sure why your build is hanging.. > > balay@kpro petsc % ./configure --download-eigen --download-pragmatic --download-metis --download-mpich > > Satish > > On Tue, 24 Mar 2020, Jacob Faibussowitsch wrote: > >> Hello all, >> >> Trying to install pragmatic using petsc, which requires Eigen package. I am installing these packages using arch/lib/petsc/conf/reconfigure —download-eigen —download-pragmatic. This is as far as it gets: >> >> /reconfigure-64-arch-darwin-c-debug.py --download-eigen >> =============================================================================== >> Configuring PETSc to compile on your system >> =============================================================================== >> =============================================================================== >> Trying to download git://https://gitlab.com/libeigen/eigen for EIGEN =============================================================================== >> =============================================================================== >> Configuring EIGEN with cmake, this may take several minutes =============================================================================== >> >> I have let this run for ~30 minute periods 3 times and it never progresses. I think that it does finish configure, but never progresses to the make stage. >> >> I have checked htop (brew version of top) and the only relevant processes running are bash with ~5% CPU. This is less robust but on my setup the window bar (where the close, minimize, etc buttons are) of the terminal window also reports the currently running process and this also only shows bash. For reference when configure is running normally, the top bar switches between clang, ld, expr, cmake, g++, etc. System resources used usually shoots up to ~40-90% as it gets into the different configures and makes for each package. >> >> I have tried this on 2 different petsc configurations; --with-64-bit-indices=0/1, since this originally didn’t work with 64 bit indices so I figured that may be the issue. I have also cleared my ccache completely using ccache -c -C and rerun both configures. >> >> I have attached both configure.logs, note configure.log —> —with-64-bit-indices=1, configure.log.bkp —> —with-64-bit-indices=0 >> >> >> Best regards, >> >> Jacob Faibussowitsch >> (Jacob Fai - booss - oh - vitch) >> Cell: (312) 694-3391 >>

1 0