March 2020 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Question about Binary-IO in READ mode with POSIX APIs
by Jed Brown 16 Mar '20

16 Mar '20

Lisandro Dalcin <dalcinl(a)gmail.com> writes: > Currently, binary viewers using POSIX file descriptors with READ mode open > the file in ALL processes in the communicator. For WRITE mode, only process > zero opens the file. > > The current PetscViewerBynaryXXX APIs make it really unnecessary to open > the file in all processes for READ. I would like to get rid of that and > always open on rank 0 for both READ or WROTE. I think we should use MPI-IO by default, and advise that people use it whenever they can. I'm not sure of this suggested change, in that a "bad for MPI-IO" workload (like each rank randomly seeking around a big file) might not be better with rank 0 acting as a service rank. > Such change may impact third-party code using low-level IO. However, > note that after MR 2594 > https://gitlab.com/petsc/petsc/-/merge_requests/2594 gets merged, > third party code should hardly need to access the file > descriptors/MPI_File to perform IO (this would require some minor and > relatively easy improvements to subviewers that I'm planning). > > In short, I would like to change binary viewers in READ mode using POSIX > file descriptors to open files ONLY in rank 0 rather than ALL ranks as it > is now. Any objections? > > PS: After all the enhancements to binary viewers, we should consider > whether we should turn MPI-IO on by default. > > -- > Lisandro Dalcin > ============ > Research Scientist > Extreme Computing Research Center (ECRC) > King Abdullah University of Science and Technology (KAUST) > http://ecrc.kaust.edu.sa/

1 0

Question about Binary-IO in READ mode with POSIX APIs
by Lisandro Dalcin 16 Mar '20

16 Mar '20

Currently, binary viewers using POSIX file descriptors with READ mode open the file in ALL processes in the communicator. For WRITE mode, only process zero opens the file. The current PetscViewerBynaryXXX APIs make it really unnecessary to open the file in all processes for READ. I would like to get rid of that and always open on rank 0 for both READ or WROTE. Such change may impact third-party code using low-level IO. However, note that after MR 2594 https://gitlab.com/petsc/petsc/-/merge_requests/2594 gets merged, third party code should hardly need to access the file descriptors/MPI_File to perform IO (this would require some minor and relatively easy improvements to subviewers that I'm planning). In short, I would like to change binary viewers in READ mode using POSIX file descriptors to open files ONLY in rank 0 rather than ALL ranks as it is now. Any objections? PS: After all the enhancements to binary viewers, we should consider whether we should turn MPI-IO on by default. -- Lisandro Dalcin ============ Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/

1 0

Re: [petsc-dev] Truly minimal configure
by Jed Brown 15 Mar '20

15 Mar '20

Patrick Sanan <patrick.sanan(a)gmail.com> writes: >> At a high level, we see here that the cost of make allcite is dominated >> by recursive make tree traversal. >> >> Overhead Command Shared Object >> 35.81% sh libc-2.31.so >> 13.31% make make >> 11.18% make libc-2.31.so >> 7.63% sh ld-2.31.so >> 5.84% make ld-2.31.so >> 4.20% doctext doctext >> 4.07% doctext libc-2.31.so >> 3.88% doctext ld-2.31.so >> 3.64% sh bash >> 2.79% rm libc-2.31.so >> 1.99% rm ld-2.31.so >> 1.56% sh [unknown] >> 1.14% doctext [unknown] >> 0.56% make [unknown] >> 0.55% python libpython3.8.so.1.0 > How'd you get this timing? $ perf record make PETSC_ARCH=uni-none allcite LOC=$PETSC_DIR/dochtmlmap DOCTEXT=doctext [...] $ perf report --sort=comm,dso

1 0

Re: [petsc-dev] Truly minimal configure
by Patrick Sanan 15 Mar '20

15 Mar '20

> Am 15.03.2020 um 16:06 schrieb Jed Brown <jed(a)jedbrown.org>: > > Patrick Sanan <patrick.sanan(a)gmail.com> writes: > >> I want to generate docs/manualpages/htmlmap as quickly as possible, from scratch (a clone on ReadTheDocs), so I want a fast configure which will let me run "make allcite". >> >> The below will work, I think, but I'm curious whether there's an even faster known way. >> >> Also note that I had to explicitly turn off some MKL stuff, because it's included by default and depends on blaslapack. >> >> >> './configure', >> '--with-mpi=0', >> '--with-blaslapack=0', >> '--with-fortran=0', >> '--with-cxx=0', >> '--with-mkl_sparse_optimize=0', >> '--with-mkl_sparse=0', > > These checks should probably be conditional on having found blaslapack, > but you don't have a working PETSc this way (it'll fail to link > examples), and as far as I can tell, you're just using configure to > install sowing and not need to hack the makefiles to do the tree > traversal without including $PETSC_ARCH/lib/petsc/conf/petscvariables. > > This seems fine and we can eventually move to calling doctext from > Python or doing its job in Python (as part of the last step, moving man > pages to Sphinx). > > At a high level, we see here that the cost of make allcite is dominated > by recursive make tree traversal. > > Overhead Command Shared Object > 35.81% sh libc-2.31.so > 13.31% make make > 11.18% make libc-2.31.so > 7.63% sh ld-2.31.so > 5.84% make ld-2.31.so > 4.20% doctext doctext > 4.07% doctext libc-2.31.so > 3.88% doctext ld-2.31.so > 3.64% sh bash > 2.79% rm libc-2.31.so > 1.99% rm ld-2.31.so > 1.56% sh [unknown] > 1.14% doctext [unknown] > 0.56% make [unknown] > 0.55% python libpython3.8.so.1.0 How'd you get this timing?

1 0

Re: [petsc-dev] Truly minimal configure
by Jed Brown 15 Mar '20

15 Mar '20

Satish Balay via petsc-dev <petsc-dev(a)mcs.anl.gov> writes: > On Sun, 15 Mar 2020, Patrick Sanan wrote: > >> I want to generate docs/manualpages/htmlmap as quickly as possible, from scratch (a clone on ReadTheDocs), so I want a fast configure which will let me run "make allcite". >> >> The below will work, I think, but I'm curious whether there's an even faster known way. >> >> Also note that I had to explicitly turn off some MKL stuff, because it's included by default and depends on blaslapack. >> >> >> './configure', >> '--with-mpi=0', >> '--with-blaslapack=0', > > I'm surprised this works. I would just do --with-blaslapack-lib='-llapack -lblas' - assuming system blas/lapack are installed. > [Similarly with other option that are known values --with-cc=gcc etc..] > > Also you would need sowing,c2html for some of the docs [perhaps not allcite] - the fastest way likely is to have them in PATH Patrick is using --download-sowing to build on readthedocs.org (see https://docs.petsc.org/en/psanan-docs-sphinx-dev-manual/) He has no intent to build PETSc, only to build the man page index. https://gitlab.com/petsc/petsc/-/merge_requests/2598/diffs#8c8ccef5b7d76705…

1 0

Re: [petsc-dev] Truly minimal configure
by Satish Balay 15 Mar '20

15 Mar '20

On Sun, 15 Mar 2020, Patrick Sanan wrote: > I want to generate docs/manualpages/htmlmap as quickly as possible, from scratch (a clone on ReadTheDocs), so I want a fast configure which will let me run "make allcite". > > The below will work, I think, but I'm curious whether there's an even faster known way. > > Also note that I had to explicitly turn off some MKL stuff, because it's included by default and depends on blaslapack. > > > './configure', > '--with-mpi=0', > '--with-blaslapack=0', I'm surprised this works. I would just do --with-blaslapack-lib='-llapack -lblas' - assuming system blas/lapack are installed. [Similarly with other option that are known values --with-cc=gcc etc..] Also you would need sowing,c2html for some of the docs [perhaps not allcite] - the fastest way likely is to have them in PATH > '--with-fortran=0', > '--with-cxx=0', > '--with-mkl_sparse_optimize=0', > '--with-mkl_sparse=0', > And disable the some of the stuff that might be auto-enabled.. config/BuildSystem/config/packages/BlasLapack.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/MPI.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/X.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/c2html.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/cmake.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/cxxlibs.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/flibs.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/mathlib.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/mkl_sparse.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/mkl_sparse_optimize.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/pthread.py: self.lookforbydefault = 1 config/BuildSystem/config/packages/regex.py: self.lookforbydefault = 1 i.e ./configure --with-cc=gcc --with-fc=0 --with-cxx=0 --with-mpi=0 --with-blaslapack-lib='-llapack -lblas' --with-c2html=1 --with-sowing=1 --with-x=0 --with-cmake=0 --with-pthread=0 --with-regexp=0 However there is some dependency between sowing and fortranstubs [perhaps this needs fixing] - due to this - we still need --with-fc=gfortran Satish

1 0

Re: [petsc-dev] Truly minimal configure
by Jed Brown 15 Mar '20

15 Mar '20

Patrick Sanan <patrick.sanan(a)gmail.com> writes: > I want to generate docs/manualpages/htmlmap as quickly as possible, from scratch (a clone on ReadTheDocs), so I want a fast configure which will let me run "make allcite". > > The below will work, I think, but I'm curious whether there's an even faster known way. > > Also note that I had to explicitly turn off some MKL stuff, because it's included by default and depends on blaslapack. > > > './configure', > '--with-mpi=0', > '--with-blaslapack=0', > '--with-fortran=0', > '--with-cxx=0', > '--with-mkl_sparse_optimize=0', > '--with-mkl_sparse=0', These checks should probably be conditional on having found blaslapack, but you don't have a working PETSc this way (it'll fail to link examples), and as far as I can tell, you're just using configure to install sowing and not need to hack the makefiles to do the tree traversal without including $PETSC_ARCH/lib/petsc/conf/petscvariables. This seems fine and we can eventually move to calling doctext from Python or doing its job in Python (as part of the last step, moving man pages to Sphinx). At a high level, we see here that the cost of make allcite is dominated by recursive make tree traversal. Overhead Command Shared Object 35.81% sh libc-2.31.so 13.31% make make 11.18% make libc-2.31.so 7.63% sh ld-2.31.so 5.84% make ld-2.31.so 4.20% doctext doctext 4.07% doctext libc-2.31.so 3.88% doctext ld-2.31.so 3.64% sh bash 2.79% rm libc-2.31.so 1.99% rm ld-2.31.so 1.56% sh [unknown] 1.14% doctext [unknown] 0.56% make [unknown] 0.55% python libpython3.8.so.1.0

1 0

Truly minimal configure
by Patrick Sanan 15 Mar '20

15 Mar '20

I want to generate docs/manualpages/htmlmap as quickly as possible, from scratch (a clone on ReadTheDocs), so I want a fast configure which will let me run "make allcite". The below will work, I think, but I'm curious whether there's an even faster known way. Also note that I had to explicitly turn off some MKL stuff, because it's included by default and depends on blaslapack. './configure', '--with-mpi=0', '--with-blaslapack=0', '--with-fortran=0', '--with-cxx=0', '--with-mkl_sparse_optimize=0', '--with-mkl_sparse=0',

1 0

No PETSc User Meeting in 2020
by Karl Rupp 14 Mar '20

14 Mar '20

Dear PETSc developers and PETSc users, due to the recent Covid-19 outbreak in Europe there will not be a PETSc User Meeting this year. We are looking into alternatives for keeping in touch with our user base, e.g. via webinars. Suggestions welcome :-) Thanks and best regards, Karl

1 0

Re: [petsc-dev] petsc release plan for march/2020
by Jed Brown 13 Mar '20

13 Mar '20

Satish Balay <balay(a)mcs.anl.gov> writes: >> > Merges after freeze should contain only fixes that would normally be acceptable to maint workflow. >> > >> > I've changed the current milestone 'v3.13' to 'master' and created a new one 'v3.13-release' >> >> Why this way? Is the idea to bulk-offload everything we had intended to >> get in this release? I feel like we should ping those merge >> requests/issues asking people to either commit to completing them or >> defer to post-release. > > Currently there are about 50 MRs - likely most of them likely inactive. > > My thought is to make the default 'mater' (i.e post release) - and use 'v3.13-release' for MRs we are actively working for the release. The other way of identifying inactive MRs or the ones not destined for release (and marking them as such) might not be easy. And usually the MR author is the best judge of this (inactive, active for release, active for later). And if active - can easily switch milestore to 'v3.13-release' Only 19 of which had set the milestone (now called 'master'). > Sure - we should check with MR authors about this. In essence - this e-mail is part of that check. > > BTW: I think we should work on getting as many MRs into release as feasible - but not scramble to get everything in. Yep, agreed.

1 0