August 2018 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Gathering information: types of interpolations in PETSc and which ones are supported?
by Lawrence Mitchell 31 Aug '18

31 Aug '18

> On 31 Aug 2018, at 20:52, Matthew Knepley <knepley(a)gmail.com> wrote: > > Yes. Unfortunately, the same word is used by topology people. I am willing to change this since very few people use it, when they do its > only used once, and its completely different. It refers to figuring out the edges and faces automatically in a mesh when you get only cells > and vertices. Better name? DMPlexCompleteToplogy? Lawrence

1 0

Re: [petsc-dev] Gathering information: types of interpolations in PETSc and which ones are supported?
by Smith, Barry F. 31 Aug '18

31 Aug '18

> On Aug 31, 2018, at 2:52 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > > On Fri, Aug 31, 2018 at 2:20 PM Smith, Barry F. <bsmith(a)mcs.anl.gov> wrote: > > PETSc developers, > > There are a variety of "interpolation" modules in PETSc but the documentation is scattered (mostly missing). Could everyone who knows anything about the various modules provide a little information about which modes exist as interfaces and which have actual supporting code and expected usage? Anything duplicative? > > 1) nested DM to DM (mesh to mesh) of the same type of DM seems to be supported for DMDA and DMPLEX using DMCreateInterpolation(). But what is DMPlexComputeInterpolatorNested(DM, DM, Mat, void *), how is it different? (Note usr context is not used) > > DMCreateInterpolation() says nothing about nesting, and in fact lets people create arbitrary algebraic interpolation. For DMPlex, we support > both nested and non-nested MG (Patrick says there is a subtle bug in non-nested), but my tests pass for this. The Nested case is obviously much easier and has a fixed element matrix, whereas non-nested uses point location. > > 2) non-nested DM to DM. DMPlexComputeInterpolatorGeneral(DM, DM, Mat, void *); (and what does "local portion" mean?) (also the usr context? is not used). > > We can take out "local portion". That was me talking to myself. Why do DMPlexComputeInterpolatorNested() and DMPlexComputeInterpolatorGeneral() exist if DMCreateInterpolation() handles both nested and non-nested? Does DMCreateInterpolation() use them? If so they should be made private and there name changed: for example DMCreateInterpolation_Plex_Nested() and DMCreateInterpolation_Plex_General(). Having multiple interfaces with different names that do the same thing is very confusing and not the way we do things in PETSc. > > 3) DM to a set of points (mesh to points) with DMInterpolationInfo and the routines DMInterpolationEvaluate() etc. Is this fully implemented for DMPLEX, DMDA? > > This relies on the DMLocatePoints(), which I think was implemented for both, but if it isn't for DMDA, its trivial and we should do it. > > Parallel, does the user need to know which process the points are on or is that all figured out? > > That is figured out. Actually, its most of the reason that this exists. However, I wrote this a long time ago. It should go away in favor of DMSwarm. > > 4) Points to a DM. Is this supported (should be?) by DMSWARM? In fact should 3) work with DMSWARM as the set of points and not have its own construct (DMInterpolationInfo)? > > This is supported. However, what is currently in there is custom code Dave wrote which only works for P1. We now have code > that does this for any element and in all dimensions. Its in a branch and will get merged shortly (next week) since it passes all > tests. However, we need some more time to fully integrate it into the interface currently in DMSwarm. > > Another problem is that Dave has a different definition of points than the plasma people. We need a nice way > to switch between these perspectives when doing interpolation, which is what will take us a little time when we are integrating. > > 5) Points to points? (Done indirectly by interpolating to a DM then back to the other points)? > > Yes, that is how I would do it. > > There is a routine DMPlexInterpolate() is this mis-named/confusing thing? Interpolate seems to mean something slight different here. > > Yes. Unfortunately, the same word is used by topology people. I am willing to change this since very few people use it, when they do its > only used once, and its completely different. It refers to figuring out the edges and faces automatically in a mesh when you get only cells > and vertices. Better name? > > Thanks, > > Matt > > All of this explanation could go into the users manual (or FAQ for now). > > Thanks for any explanations, > > Barry > > > > > > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/

1 0

Re: [petsc-dev] Gathering information: types of interpolations in PETSc and which ones are supported?
by Matthew Knepley 31 Aug '18

31 Aug '18

On Fri, Aug 31, 2018 at 2:20 PM Smith, Barry F. <bsmith(a)mcs.anl.gov> wrote: > > PETSc developers, > > There are a variety of "interpolation" modules in PETSc but the > documentation is scattered (mostly missing). Could everyone who knows > anything about the various modules provide a little information about which > modes exist as interfaces and which have actual supporting code and > expected usage? Anything duplicative? > > 1) nested DM to DM (mesh to mesh) of the same type of DM seems to be > supported for DMDA and DMPLEX using DMCreateInterpolation(). But what is > DMPlexComputeInterpolatorNested(DM, DM, Mat, void *), how is it different? > (Note usr context is not used) > DMCreateInterpolation() says nothing about nesting, and in fact lets people create arbitrary algebraic interpolation. For DMPlex, we support both nested and non-nested MG (Patrick says there is a subtle bug in non-nested), but my tests pass for this. The Nested case is obviously much easier and has a fixed element matrix, whereas non-nested uses point location. > 2) non-nested DM to DM. DMPlexComputeInterpolatorGeneral(DM, DM, Mat, void > *); (and what does "local portion" mean?) (also the usr context? is not > used). > We can take out "local portion". That was me talking to myself. > 3) DM to a set of points (mesh to points) with DMInterpolationInfo and > the routines DMInterpolationEvaluate() etc. Is this fully implemented for > DMPLEX, DMDA? This relies on the DMLocatePoints(), which I think was implemented for both, but if it isn't for DMDA, its trivial and we should do it. > Parallel, does the user need to know which process the points are on or is > that all figured out? > That is figured out. Actually, its most of the reason that this exists. However, I wrote this a long time ago. It should go away in favor of DMSwarm. > 4) Points to a DM. Is this supported (should be?) by DMSWARM? In fact > should 3) work with DMSWARM as the set of points and not have its own > construct (DMInterpolationInfo)? > This is supported. However, what is currently in there is custom code Dave wrote which only works for P1. We now have code that does this for any element and in all dimensions. Its in a branch and will get merged shortly (next week) since it passes all tests. However, we need some more time to fully integrate it into the interface currently in DMSwarm. Another problem is that Dave has a different definition of points than the plasma people. We need a nice way to switch between these perspectives when doing interpolation, which is what will take us a little time when we are integrating. > 5) Points to points? (Done indirectly by interpolating to a DM then back > to the other points)? > Yes, that is how I would do it. > There is a routine DMPlexInterpolate() is this mis-named/confusing thing? > Interpolate seems to mean something slight different here. > Yes. Unfortunately, the same word is used by topology people. I am willing to change this since very few people use it, when they do its only used once, and its completely different. It refers to figuring out the edges and faces automatically in a mesh when you get only cells and vertices. Better name? Thanks, Matt > All of this explanation could go into the users manual (or FAQ for now). > > Thanks for any explanations, > > Barry > > > > > > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Gathering information: types of interpolations in PETSc and which ones are supported?
by Smith, Barry F. 31 Aug '18

31 Aug '18

PETSc developers, There are a variety of "interpolation" modules in PETSc but the documentation is scattered (mostly missing). Could everyone who knows anything about the various modules provide a little information about which modes exist as interfaces and which have actual supporting code and expected usage? Anything duplicative? 1) nested DM to DM (mesh to mesh) of the same type of DM seems to be supported for DMDA and DMPLEX using DMCreateInterpolation(). But what is DMPlexComputeInterpolatorNested(DM, DM, Mat, void *), how is it different? (Note usr context is not used) 2) non-nested DM to DM. DMPlexComputeInterpolatorGeneral(DM, DM, Mat, void *); (and what does "local portion" mean?) (also the usr context? is not used). 3) DM to a set of points (mesh to points) with DMInterpolationInfo and the routines DMInterpolationEvaluate() etc. Is this fully implemented for DMPLEX, DMDA? Parallel, does the user need to know which process the points are on or is that all figured out? 4) Points to a DM. Is this supported (should be?) by DMSWARM? In fact should 3) work with DMSWARM as the set of points and not have its own construct (DMInterpolationInfo)? 5) Points to points? (Done indirectly by interpolating to a DM then back to the other points)? There is a routine DMPlexInterpolate() is this mis-named/confusing thing? Interpolate seems to mean something slight different here. All of this explanation could go into the users manual (or FAQ for now). Thanks for any explanations, Barry

1 0

Re: [petsc-dev] Compiling PETsc against openblas on Linux
by Satish Balay 31 Aug '18

31 Aug '18

It should work. configure.log will have details.. Satish ---------- balay@asterix /home/balay/petsc (maint=) $ ./configure --with-blaslapack-lib=-lopenblas =============================================================================== Configuring PETSc to compile on your system =============================================================================== =============================================================================== It appears you do not have valgrind installed on your system. We HIGHLY recommend you install it from www.valgrind.org Or install valgrind-devel or equivalent using your package manager. Then rerun ./configure =============================================================================== Compilers: C Compiler: mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g3 C++ Compiler: mpicxx -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g Fortran Compiler: mpif90 -Wall -ffree-line-length-0 -Wno-unused-dummy-argument -g Linkers: Shared linker: mpicc -shared -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g3 Dynamic linker: mpicc -shared -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g3 make: BLAS/LAPACK: -lopenblas MPI: cmake: X: Library: -lX11 c2html: Arch: pthread: Library: -lpthread PETSc: PETSC_ARCH: arch-linux2-c-debug PETSC_DIR: /home/balay/petsc Scalar type: real Precision: double Clanguage: C Integer size: 32 shared libraries: enabled Memory alignment: 16 xxx=========================================================================xxx Configure stage complete. Now build PETSc libraries with (gnumake build): make PETSC_DIR=/home/balay/petsc PETSC_ARCH=arch-linux2-c-debug all xxx=========================================================================xxx balay@asterix /home/balay/petsc (maint=) $ On Fri, 31 Aug 2018, Antonio wrote: > It's failing for missing (not required) links: > > Executing: mpicc -o /tmp/petsc-FYncGY/config.libraries/conftest > -Wl,-z,relro -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g3 > /tmp/petsc-FYncGY/config.libraries/conftest.o -lsunperf -lfui -lfsu > -lsunmath -lm -lstdc++ -ldl -L/usr/lib64/openmpi/lib -lmpi_usempif08 > -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm > -L/usr/lib/gcc/x86_64-redhat-linux/8 -L/usr/lib64/openmpi/lib -lgfortran > -lm -lgcc_s -lquadmath -lpthread -lstdc++ -ldl > Possible ERROR while running linker: exit code 256 > stderr: > /usr/bin/ld: cannot find -lsunperf > /usr/bin/ld: cannot find -lfui > /usr/bin/ld: cannot find -lfsu > /usr/bin/ld: cannot find -lsunmath > collect2: error: ld returned 1 exit status > Popping language C > Unknown name mangling in BLAS/LAPACK > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL/9.0 > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL/8.1.1 > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL/8.1 > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL/8.0.1 > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL/8.0 > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL72 > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL70 > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL61 > MKL Path not found.. skipping: /cygdrive/c/Program > Files/Intel/MKL > **** Configure header /tmp/petsc-FYncGY/confdefs.h **** > > On 31/08/2018 19:41, Smith, Barry F. wrote: > > > > Ah, sorry for the confusion. We do support --download-openblas where PETSc will download and install and OpenBlas for you but if you have it already installed then you need to treat it just as a BLASLAPACK library so use > > > > > >> --with-blaslapack-include=%{_includedir} \ > >> --with-blaslapack-lib=%{_libdir}/libopenblas.so \ > >> --known-64-bit-blas-indices=0 \ > > > > If this fails then mail configure.log to petsc-maint(a)mcs.anl.gov > > > > Barry > > > >

1 0

Re: [petsc-dev] Compiling PETsc against openblas on Linux
by Antonio 31 Aug '18

31 Aug '18

It's failing for missing (not required) links: Executing: mpicc -o /tmp/petsc-FYncGY/config.libraries/conftest -Wl,-z,relro -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g3 /tmp/petsc-FYncGY/config.libraries/conftest.o -lsunperf -lfui -lfsu -lsunmath -lm -lstdc++ -ldl -L/usr/lib64/openmpi/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -L/usr/lib/gcc/x86_64-redhat-linux/8 -L/usr/lib64/openmpi/lib -lgfortran -lm -lgcc_s -lquadmath -lpthread -lstdc++ -ldl Possible ERROR while running linker: exit code 256 stderr: /usr/bin/ld: cannot find -lsunperf /usr/bin/ld: cannot find -lfui /usr/bin/ld: cannot find -lfsu /usr/bin/ld: cannot find -lsunmath collect2: error: ld returned 1 exit status Popping language C Unknown name mangling in BLAS/LAPACK MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL/9.0 MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL/8.1.1 MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL/8.1 MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL/8.0.1 MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL/8.0 MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL72 MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL70 MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL61 MKL Path not found.. skipping: /cygdrive/c/Program Files/Intel/MKL **** Configure header /tmp/petsc-FYncGY/confdefs.h **** On 31/08/2018 19:41, Smith, Barry F. wrote: > > Ah, sorry for the confusion. We do support --download-openblas where PETSc will download and install and OpenBlas for you but if you have it already installed then you need to treat it just as a BLASLAPACK library so use > > >> --with-blaslapack-include=%{_includedir} \ >> --with-blaslapack-lib=%{_libdir}/libopenblas.so \ >> --known-64-bit-blas-indices=0 \ > > If this fails then mail configure.log to petsc-maint(a)mcs.anl.gov > > Barry > -- --- Antonio Trande Fedora Project mailto 'sagitter at fedoraproject dot org' GPG key: 0x5E212EE1D35568BE GPG key server: https://keys.fedoraproject.org/

1 0

Re: [petsc-dev] Compiling PETsc against openblas on Linux
by Satish Balay 31 Aug '18

31 Aug '18

You would use: --with-blas-lapack-lib=%{_libdir}/libopenblas.so The options --with-openblas-include --with-openblas-lib etc are a bit misleading [due to the way some of the blas packages ae organized] Satish On Fri, 31 Aug 2018, Antonio Trande wrote: > Hello everyone. > > How to configure correctly PETsc compilation against OpenBLAS? > > 'openblas' libraries are > > /usr/include/openblas/* > /usr/lib(64)/libopenblas.so > /usr/lib(64)/libopenblas.a > /usr/lib(64)/libopenblas64.so > > If i use > > .. > --with-openblas=1 \ > --with-openblas-include=%{_includedir} \ > --with-openblas-lib=%{_libdir}/libopenblas.so \ > --known-64-bit-blas-indices=0 \ > .. > > Configure fails with > > TEST configureLibrary from > config.packages.openblas(/home/sagitter/rpmbuild/BUILD/petsc-3.9.3/petsc-3.9.3/config/BuildSystem/config/packages/openblas.py:25) > TESTING: configureLibrary from > config.packages.openblas(config/BuildSystem/config/packages/openblas.py:25) > ================================================================================== > Checking for a functional openblas > Not checking for library in User specified OPENBLAS > libraries: [] because no functions given to check for > ================================================================================ > TEST check from > config.libraries(/home/sagitter/rpmbuild/BUILD/petsc-3.9.3/petsc-3.9.3/config/BuildSystem/config/libraries.py:155) > TESTING: check from > config.libraries(config/BuildSystem/config/libraries.py:155) > Checks that the library "libName" contains "funcs", and if it does > defines HAVE_LIB"libName" > - libDir may be a list of directories > - libName may be a list of library names > No functions to check for in library [] [] > Checking for headers User specified OPENBLAS libraries: [] > **** Configure header /tmp/petsc-y1LlML/confdefs.h **** > >

1 0

Re: [petsc-dev] Compiling PETsc against openblas on Linux
by Smith, Barry F. 31 Aug '18

31 Aug '18

Ah, sorry for the confusion. We do support --download-openblas where PETSc will download and install and OpenBlas for you but if you have it already installed then you need to treat it just as a BLASLAPACK library so use > --with-blaslapack-include=%{_includedir} \ > --with-blaslapack-lib=%{_libdir}/libopenblas.so \ > --known-64-bit-blas-indices=0 \ If this fails then mail configure.log to petsc-maint(a)mcs.anl.gov Barry > On Aug 31, 2018, at 12:16 PM, Antonio Trande <anto.trande(a)gmail.com> wrote: > > Hello everyone. > > How to configure correctly PETsc compilation against OpenBLAS? > > 'openblas' libraries are > > /usr/include/openblas/* > /usr/lib(64)/libopenblas.so > /usr/lib(64)/libopenblas.a > /usr/lib(64)/libopenblas64.so > > If i use > > .. > --with-openblas=1 \ > --with-openblas-include=%{_includedir} \ > --with-openblas-lib=%{_libdir}/libopenblas.so \ > --known-64-bit-blas-indices=0 \ > .. > > Configure fails with > > TEST configureLibrary from > config.packages.openblas(/home/sagitter/rpmbuild/BUILD/petsc-3.9.3/petsc-3.9.3/config/BuildSystem/config/packages/openblas.py:25) > TESTING: configureLibrary from > config.packages.openblas(config/BuildSystem/config/packages/openblas.py:25) > ================================================================================== > Checking for a functional openblas > Not checking for library in User specified OPENBLAS > libraries: [] because no functions given to check for > ================================================================================ > TEST check from > config.libraries(/home/sagitter/rpmbuild/BUILD/petsc-3.9.3/petsc-3.9.3/config/BuildSystem/config/libraries.py:155) > TESTING: check from > config.libraries(config/BuildSystem/config/libraries.py:155) > Checks that the library "libName" contains "funcs", and if it does > defines HAVE_LIB"libName" > - libDir may be a list of directories > - libName may be a list of library names > No functions to check for in library [] [] > Checking for headers User specified OPENBLAS libraries: [] > **** Configure header /tmp/petsc-y1LlML/confdefs.h **** > > -- > --- > Antonio Trande > Fedora Project > mailto 'sagitter at fedoraproject dot org' > GPG key: 0x5E212EE1D35568BE > GPG key server: https://keys.fedoraproject.org/

1 0

Compiling PETsc against openblas on Linux
by Antonio Trande 31 Aug '18

31 Aug '18

Hello everyone. How to configure correctly PETsc compilation against OpenBLAS? 'openblas' libraries are /usr/include/openblas/* /usr/lib(64)/libopenblas.so /usr/lib(64)/libopenblas.a /usr/lib(64)/libopenblas64.so If i use .. --with-openblas=1 \ --with-openblas-include=%{_includedir} \ --with-openblas-lib=%{_libdir}/libopenblas.so \ --known-64-bit-blas-indices=0 \ .. Configure fails with TEST configureLibrary from config.packages.openblas(/home/sagitter/rpmbuild/BUILD/petsc-3.9.3/petsc-3.9.3/config/BuildSystem/config/packages/openblas.py:25) TESTING: configureLibrary from config.packages.openblas(config/BuildSystem/config/packages/openblas.py:25) ================================================================================== Checking for a functional openblas Not checking for library in User specified OPENBLAS libraries: [] because no functions given to check for ================================================================================ TEST check from config.libraries(/home/sagitter/rpmbuild/BUILD/petsc-3.9.3/petsc-3.9.3/config/BuildSystem/config/libraries.py:155) TESTING: check from config.libraries(config/BuildSystem/config/libraries.py:155) Checks that the library "libName" contains "funcs", and if it does defines HAVE_LIB"libName" - libDir may be a list of directories - libName may be a list of library names No functions to check for in library [] [] Checking for headers User specified OPENBLAS libraries: [] **** Configure header /tmp/petsc-y1LlML/confdefs.h **** -- --- Antonio Trande Fedora Project mailto 'sagitter at fedoraproject dot org' GPG key: 0x5E212EE1D35568BE GPG key server: https://keys.fedoraproject.org/

1 0

Re: [petsc-dev] PETSc 3.10: September 4, 2018, is the cutoff-date for new features
by Karl Rupp 30 Aug '18

30 Aug '18

Dear PETSc developers, this is a gentle reminder for the cutoff-date on September 4. Best regards, Karli On 07/27/2018 02:41 AM, Karl Rupp wrote: > Dear PETSc developers, > > in order to ensure a PETSc 3.10 release no later than by the end of > September (possibly earlier), we agreed on September 4, 2018, as the > cut-off date for new features. Please make sure that a pull request has > been opened on Bitbucket by this time. This is the preferred model over > merging to next directly, since our Jenkins test infrastructure will > test each PR separately (thus providing faster feedback and thus allow > for faster integration). > > And while we are at it: Please keep in mind that new features also > require appropriate documentation. :-) > > Thanks and best regards, > Karli

1 0