July 2013 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Deprecated PETSc functions and Fortran codes?
by Jed Brown 31 Jul '13

31 Jul '13

Richard Tran Mills <rtm(a)eecs.utk.edu> writes: > Hi Folks, > > Is there a way to deprecate PETSc functions that will work with Fortran > codes? In include/petscdmda.h I have > > PETSC_DEPRECATED("Use DMLocalToLocalBegin()") PETSC_STATIC_INLINE > PetscErrorCode DMDALocalToLocalBegin(DM dm,Vec g,InsertMode mode,Vec l) > {return DMLocalToLocalBegin(dm,g,mode,l);} I am not aware of any way to do something similar in Fortran, for any specific vendor. If you figure out how, we'd be happy to add it.

1 0

Re: [petsc-dev] Segfault in DMPlexDistribute
by Chris Eldred 31 Jul '13

31 Jul '13

That works great, thanks! -Chris On Wed, Jul 31, 2013 at 4:59 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Chris Eldred <chris.eldred(a)gmail.com> writes: > > if (rank==0) then > > call DMPlexCreateFromCellList(PETSC_COMM_WORLD, 2, 2, 4, 3, 1 , > cell_list > > , 2 , vertex_coords , dm, ierr) > > The problem here is that after this line, dm is garbage on rank 1. You > want to start by creating the DM on COMM_WORLD, but have all the cells > on rank 0. Try the attached. > > -- Chris Eldred DOE Computational Science Graduate Fellow Graduate Student, Atmospheric Science, Colorado State University B.S. Applied Computational Physics, Carnegie Mellon University, 2009 chris.eldred(a)gmail.com / celdred(a)atmos.colostate.edu

1 0

Re: [petsc-dev] Segfault in DMPlexDistribute
by Jed Brown 31 Jul '13

31 Jul '13

Chris Eldred <chris.eldred(a)gmail.com> writes: > if (rank==0) then > call DMPlexCreateFromCellList(PETSC_COMM_WORLD, 2, 2, 4, 3, 1 , cell_list > , 2 , vertex_coords , dm, ierr) The problem here is that after this line, dm is garbage on rank 1. You want to start by creating the DM on COMM_WORLD, but have all the cells on rank 0. Try the attached.

1 0

Re: [petsc-dev] Segfault in DMPlexDistribute
by Barry Smith 31 Jul '13

31 Jul '13

http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind On Jul 31, 2013, at 12:49 PM, Chris Eldred <chris.eldred(a)gmail.com> wrote: > I am getting a segmentation fault when I try to distribute a DMPlex mesh: > > [1]PETSC ERROR: ------------------------------------------------------------------------ > [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range > [1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > [1]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors > [1]PETSC ERROR: likely location of problem given in stack below > [1]PETSC ERROR: --------------------- Stack Frames ------------------------------------ > [1]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, > [1]PETSC ERROR: INSTEAD the line number of the start of the function > [1]PETSC ERROR: is given. > [1]PETSC ERROR: [1] PetscSFCreateEmbeddedSF line 863 /home/user/Desktop/LIBRARIES/petsc/src/vec/is/sf/interface/sf.c > [1]PETSC ERROR: [1] PetscSFDistributeSection line 1755 /home/user/Desktop/LIBRARIES/petsc/src/vec/is/utils/vsectionis.c > [1]PETSC ERROR: [1] DMPlexDistribute line 2771 /home/user/Desktop/LIBRARIES/petsc/src/dm/impls/plex/plex.c > [1]PETSC ERROR: --------------------- Error Message ------------------------------------ > [1]PETSC ERROR: Signal received! > [1]PETSC ERROR: ------------------------------------------------------------------------ > [1]PETSC ERROR: Petsc Development GIT revision: dd831807ccae33b4bba0c6456b55575ff17239c3 GIT Date: 2013-06-03 14:26:06 -0600 > [1]PETSC ERROR: See docs/changes/index.html for recent updates. > [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [1]PETSC ERROR: See docs/index.html for manual pages. > [1]PETSC ERROR: ------------------------------------------------------------------------ > [1]PETSC ERROR: ./test on a arch-linux2-cxx-debug named Puget-101334 by user Wed Jul 31 12:44:44 2013 > [1]PETSC ERROR: Libraries linked from /home/user/Desktop/LIBRARIES/petsc/arch-linux2-cxx-debug/lib > [1]PETSC ERROR: Configure run at Mon Jun 3 14:29:12 2013 > [1]PETSC ERROR: Configure options --download-boost --download-chaco --download-ctetgen --download-f-blas-lapack --download-fiat --download-generator --download-hdf5 --download-metis --download-ml --download-mpich --download-parmetis --download-scientificpython --download-triangle --with-clanguage=cxx --with-dynamic-loading --with-openmp --with-pthreadclasses --with-shared-libraries --with-sieve --with-threadcomm PETSC_ARCH=arch-linux2-cxx-debug > [1]PETSC ERROR: ------------------------------------------------------------------------ > [1]PETSC ERROR: User provided function() line 0 in unknown directory unknown file > application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1 > [cli_1]: aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1 > > I am building the mesh using DMPlexCreateFromCellList but I get the exact same error occurring if I build it using DMPlexSetCone, etc. It is the sample mesh from http://www.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/sieve.pdf (figure 2) so very simple. > > Here is my relevant code: > > cell_list = (/ 0 , 1 , 2 , 1 , 3 , 2 /) > vertex_coords = 0.0 > > call DMPlexCreateFromCellList(PETSC_COMM_WORLD, 2, 2, 4, 3, 1 , cell_list , 2 , vertex_coords , dm, ierr) > CHKERRQ(ierr) > > call DMPlexDistribute(dm,'chaco',0,dm_distrib,ierr) > CHKERRQ(ierr) > > I am running the code with mpirun -np 2 ./test (it also segfaults with higher values for np). > > Any ideas? The code works fine with mpirun -np 1 (presumably since it is not actually attempting to distribute the mesh). > > Thanks, > -Chris Eldred > > -- > Chris Eldred > DOE Computational Science Graduate Fellow > Graduate Student, Atmospheric Science, Colorado State University > B.S. Applied Computational Physics, Carnegie Mellon University, 2009 > chris.eldred(a)gmail.com / celdred(a)atmos.colostate.edu

1 0

Re: [petsc-dev] future of PetscSubcomm
by Barry Smith 31 Jul '13

31 Jul '13

https://bitbucket.org/petsc/petsc/issue/46/replace-petscsubcomm-with-the-in… On Jul 31, 2013, at 10:59 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Barry Smith <bsmith(a)mcs.anl.gov> writes: > >> Jed, >> >> Hong and I were discussing MatGetRedundantMatrix() etc and she >> mention that you don't like PetscSubcomm. I suggested removing the >> PetscSubcomm object from PETSc but retaining the functionality of >> PetscSubcomm by putting the subcomm information into an MPI >> attribute inside an MPI_Comm (so this new comm behaves like a >> MPI_Comm subclass with extra data over a regular MPI_Comm) and >> having something like PetscCommSublimate(comm, …., &subcomm with >> attribute). Then in all the places in PETSc that take PetscSubcomm >> they would instead take a MPI_Comm and check for the attribute and >> use the additional information if it is there. > > I would be inclined to remove dependence on this extra information > whenever possible. For example, I think it is counter-productive in > MatGetRedundantMatrix because it imposes a distribution that has poor > locality. But if we need that extra information, or can do something > better if it is there, I prefer attaching it as an attribute to having a > wrapper (PetscSubcomm). > >> Managing destruction will be tricky due to possible reference >> loops but presumably we can figure out a way to do it. > > What is missing in the MPI attribute callbacks to implement reference > counting? (We already use it for this purpose.)

1 0

Re: [petsc-dev] Segfault in DMPlexDistribute
by Chris Eldred 31 Jul '13

31 Jul '13

I wrapped DMPlexCreateFromCellList so that only rank 0 executes it, now I get this new error for np >= 2 (still works fine with np=1). The new error is: [1]PETSC ERROR: --------------------- Error Message ------------------------------------ [1]PETSC ERROR: Invalid argument! [1]PETSC ERROR: Wrong type of object: Parameter # 1! [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: Petsc Development GIT revision: dd831807ccae33b4bba0c6456b55575ff17239c3 GIT Date: 2013-06-03 14:26:06 -0600 [1]PETSC ERROR: See docs/changes/index.html for recent updates. [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [1]PETSC ERROR: See docs/index.html for manual pages. [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: ./test on a arch-linux2-cxx-debug named Puget-101334 by user Wed Jul 31 16:37:07 2013 [1]PETSC ERROR: Libraries linked from /home/user/Desktop/LIBRARIES/petsc/arch-linux2-cxx-debug/lib [1]PETSC ERROR: Configure run at Mon Jun 3 14:29:12 2013 [1]PETSC ERROR: Configure options --download-boost --download-chaco --download-ctetgen --download-f-blas-lapack --download-fiat --download-generator --download-hdf5 --download-metis --download-ml --download-mpich --download-parmetis --download-scientificpython --download-triangle --with-clanguage=cxx --with-dynamic-loading --with-openmp --with-pthreadclasses --with-shared-libraries --with-sieve --with-threadcomm PETSC_ARCH=arch-linux2-cxx-debug [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: DMPlexDistribute() line 2772 in /home/user/Desktop/LIBRARIES/petsc/src/dm/impls/plex/plex.c Should I be doing some sort of collective instantiation on dm or dm_distrib before I attempt to distribute dm? I tried adding call DMPlexCreate(PETSC_COMM_WORLD, dm, ierr) CHKERRQ(ierr) before the call to DMPlexCreateFromCellList but then the code just hangs for np >= 2 (again np=1 is fine). A complete test code is below: program test implicit none #include <finclude/petsc.h90> DM :: dm, dm_distrib PetscViewer :: viewer PetscErrorCode :: ierr PetscInt :: rank PetscInt, dimension(6) :: cell_list PetscReal, dimension(8) :: vertex_coords call PetscInitialize(PETSC_NULL_CHARACTER,ierr) CHKERRQ(ierr) call MPI_Comm_rank(PETSC_COMM_WORLD,rank,ierr) CHKERRQ(ierr) cell_list = (/ 0 , 1 , 2 , 1 , 3 , 2 /) vertex_coords = 0.0 !call DMPlexCreate(PETSC_COMM_WORLD, dm, ierr) !CHKERRQ(ierr) if (rank==0) then call DMPlexCreateFromCellList(PETSC_COMM_WORLD, 2, 2, 4, 3, 1 , cell_list , 2 , vertex_coords , dm, ierr) CHKERRQ(ierr) end if call DMPlexDistribute(dm,'chaco',1,dm_distrib,ierr) CHKERRQ(ierr) call PetscFinalize(ierr) CHKERRQ(ierr) end program test -Chris On Wed, Jul 31, 2013 at 4:21 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Chris Eldred <chris.eldred(a)gmail.com> writes: > > Here is my relevant code: > > > > cell_list = (/ 0 , 1 , 2 , 1 , 3 , 2 /) > > vertex_coords = 0.0 > > > > call DMPlexCreateFromCellList(PETSC_COMM_WORLD, 2, 2, 4, 3, 1 , > cell_list > > , 2 , vertex_coords , dm, ierr) > > CHKERRQ(ierr) > > Which process executes this line? DMPlexDistribute expects rank 0 to > hold the mesh to be distributed. If this doesn't fix the problem, can > you send a test code so we can debug and add it to our test suite? > > > call DMPlexDistribute(dm,'chaco',0,dm_distrib,ierr) > > CHKERRQ(ierr) > > > > I am running the code with mpirun -np 2 ./test (it also segfaults with > > higher values for np). > > > > Any ideas? The code works fine with mpirun -np 1 (presumably since it is > > not actually attempting to distribute the mesh). > > -- Chris Eldred DOE Computational Science Graduate Fellow Graduate Student, Atmospheric Science, Colorado State University B.S. Applied Computational Physics, Carnegie Mellon University, 2009 chris.eldred(a)gmail.com / celdred(a)atmos.colostate.edu

1 0

Re: [petsc-dev] Segfault in DMPlexDistribute
by Jed Brown 31 Jul '13

31 Jul '13

Chris Eldred <chris.eldred(a)gmail.com> writes: > Here is my relevant code: > > cell_list = (/ 0 , 1 , 2 , 1 , 3 , 2 /) > vertex_coords = 0.0 > > call DMPlexCreateFromCellList(PETSC_COMM_WORLD, 2, 2, 4, 3, 1 , cell_list > , 2 , vertex_coords , dm, ierr) > CHKERRQ(ierr) Which process executes this line? DMPlexDistribute expects rank 0 to hold the mesh to be distributed. If this doesn't fix the problem, can you send a test code so we can debug and add it to our test suite? > call DMPlexDistribute(dm,'chaco',0,dm_distrib,ierr) > CHKERRQ(ierr) > > I am running the code with mpirun -np 2 ./test (it also segfaults with > higher values for np). > > Any ideas? The code works fine with mpirun -np 1 (presumably since it is > not actually attempting to distribute the mesh).

1 0

Segfault in DMPlexDistribute
by Chris Eldred 31 Jul '13

31 Jul '13

I am getting a segmentation fault when I try to distribute a DMPlex mesh: [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range [1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [1]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors [1]PETSC ERROR: likely location of problem given in stack below [1]PETSC ERROR: --------------------- Stack Frames ------------------------------------ [1]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [1]PETSC ERROR: INSTEAD the line number of the start of the function [1]PETSC ERROR: is given. [1]PETSC ERROR: [1] PetscSFCreateEmbeddedSF line 863 /home/user/Desktop/LIBRARIES/petsc/src/vec/is/sf/interface/sf.c [1]PETSC ERROR: [1] PetscSFDistributeSection line 1755 /home/user/Desktop/LIBRARIES/petsc/src/vec/is/utils/vsectionis.c [1]PETSC ERROR: [1] DMPlexDistribute line 2771 /home/user/Desktop/LIBRARIES/petsc/src/dm/impls/plex/plex.c [1]PETSC ERROR: --------------------- Error Message ------------------------------------ [1]PETSC ERROR: Signal received! [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: Petsc Development GIT revision: dd831807ccae33b4bba0c6456b55575ff17239c3 GIT Date: 2013-06-03 14:26:06 -0600 [1]PETSC ERROR: See docs/changes/index.html for recent updates. [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [1]PETSC ERROR: See docs/index.html for manual pages. [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: ./test on a arch-linux2-cxx-debug named Puget-101334 by user Wed Jul 31 12:44:44 2013 [1]PETSC ERROR: Libraries linked from /home/user/Desktop/LIBRARIES/petsc/arch-linux2-cxx-debug/lib [1]PETSC ERROR: Configure run at Mon Jun 3 14:29:12 2013 [1]PETSC ERROR: Configure options --download-boost --download-chaco --download-ctetgen --download-f-blas-lapack --download-fiat --download-generator --download-hdf5 --download-metis --download-ml --download-mpich --download-parmetis --download-scientificpython --download-triangle --with-clanguage=cxx --with-dynamic-loading --with-openmp --with-pthreadclasses --with-shared-libraries --with-sieve --with-threadcomm PETSC_ARCH=arch-linux2-cxx-debug [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: User provided function() line 0 in unknown directory unknown file application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1 [cli_1]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1 I am building the mesh using DMPlexCreateFromCellList but I get the exact same error occurring if I build it using DMPlexSetCone, etc. It is the sample mesh from http://www.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/sieve.pdf(fig… 2) so very simple. Here is my relevant code: cell_list = (/ 0 , 1 , 2 , 1 , 3 , 2 /) vertex_coords = 0.0 call DMPlexCreateFromCellList(PETSC_COMM_WORLD, 2, 2, 4, 3, 1 , cell_list , 2 , vertex_coords , dm, ierr) CHKERRQ(ierr) call DMPlexDistribute(dm,'chaco',0,dm_distrib,ierr) CHKERRQ(ierr) I am running the code with mpirun -np 2 ./test (it also segfaults with higher values for np). Any ideas? The code works fine with mpirun -np 1 (presumably since it is not actually attempting to distribute the mesh). Thanks, -Chris Eldred -- Chris Eldred DOE Computational Science Graduate Fellow Graduate Student, Atmospheric Science, Colorado State University B.S. Applied Computational Physics, Carnegie Mellon University, 2009 chris.eldred(a)gmail.com / celdred(a)atmos.colostate.edu

1 0

Re: [petsc-dev] future of PetscSubcomm
by Jed Brown 31 Jul '13

31 Jul '13

Barry Smith <bsmith(a)mcs.anl.gov> writes: > Jed, > > Hong and I were discussing MatGetRedundantMatrix() etc and she > mention that you don't like PetscSubcomm. I suggested removing the > PetscSubcomm object from PETSc but retaining the functionality of > PetscSubcomm by putting the subcomm information into an MPI > attribute inside an MPI_Comm (so this new comm behaves like a > MPI_Comm subclass with extra data over a regular MPI_Comm) and > having something like PetscCommSublimate(comm, …., &subcomm with > attribute). Then in all the places in PETSc that take PetscSubcomm > they would instead take a MPI_Comm and check for the attribute and > use the additional information if it is there. I would be inclined to remove dependence on this extra information whenever possible. For example, I think it is counter-productive in MatGetRedundantMatrix because it imposes a distribution that has poor locality. But if we need that extra information, or can do something better if it is there, I prefer attaching it as an attribute to having a wrapper (PetscSubcomm). > Managing destruction will be tricky due to possible reference > loops but presumably we can figure out a way to do it. What is missing in the MPI attribute callbacks to implement reference counting? (We already use it for this purpose.)

1 0

Re: [petsc-dev] Q/A: submiminor on PETSc
by Jed Brown 31 Jul '13

31 Jul '13

"Toby D. Young" <tyoung(a)ippt.pan.pl> writes: > PS. Can I get my commit permissions back for PETSc dev please? Do you have a bitbucket account? We are doing all development in topic branches, which you can create in your own fork. If you're doing frequent development, we can add you to the PETSc team so that you can push branches there. See the branch workflow documentation here. https://bitbucket.org/petsc/petsc/wiki/

1 0