August 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] clique
by Jack Poulson 31 Aug '12

31 Aug '12

Jed, I completely agree that it is a nightmare. All that I want is (parallel) nodal bisection. Right now, the best solution is to create two trivial source files: https://github.com/poulson/Clique/blob/master/src/parmetis/ParallelBisect.c https://github.com/poulson/Clique/blob/master/src/metis/Bisect.c which I removed them from the ParMeTiS source tree upon request. It seems that, given the nightmare of forcing the user to provide me with the directory containing their ParMeTiS thread-local support configuration file, that I should just modify the symbol names of the ParMeTiS routines which I include with Clique. This should fix all of the problems. Jack On Fri, Aug 31, 2012 at 2:28 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Jack, this bundling strategy is a distribution nightmare. Suppose that > some other package "X" independent of clique also bundles Elemental. How > does an application (or another library) use both X and clique? > On Aug 31, 2012 1:49 PM, "Satish Balay" <balay(a)mcs.anl.gov> wrote: > >> Should have cc:d to petsc-dev >> >> context: >> >> clique packages metis,parmetis,elemental sources in a single bundle. >> So we were attempting to build them separately [from petsc configure]. >> >> Jack has extensions to metis and parmetis functionality - thats used >> in clique. We tried to split this out into separate -lparmetis-addons >> library [using parmetis public includes] so that our current >> -lparmetis could be used. But this doesn't work as it needs parmetis >> private includes. >> >> So looks like we'll have to add these extensions to the metis/parmetis >> tarballs we distribute [and have common metis/parmetis sources >> available for both petsc an clique] >> >> Satish >> >> On Fri, 31 Aug 2012, Satish Balay wrote: >> >> > On Fri, 31 Aug 2012, Satish Balay wrote: >> > >> > > On Fri, 31 Aug 2012, Jack Poulson wrote: >> > > >> > > > > Also does clique also build elemental? Can we split it up aswell >> with >> > > > > MANUAL_ELEMENTAL, ELEMENTAL_INCLUDE_DIR, ELEMENTAL_LIBS options? >> > > > > >> > > > > >> > > > I will take care of all of these issues as soon as I get a chance. >> Clique >> > > > does currently build Elemental (it sits in external/elemental). If >> both >> > > > Clique and Elemental are both to be used in the same package, I >> would think >> > > > it would make sense to just install Clique and use its version of >> > > > Elemental. Is there a problem with this approach? It ensures >> compatibility. >> > > >> > > I'll have to check this - but 2 ways of building the same thing is not >> > > easy in petsc build tools. Esp when we provide --with-elemental-dir >> > > option to configure. >> > > >> > > Sure we'll have to make sure the compatible elemental,clique are >> > > installed - when both are installed by petsc [or by user] >> > >> > Jack, >> > >> > [I'm having to add the flag BUILD_PARMETIS=0.] But now I get: >> > >> > cd /home/balay/spetsc/externalpackages/clique/arch-clique/src/parmetis >> && /usr/lib64/ccache/gcc -Wall -Wwrite-strings -Wno-strict-aliasing >> -Wno-unknown-pragmas -g3 -fno-inline -O0 >> -I/home/balay/spetsc/externalpackages/clique/arch-clique/external/elemental/include >> -I/home/balay/soft/linux64/mpich2-1.1/include >> -I/home/balay/spetsc/arch-clique/include >> -I/home/balay/spetsc/externalpackages/clique/external/parmetis/include >> -I/home/balay/spetsc/externalpackages/clique/external/parmetis/libparmetis >> -I/home/balay/spetsc/externalpackages/clique/external/parmetis/metis/GKlib >> -I/home/balay/spetsc/externalpackages/clique/external/parmetis/metis/include >> -o CMakeFiles/parmetis-addons.dir/ParallelBisect.c.o -c >> /home/balay/spetsc/externalpackages/clique/src/parmetis/ParallelBisect.c >> > >> /home/balay/spetsc/externalpackages/clique/src/parmetis/ParallelBisect.c:5:25: >> fatal error: parmetislib.h: No such file or directory >> > >> > Looks like this code needs parmetis private includes. So the current >> > stratergy [of splitting into -lparmetis-addons] isn't working. >> > >> > Perhaps we'll have to use a single patched metis/parmetis tarball for >> > both petsc and clique afterall.. >> > >> > Satish >> > >> >>

1 0

Re: [petsc-dev] clique
by Satish Balay 31 Aug '12

31 Aug '12

The fix I'm hoping for is: the bundled software defaults to using internal pckage - but the build tools also support --with-package-include,--with-package-lib options for externally build version of the internal packages. this is partly done for metis/parmetis within clique but there is this other issue with extensions to metis/parmetis we have to fix. Satish On Fri, 31 Aug 2012, Jed Brown wrote: > Jack, this bundling strategy is a distribution nightmare. Suppose that some > other package "X" independent of clique also bundles Elemental. How does an > application (or another library) use both X and clique? > On Aug 31, 2012 1:49 PM, "Satish Balay" <balay(a)mcs.anl.gov> wrote: > > > Should have cc:d to petsc-dev > > > > context: > > > > clique packages metis,parmetis,elemental sources in a single bundle. > > So we were attempting to build them separately [from petsc configure]. > > > > Jack has extensions to metis and parmetis functionality - thats used > > in clique. We tried to split this out into separate -lparmetis-addons > > library [using parmetis public includes] so that our current > > -lparmetis could be used. But this doesn't work as it needs parmetis > > private includes. > > > > So looks like we'll have to add these extensions to the metis/parmetis > > tarballs we distribute [and have common metis/parmetis sources > > available for both petsc an clique] > > > > Satish > > > > On Fri, 31 Aug 2012, Satish Balay wrote: > > > > > On Fri, 31 Aug 2012, Satish Balay wrote: > > > > > > > On Fri, 31 Aug 2012, Jack Poulson wrote: > > > > > > > > > > Also does clique also build elemental? Can we split it up aswell > > with > > > > > > MANUAL_ELEMENTAL, ELEMENTAL_INCLUDE_DIR, ELEMENTAL_LIBS options? > > > > > > > > > > > > > > > > > I will take care of all of these issues as soon as I get a chance. > > Clique > > > > > does currently build Elemental (it sits in external/elemental). If > > both > > > > > Clique and Elemental are both to be used in the same package, I > > would think > > > > > it would make sense to just install Clique and use its version of > > > > > Elemental. Is there a problem with this approach? It ensures > > compatibility. > > > > > > > > I'll have to check this - but 2 ways of building the same thing is not > > > > easy in petsc build tools. Esp when we provide --with-elemental-dir > > > > option to configure. > > > > > > > > Sure we'll have to make sure the compatible elemental,clique are > > > > installed - when both are installed by petsc [or by user] > > > > > > Jack, > > > > > > [I'm having to add the flag BUILD_PARMETIS=0.] But now I get: > > > > > > cd /home/balay/spetsc/externalpackages/clique/arch-clique/src/parmetis > > && /usr/lib64/ccache/gcc -Wall -Wwrite-strings -Wno-strict-aliasing > > -Wno-unknown-pragmas -g3 -fno-inline -O0 > > -I/home/balay/spetsc/externalpackages/clique/arch-clique/external/elemental/include > > -I/home/balay/soft/linux64/mpich2-1.1/include > > -I/home/balay/spetsc/arch-clique/include > > -I/home/balay/spetsc/externalpackages/clique/external/parmetis/include > > -I/home/balay/spetsc/externalpackages/clique/external/parmetis/libparmetis > > -I/home/balay/spetsc/externalpackages/clique/external/parmetis/metis/GKlib > > -I/home/balay/spetsc/externalpackages/clique/external/parmetis/metis/include > > -o CMakeFiles/parmetis-addons.dir/ParallelBisect.c.o -c > > /home/balay/spetsc/externalpackages/clique/src/parmetis/ParallelBisect.c > > > > > /home/balay/spetsc/externalpackages/clique/src/parmetis/ParallelBisect.c:5:25: > > fatal error: parmetislib.h: No such file or directory > > > > > > Looks like this code needs parmetis private includes. So the current > > > stratergy [of splitting into -lparmetis-addons] isn't working. > > > > > > Perhaps we'll have to use a single patched metis/parmetis tarball for > > > both petsc and clique afterall.. > > > > > > Satish > > > > > > > >

1 0

Re: [petsc-dev] clique
by Jed Brown 31 Aug '12

31 Aug '12

Jack, this bundling strategy is a distribution nightmare. Suppose that some other package "X" independent of clique also bundles Elemental. How does an application (or another library) use both X and clique? On Aug 31, 2012 1:49 PM, "Satish Balay" <balay(a)mcs.anl.gov> wrote: > Should have cc:d to petsc-dev > > context: > > clique packages metis,parmetis,elemental sources in a single bundle. > So we were attempting to build them separately [from petsc configure]. > > Jack has extensions to metis and parmetis functionality - thats used > in clique. We tried to split this out into separate -lparmetis-addons > library [using parmetis public includes] so that our current > -lparmetis could be used. But this doesn't work as it needs parmetis > private includes. > > So looks like we'll have to add these extensions to the metis/parmetis > tarballs we distribute [and have common metis/parmetis sources > available for both petsc an clique] > > Satish > > On Fri, 31 Aug 2012, Satish Balay wrote: > > > On Fri, 31 Aug 2012, Satish Balay wrote: > > > > > On Fri, 31 Aug 2012, Jack Poulson wrote: > > > > > > > > Also does clique also build elemental? Can we split it up aswell > with > > > > > MANUAL_ELEMENTAL, ELEMENTAL_INCLUDE_DIR, ELEMENTAL_LIBS options? > > > > > > > > > > > > > > I will take care of all of these issues as soon as I get a chance. > Clique > > > > does currently build Elemental (it sits in external/elemental). If > both > > > > Clique and Elemental are both to be used in the same package, I > would think > > > > it would make sense to just install Clique and use its version of > > > > Elemental. Is there a problem with this approach? It ensures > compatibility. > > > > > > I'll have to check this - but 2 ways of building the same thing is not > > > easy in petsc build tools. Esp when we provide --with-elemental-dir > > > option to configure. > > > > > > Sure we'll have to make sure the compatible elemental,clique are > > > installed - when both are installed by petsc [or by user] > > > > Jack, > > > > [I'm having to add the flag BUILD_PARMETIS=0.] But now I get: > > > > cd /home/balay/spetsc/externalpackages/clique/arch-clique/src/parmetis > && /usr/lib64/ccache/gcc -Wall -Wwrite-strings -Wno-strict-aliasing > -Wno-unknown-pragmas -g3 -fno-inline -O0 > -I/home/balay/spetsc/externalpackages/clique/arch-clique/external/elemental/include > -I/home/balay/soft/linux64/mpich2-1.1/include > -I/home/balay/spetsc/arch-clique/include > -I/home/balay/spetsc/externalpackages/clique/external/parmetis/include > -I/home/balay/spetsc/externalpackages/clique/external/parmetis/libparmetis > -I/home/balay/spetsc/externalpackages/clique/external/parmetis/metis/GKlib > -I/home/balay/spetsc/externalpackages/clique/external/parmetis/metis/include > -o CMakeFiles/parmetis-addons.dir/ParallelBisect.c.o -c > /home/balay/spetsc/externalpackages/clique/src/parmetis/ParallelBisect.c > > > /home/balay/spetsc/externalpackages/clique/src/parmetis/ParallelBisect.c:5:25: > fatal error: parmetislib.h: No such file or directory > > > > Looks like this code needs parmetis private includes. So the current > > stratergy [of splitting into -lparmetis-addons] isn't working. > > > > Perhaps we'll have to use a single patched metis/parmetis tarball for > > both petsc and clique afterall.. > > > > Satish > > > >

1 0

Re: [petsc-dev] clique
by Satish Balay 31 Aug '12

31 Aug '12

Should have cc:d to petsc-dev context: clique packages metis,parmetis,elemental sources in a single bundle. So we were attempting to build them separately [from petsc configure]. Jack has extensions to metis and parmetis functionality - thats used in clique. We tried to split this out into separate -lparmetis-addons library [using parmetis public includes] so that our current -lparmetis could be used. But this doesn't work as it needs parmetis private includes. So looks like we'll have to add these extensions to the metis/parmetis tarballs we distribute [and have common metis/parmetis sources available for both petsc an clique] Satish On Fri, 31 Aug 2012, Satish Balay wrote: > On Fri, 31 Aug 2012, Satish Balay wrote: > > > On Fri, 31 Aug 2012, Jack Poulson wrote: > > > > > > Also does clique also build elemental? Can we split it up aswell with > > > > MANUAL_ELEMENTAL, ELEMENTAL_INCLUDE_DIR, ELEMENTAL_LIBS options? > > > > > > > > > > > I will take care of all of these issues as soon as I get a chance. Clique > > > does currently build Elemental (it sits in external/elemental). If both > > > Clique and Elemental are both to be used in the same package, I would think > > > it would make sense to just install Clique and use its version of > > > Elemental. Is there a problem with this approach? It ensures compatibility. > > > > I'll have to check this - but 2 ways of building the same thing is not > > easy in petsc build tools. Esp when we provide --with-elemental-dir > > option to configure. > > > > Sure we'll have to make sure the compatible elemental,clique are > > installed - when both are installed by petsc [or by user] > > Jack, > > [I'm having to add the flag BUILD_PARMETIS=0.] But now I get: > > cd /home/balay/spetsc/externalpackages/clique/arch-clique/src/parmetis && /usr/lib64/ccache/gcc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 -I/home/balay/spetsc/externalpackages/clique/arch-clique/external/elemental/include -I/home/balay/soft/linux64/mpich2-1.1/include -I/home/balay/spetsc/arch-clique/include -I/home/balay/spetsc/externalpackages/clique/external/parmetis/include -I/home/balay/spetsc/externalpackages/clique/external/parmetis/libparmetis -I/home/balay/spetsc/externalpackages/clique/external/parmetis/metis/GKlib -I/home/balay/spetsc/externalpackages/clique/external/parmetis/metis/include -o CMakeFiles/parmetis-addons.dir/ParallelBisect.c.o -c /home/balay/spetsc/externalpackages/clique/src/parmetis/ParallelBisect.c > /home/balay/spetsc/externalpackages/clique/src/parmetis/ParallelBisect.c:5:25: fatal error: parmetislib.h: No such file or directory > > Looks like this code needs parmetis private includes. So the current > stratergy [of splitting into -lparmetis-addons] isn't working. > > Perhaps we'll have to use a single patched metis/parmetis tarball for > both petsc and clique afterall.. > > Satish >

1 0

Fwd: [petsc-users] pctype hmpi
by Barry Smith 31 Aug '12

31 Aug '12

HMPI usage in PETSc relies on sending function pointer addresses between MPI processes; this use to be fine since you start up several copies of the same executable on a machine each has the same virtual addresses for instructions. Yes, this a bit hacky but it worked great. With ASLR this model breaks. On Linux I found a way supposedly to turn off ASLR using sysctl -w kernel.randomize_va_space=0 but on Apple there does not seem to be a system wide way to turn it off easily. What to do? It looks like I have to switch to requiring shared libraries and passing string function names and dynamically access them. Any thoughts on alternatives. Barry Begin forwarded message: > From: Barry Smith <bsmith(a)mcs.anl.gov> > Subject: Re: [petsc-users] pctype hmpi > Date: August 31, 2012 1:17:07 PM CDT > To: PETSc users list <petsc-users(a)mcs.anl.gov> > Reply-To: PETSc users list <petsc-users(a)mcs.anl.gov> > > > On Aug 30, 2012, at 10:02 PM, George Pau <gpau(a)lbl.gov> wrote: > >> Hi Barry, >> >> I tried with the addition of >> >> -hmpi_spawn_size 3 >> >> but I am still getting the same error though. > > The EXACT same error? Or some other error? > > What happens if you run with the -hmpi_merge_size <size> option instead? > > Barry > > 1) I am getting a crash with the spawn version I suspect is due to bugs in the MPICH version I am using related to spawn. > > 2) I am getting errors with the merge version due to Apple's ASLR which they make hard to turn off. > > >> I am using mpich2. Any other options to try? >> >> George >> >> >> On Aug 30, 2012, at 7:28 PM, Barry Smith wrote: >> >>> >>> On Aug 30, 2012, at 7:24 PM, George Pau <gpau(a)lbl.gov> wrote: >>> >>>> Hi, >>>> >>>> I have some issues using the -pctype hmpi. I used the same setting found at >>>> >>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCHMPI.html >>>> >>>> i.e. >>>> -pc_type hmpi >>>> -ksp_type preonly >>>> -hmpi_ksp_type cg >>>> -hmpi_pc_type hypre >>>> -hmpi_pc_hypre_type boomeramg >>>> >>>> My command is >>>> >>>> mpiexec -n 1 myprogram >>> >>> Sorry the documentation doesn't make this clearer. You need to start PETSc with special options to get the "worker" processes initialized. From the manual page for PCHMPI it has >>> >>> See PetscHMPIMerge() and PetscHMPISpawn() for two ways to start up MPI for use with this preconditioner >>> >>> This will tell you want option to start PETSc up with. >>> >>> I will fix the PC so that it prints a far more useful error message. >>> >>> >>> >>> Barry >>> >>> >>>> >>>> But, I get >>>> >>>> [gilbert:4041] *** An error occurred in MPI_Bcast >>>> [gilbert:4041] *** on communicator MPI_COMM_WORLD >>>> [gilbert:4041] *** MPI_ERR_COMM: invalid communicator >>>> [gilbert:4041] *** MPI_ERRORS_ARE_FATAL (your MPI job will now abort) >>>> >>>> with openmpi. I get similar error with mpich2 >>>> >>>> Fatal error in PMPI_Bcast: Invalid communicator, error stack: >>>> PMPI_Bcast(1478): MPI_Bcast(buf=0x7fffb683479c, count=1, MPI_INT, root=0, comm=0x0) failed >>>> PMPI_Bcast(1418): Invalid communicator >>>> >>>> I couldn't figure out what is wrong. My petsc is version 3.3.3 and the configuration is -with-debugging=0 --with-mpi-dir=/usr/lib/openmpi --download-hypre=1 and I am on a Ubuntu machine. >>>> >>>> Note that with the default pc_type and ksp_type, everything is fine. It was also tested with multiple processors. I wondering whether there are some options that I am not specifying correctly? >>>> >>>> -- >>>> George Pau >>>> Earth Sciences Division >>>> Lawrence Berkeley National Laboratory >>>> One Cyclotron, MS 74-120 >>>> Berkeley, CA 94720 >>>> >>>> (510) 486-7196 >>>> gpau(a)lbl.gov >>>> http://esd.lbl.gov/about/staff/georgepau/ >>>> >>> >> >

1 0

components in PetscSection
by Chris Eldred 31 Aug '12

31 Aug '12

I was wondering how components were treated in PetscSection- my understanding is that a field with multiple components and multiple dofs would have those dofs duplicated for each component, but that does not appear to be the case. For example, let's say I have a field with 2 components and 2 dofs that covers 10 points. I expected that it would have size 40: 4 dofs per point x 10 points. Instead, it appears to have size 20: 2 dofs x 10 pts. What then are components in PetscSection- is it just a way of telling the code that the dofs associated with a field at a point are really split across multiple components (ie for my example, I should define 4 dofs at each pt instead of 2)? How does that then interact with the re-ordering done in DMComplexVecGetClosure? -- Chris Eldred DOE Computational Science Graduate Fellow Graduate Student, Atmospheric Science, Colorado State University B.S. Applied Computational Physics, Carnegie Mellon University, 2009 chris.eldred(a)gmail.com

1 0

Re: [petsc-dev] DMSetDefaultSection destroys old sections
by Matthew Knepley 30 Aug '12

30 Aug '12

On Thu, Aug 30, 2012 at 6:04 PM, Chris Eldred <chris.eldred(a)gmail.com>wrote: > Does this mean that I need a different DMComplex object for each > section that I want to use? It seems like that would get expensive > memory-wise pretty fast (unless DMComplexClone is just a shallow > copy?) > It is a shallow copy. > The reason I would like separate sections for each field is that > different fields might have different stencil sizes- and there are a > lot of constant fields that don't need any sort of stencils since they > are generated and used only locally. So by having separate sections > (and SF) for each field I can control what gets ghosted/communicated > and what is not. > Yes, you want different DMs. The way I organize things is to have a DM for the solve, with all the unknown fields, and one for all auxiliary fields. Then if you want specific subsets of fields, you use DMCreateSubDM() which uses DMComplexClone() underneath. Matt > On Thu, Aug 30, 2012 at 4:54 PM, Matthew Knepley <knepley(a)gmail.com> > wrote: > > On Thu, Aug 30, 2012 at 5:51 PM, Chris Eldred <chris.eldred(a)gmail.com> > > wrote: > >> > >> It appears that PetscObjectReference works for DM and Vec (in > >> Fortran), but not for Section. Is this possibly related to the Fortran > >> stub/binding for PetscObjectReference? > > > > > > Sorry, I was not paying attention. PetscSection is not a PetscObject. > That > > is also why it > > is not referenced. It was originally designed to be a helper object. So, > in > > order to > > do what you want, currently (and actually this is how the other PETSc > people > > want > > DM to behave), you should do > > > > call DMComplexClone(dm, dmNew, ierr) > > call DMSetDefaultSection(dmNew, sectionNew, ierr) > > > > Matt > > > >> > >> On Thu, Aug 30, 2012 at 4:44 PM, Jed Brown <jedbrown(a)mcs.anl.gov> > wrote: > >> > You must have something else going on. The following works fine, for > >> > example. > >> > > >> > diff --git a/src/snes/examples/tutorials/ex5f.F > >> > b/src/snes/examples/tutorials/ex5f.F > >> > --- a/src/snes/examples/tutorials/ex5f.F > >> > +++ b/src/snes/examples/tutorials/ex5f.F > >> > @@ -58,6 +58,7 @@ > >> > PetscErrorCode ierr > >> > PetscReal lambda_max,lambda_min > >> > PetscBool flg > >> > + DM da2 > >> > > >> > > >> > ! Note: Any user-defined Fortran routines (such as > FormJacobianLocal) > >> > @@ -144,6 +145,9 @@ > >> > call DMDASetLocalFunction(da,FormFunctionLocal,ierr) > >> > call DMDASetLocalJacobian(da,FormJacobianLocal,ierr) > >> > call SNESSetDM(snes,da,ierr) > >> > + da2 = da > >> > + call PetscObjectReference(da2,ierr) > >> > + call DMDestroy(da2,ierr) > >> > > >> > ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > >> > ! Customize nonlinear solver; set runtime options > >> > > >> > > >> > On Thu, Aug 30, 2012 at 5:36 PM, Chris Eldred <chris.eldred(a)gmail.com > > > >> > wrote: > >> >> > >> >> I get an error when I try to use that: > >> >> > >> >> [0]PETSC ERROR: --------------------- Error Message > >> >> ------------------------------------ > >> >> [0]PETSC ERROR: Corrupt argument: > >> >> see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind! > >> >> [0]PETSC ERROR: Invalid type of object: Parameter # 1! > >> >> [0]PETSC ERROR: > >> >> > >> >> > ------------------------------------------------------------------------ > >> >> [0]PETSC ERROR: Petsc Development HG revision: > >> >> 56f27b189e4fd57f5a4b7ffd6aa6bb08bd8a4d5b HG Date: Wed Aug 29 > 12:03:50 > >> >> 2012 -0500 > >> >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. > >> >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > >> >> [0]PETSC ERROR: See docs/index.html for manual pages. > >> >> [0]PETSC ERROR: > >> >> > >> >> > ------------------------------------------------------------------------ > >> >> [0]PETSC ERROR: ./swe on a arch-linu named Puget-101334 by user Thu > >> >> Aug 30 16:35:51 2012 > >> >> [0]PETSC ERROR: Libraries linked from > >> >> /home/user/Desktop/LIBRARIES/petsc-dev/arch-linux2-cxx-debug/lib > >> >> [0]PETSC ERROR: Configure run at Wed Aug 29 12:49:26 2012 > >> >> [0]PETSC ERROR: Configure options --download-boost --download-chaco > >> >> --download-ctetgen --download-f-blas-lapack --download-fiat > >> >> --download-generator --download-metis --download-ml --download-mpich > >> >> --download-parmetis --download-scientificpython --download-triangle > >> >> --with-clanguage=cxx --with-dynamic-loading --with-shared-libraries > >> >> --with-sieve PETSC_ARCH=arch-linux2-cxx-debug > >> >> [0]PETSC ERROR: > >> >> > >> >> > ------------------------------------------------------------------------ > >> >> [0]PETSC ERROR: PetscObjectReference() line 378 in > >> >> /home/user/Desktop/LIBRARIES/petsc-dev/src/sys/objects/inherit.c > >> >> application called MPI_Abort(MPI_COMM_WORLD, 64) - process 0 > >> >> [unset]: aborting job: > >> >> application called MPI_Abort(MPI_COMM_WORLD, 64) - process 0 > >> >> > >> >> > >> >> On Thu, Aug 30, 2012 at 4:32 PM, Jed Brown <jedbrown(a)mcs.anl.gov> > >> >> wrote: > >> >> > On Thu, Aug 30, 2012 at 5:30 PM, Chris Eldred > >> >> > <chris.eldred(a)gmail.com> > >> >> > wrote: > >> >> >> > >> >> >> How would I use PetscObjectReference((PetscObject)section) in > >> >> >> Fortran? > >> >> > > >> >> > > >> >> > call PetscObjectReference(section,ierr) > >> >> > > >> >> >> > >> >> >> > >> >> >> On Thu, Aug 30, 2012 at 4:16 PM, Jed Brown <jedbrown(a)mcs.anl.gov> > >> >> >> wrote: > >> >> >> > On Thu, Aug 30, 2012 at 5:12 PM, Matthew Knepley > >> >> >> > <knepley(a)gmail.com> > >> >> >> > wrote: > >> >> >> >> > >> >> >> >> Its not wrong. This is the way I wanted it. You set the > >> >> >> >> PetscSection > >> >> >> >> and > >> >> >> >> give up control. If you > >> >> >> >> do not want to give up control, then call > PetscObjectReference() > >> >> >> >> before > >> >> >> >> passing it in. > >> >> >> > > >> >> >> > > >> >> >> > Since this is (a) different from everything else in PETSc and > (b) > >> >> >> > not > >> >> >> > explicitly documented, it is doubly wrong. > >> >> >> > >> >> >> > >> >> >> > >> >> >> -- > >> >> >> Chris Eldred > >> >> >> DOE Computational Science Graduate Fellow > >> >> >> Graduate Student, Atmospheric Science, Colorado State University > >> >> >> B.S. Applied Computational Physics, Carnegie Mellon University, > 2009 > >> >> >> chris.eldred(a)gmail.com > >> >> > > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> Chris Eldred > >> >> DOE Computational Science Graduate Fellow > >> >> Graduate Student, Atmospheric Science, Colorado State University > >> >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 > >> >> chris.eldred(a)gmail.com > >> > > >> > > >> > >> > >> > >> -- > >> Chris Eldred > >> DOE Computational Science Graduate Fellow > >> Graduate Student, Atmospheric Science, Colorado State University > >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 > >> chris.eldred(a)gmail.com > > > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments > > is infinitely more interesting than any results to which their > experiments > > lead. > > -- Norbert Wiener > > > > -- > Chris Eldred > DOE Computational Science Graduate Fellow > Graduate Student, Atmospheric Science, Colorado State University > B.S. Applied Computational Physics, Carnegie Mellon University, 2009 > chris.eldred(a)gmail.com > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] DMSetDefaultSection destroys old sections
by Chris Eldred 30 Aug '12

30 Aug '12

Does this mean that I need a different DMComplex object for each section that I want to use? It seems like that would get expensive memory-wise pretty fast (unless DMComplexClone is just a shallow copy?) The reason I would like separate sections for each field is that different fields might have different stencil sizes- and there are a lot of constant fields that don't need any sort of stencils since they are generated and used only locally. So by having separate sections (and SF) for each field I can control what gets ghosted/communicated and what is not. On Thu, Aug 30, 2012 at 4:54 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > On Thu, Aug 30, 2012 at 5:51 PM, Chris Eldred <chris.eldred(a)gmail.com> > wrote: >> >> It appears that PetscObjectReference works for DM and Vec (in >> Fortran), but not for Section. Is this possibly related to the Fortran >> stub/binding for PetscObjectReference? > > > Sorry, I was not paying attention. PetscSection is not a PetscObject. That > is also why it > is not referenced. It was originally designed to be a helper object. So, in > order to > do what you want, currently (and actually this is how the other PETSc people > want > DM to behave), you should do > > call DMComplexClone(dm, dmNew, ierr) > call DMSetDefaultSection(dmNew, sectionNew, ierr) > > Matt > >> >> On Thu, Aug 30, 2012 at 4:44 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: >> > You must have something else going on. The following works fine, for >> > example. >> > >> > diff --git a/src/snes/examples/tutorials/ex5f.F >> > b/src/snes/examples/tutorials/ex5f.F >> > --- a/src/snes/examples/tutorials/ex5f.F >> > +++ b/src/snes/examples/tutorials/ex5f.F >> > @@ -58,6 +58,7 @@ >> > PetscErrorCode ierr >> > PetscReal lambda_max,lambda_min >> > PetscBool flg >> > + DM da2 >> > >> > >> > ! Note: Any user-defined Fortran routines (such as FormJacobianLocal) >> > @@ -144,6 +145,9 @@ >> > call DMDASetLocalFunction(da,FormFunctionLocal,ierr) >> > call DMDASetLocalJacobian(da,FormJacobianLocal,ierr) >> > call SNESSetDM(snes,da,ierr) >> > + da2 = da >> > + call PetscObjectReference(da2,ierr) >> > + call DMDestroy(da2,ierr) >> > >> > ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> > ! Customize nonlinear solver; set runtime options >> > >> > >> > On Thu, Aug 30, 2012 at 5:36 PM, Chris Eldred <chris.eldred(a)gmail.com> >> > wrote: >> >> >> >> I get an error when I try to use that: >> >> >> >> [0]PETSC ERROR: --------------------- Error Message >> >> ------------------------------------ >> >> [0]PETSC ERROR: Corrupt argument: >> >> see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind! >> >> [0]PETSC ERROR: Invalid type of object: Parameter # 1! >> >> [0]PETSC ERROR: >> >> >> >> ------------------------------------------------------------------------ >> >> [0]PETSC ERROR: Petsc Development HG revision: >> >> 56f27b189e4fd57f5a4b7ffd6aa6bb08bd8a4d5b HG Date: Wed Aug 29 12:03:50 >> >> 2012 -0500 >> >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> >> [0]PETSC ERROR: See docs/index.html for manual pages. >> >> [0]PETSC ERROR: >> >> >> >> ------------------------------------------------------------------------ >> >> [0]PETSC ERROR: ./swe on a arch-linu named Puget-101334 by user Thu >> >> Aug 30 16:35:51 2012 >> >> [0]PETSC ERROR: Libraries linked from >> >> /home/user/Desktop/LIBRARIES/petsc-dev/arch-linux2-cxx-debug/lib >> >> [0]PETSC ERROR: Configure run at Wed Aug 29 12:49:26 2012 >> >> [0]PETSC ERROR: Configure options --download-boost --download-chaco >> >> --download-ctetgen --download-f-blas-lapack --download-fiat >> >> --download-generator --download-metis --download-ml --download-mpich >> >> --download-parmetis --download-scientificpython --download-triangle >> >> --with-clanguage=cxx --with-dynamic-loading --with-shared-libraries >> >> --with-sieve PETSC_ARCH=arch-linux2-cxx-debug >> >> [0]PETSC ERROR: >> >> >> >> ------------------------------------------------------------------------ >> >> [0]PETSC ERROR: PetscObjectReference() line 378 in >> >> /home/user/Desktop/LIBRARIES/petsc-dev/src/sys/objects/inherit.c >> >> application called MPI_Abort(MPI_COMM_WORLD, 64) - process 0 >> >> [unset]: aborting job: >> >> application called MPI_Abort(MPI_COMM_WORLD, 64) - process 0 >> >> >> >> >> >> On Thu, Aug 30, 2012 at 4:32 PM, Jed Brown <jedbrown(a)mcs.anl.gov> >> >> wrote: >> >> > On Thu, Aug 30, 2012 at 5:30 PM, Chris Eldred >> >> > <chris.eldred(a)gmail.com> >> >> > wrote: >> >> >> >> >> >> How would I use PetscObjectReference((PetscObject)section) in >> >> >> Fortran? >> >> > >> >> > >> >> > call PetscObjectReference(section,ierr) >> >> > >> >> >> >> >> >> >> >> >> On Thu, Aug 30, 2012 at 4:16 PM, Jed Brown <jedbrown(a)mcs.anl.gov> >> >> >> wrote: >> >> >> > On Thu, Aug 30, 2012 at 5:12 PM, Matthew Knepley >> >> >> > <knepley(a)gmail.com> >> >> >> > wrote: >> >> >> >> >> >> >> >> Its not wrong. This is the way I wanted it. You set the >> >> >> >> PetscSection >> >> >> >> and >> >> >> >> give up control. If you >> >> >> >> do not want to give up control, then call PetscObjectReference() >> >> >> >> before >> >> >> >> passing it in. >> >> >> > >> >> >> > >> >> >> > Since this is (a) different from everything else in PETSc and (b) >> >> >> > not >> >> >> > explicitly documented, it is doubly wrong. >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Chris Eldred >> >> >> DOE Computational Science Graduate Fellow >> >> >> Graduate Student, Atmospheric Science, Colorado State University >> >> >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 >> >> >> chris.eldred(a)gmail.com >> >> > >> >> > >> >> >> >> >> >> >> >> -- >> >> Chris Eldred >> >> DOE Computational Science Graduate Fellow >> >> Graduate Student, Atmospheric Science, Colorado State University >> >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 >> >> chris.eldred(a)gmail.com >> > >> > >> >> >> >> -- >> Chris Eldred >> DOE Computational Science Graduate Fellow >> Graduate Student, Atmospheric Science, Colorado State University >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 >> chris.eldred(a)gmail.com > > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener -- Chris Eldred DOE Computational Science Graduate Fellow Graduate Student, Atmospheric Science, Colorado State University B.S. Applied Computational Physics, Carnegie Mellon University, 2009 chris.eldred(a)gmail.com

1 0

Re: [petsc-dev] DMSetDefaultSection destroys old sections
by Matthew Knepley 30 Aug '12

30 Aug '12

On Thu, Aug 30, 2012 at 5:51 PM, Chris Eldred <chris.eldred(a)gmail.com>wrote: > It appears that PetscObjectReference works for DM and Vec (in > Fortran), but not for Section. Is this possibly related to the Fortran > stub/binding for PetscObjectReference? > Sorry, I was not paying attention. PetscSection is not a PetscObject. That is also why it is not referenced. It was originally designed to be a helper object. So, in order to do what you want, currently (and actually this is how the other PETSc people want DM to behave), you should do call DMComplexClone(dm, dmNew, ierr) call DMSetDefaultSection(dmNew, sectionNew, ierr) Matt > On Thu, Aug 30, 2012 at 4:44 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > > You must have something else going on. The following works fine, for > > example. > > > > diff --git a/src/snes/examples/tutorials/ex5f.F > > b/src/snes/examples/tutorials/ex5f.F > > --- a/src/snes/examples/tutorials/ex5f.F > > +++ b/src/snes/examples/tutorials/ex5f.F > > @@ -58,6 +58,7 @@ > > PetscErrorCode ierr > > PetscReal lambda_max,lambda_min > > PetscBool flg > > + DM da2 > > > > > > ! Note: Any user-defined Fortran routines (such as FormJacobianLocal) > > @@ -144,6 +145,9 @@ > > call DMDASetLocalFunction(da,FormFunctionLocal,ierr) > > call DMDASetLocalJacobian(da,FormJacobianLocal,ierr) > > call SNESSetDM(snes,da,ierr) > > + da2 = da > > + call PetscObjectReference(da2,ierr) > > + call DMDestroy(da2,ierr) > > > > ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > ! Customize nonlinear solver; set runtime options > > > > > > On Thu, Aug 30, 2012 at 5:36 PM, Chris Eldred <chris.eldred(a)gmail.com> > > wrote: > >> > >> I get an error when I try to use that: > >> > >> [0]PETSC ERROR: --------------------- Error Message > >> ------------------------------------ > >> [0]PETSC ERROR: Corrupt argument: > >> see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind! > >> [0]PETSC ERROR: Invalid type of object: Parameter # 1! > >> [0]PETSC ERROR: > >> ------------------------------------------------------------------------ > >> [0]PETSC ERROR: Petsc Development HG revision: > >> 56f27b189e4fd57f5a4b7ffd6aa6bb08bd8a4d5b HG Date: Wed Aug 29 12:03:50 > >> 2012 -0500 > >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. > >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > >> [0]PETSC ERROR: See docs/index.html for manual pages. > >> [0]PETSC ERROR: > >> ------------------------------------------------------------------------ > >> [0]PETSC ERROR: ./swe on a arch-linu named Puget-101334 by user Thu > >> Aug 30 16:35:51 2012 > >> [0]PETSC ERROR: Libraries linked from > >> /home/user/Desktop/LIBRARIES/petsc-dev/arch-linux2-cxx-debug/lib > >> [0]PETSC ERROR: Configure run at Wed Aug 29 12:49:26 2012 > >> [0]PETSC ERROR: Configure options --download-boost --download-chaco > >> --download-ctetgen --download-f-blas-lapack --download-fiat > >> --download-generator --download-metis --download-ml --download-mpich > >> --download-parmetis --download-scientificpython --download-triangle > >> --with-clanguage=cxx --with-dynamic-loading --with-shared-libraries > >> --with-sieve PETSC_ARCH=arch-linux2-cxx-debug > >> [0]PETSC ERROR: > >> ------------------------------------------------------------------------ > >> [0]PETSC ERROR: PetscObjectReference() line 378 in > >> /home/user/Desktop/LIBRARIES/petsc-dev/src/sys/objects/inherit.c > >> application called MPI_Abort(MPI_COMM_WORLD, 64) - process 0 > >> [unset]: aborting job: > >> application called MPI_Abort(MPI_COMM_WORLD, 64) - process 0 > >> > >> > >> On Thu, Aug 30, 2012 at 4:32 PM, Jed Brown <jedbrown(a)mcs.anl.gov> > wrote: > >> > On Thu, Aug 30, 2012 at 5:30 PM, Chris Eldred <chris.eldred(a)gmail.com > > > >> > wrote: > >> >> > >> >> How would I use PetscObjectReference((PetscObject)section) in > Fortran? > >> > > >> > > >> > call PetscObjectReference(section,ierr) > >> > > >> >> > >> >> > >> >> On Thu, Aug 30, 2012 at 4:16 PM, Jed Brown <jedbrown(a)mcs.anl.gov> > >> >> wrote: > >> >> > On Thu, Aug 30, 2012 at 5:12 PM, Matthew Knepley < > knepley(a)gmail.com> > >> >> > wrote: > >> >> >> > >> >> >> Its not wrong. This is the way I wanted it. You set the > PetscSection > >> >> >> and > >> >> >> give up control. If you > >> >> >> do not want to give up control, then call PetscObjectReference() > >> >> >> before > >> >> >> passing it in. > >> >> > > >> >> > > >> >> > Since this is (a) different from everything else in PETSc and (b) > not > >> >> > explicitly documented, it is doubly wrong. > >> >> > >> >> > >> >> > >> >> -- > >> >> Chris Eldred > >> >> DOE Computational Science Graduate Fellow > >> >> Graduate Student, Atmospheric Science, Colorado State University > >> >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 > >> >> chris.eldred(a)gmail.com > >> > > >> > > >> > >> > >> > >> -- > >> Chris Eldred > >> DOE Computational Science Graduate Fellow > >> Graduate Student, Atmospheric Science, Colorado State University > >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 > >> chris.eldred(a)gmail.com > > > > > > > > -- > Chris Eldred > DOE Computational Science Graduate Fellow > Graduate Student, Atmospheric Science, Colorado State University > B.S. Applied Computational Physics, Carnegie Mellon University, 2009 > chris.eldred(a)gmail.com > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] DMSetDefaultSection destroys old sections
by Chris Eldred 30 Aug '12

30 Aug '12

It appears that PetscObjectReference works for DM and Vec (in Fortran), but not for Section. Is this possibly related to the Fortran stub/binding for PetscObjectReference? On Thu, Aug 30, 2012 at 4:44 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > You must have something else going on. The following works fine, for > example. > > diff --git a/src/snes/examples/tutorials/ex5f.F > b/src/snes/examples/tutorials/ex5f.F > --- a/src/snes/examples/tutorials/ex5f.F > +++ b/src/snes/examples/tutorials/ex5f.F > @@ -58,6 +58,7 @@ > PetscErrorCode ierr > PetscReal lambda_max,lambda_min > PetscBool flg > + DM da2 > > > ! Note: Any user-defined Fortran routines (such as FormJacobianLocal) > @@ -144,6 +145,9 @@ > call DMDASetLocalFunction(da,FormFunctionLocal,ierr) > call DMDASetLocalJacobian(da,FormJacobianLocal,ierr) > call SNESSetDM(snes,da,ierr) > + da2 = da > + call PetscObjectReference(da2,ierr) > + call DMDestroy(da2,ierr) > > ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > ! Customize nonlinear solver; set runtime options > > > On Thu, Aug 30, 2012 at 5:36 PM, Chris Eldred <chris.eldred(a)gmail.com> > wrote: >> >> I get an error when I try to use that: >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Corrupt argument: >> see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind! >> [0]PETSC ERROR: Invalid type of object: Parameter # 1! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Development HG revision: >> 56f27b189e4fd57f5a4b7ffd6aa6bb08bd8a4d5b HG Date: Wed Aug 29 12:03:50 >> 2012 -0500 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./swe on a arch-linu named Puget-101334 by user Thu >> Aug 30 16:35:51 2012 >> [0]PETSC ERROR: Libraries linked from >> /home/user/Desktop/LIBRARIES/petsc-dev/arch-linux2-cxx-debug/lib >> [0]PETSC ERROR: Configure run at Wed Aug 29 12:49:26 2012 >> [0]PETSC ERROR: Configure options --download-boost --download-chaco >> --download-ctetgen --download-f-blas-lapack --download-fiat >> --download-generator --download-metis --download-ml --download-mpich >> --download-parmetis --download-scientificpython --download-triangle >> --with-clanguage=cxx --with-dynamic-loading --with-shared-libraries >> --with-sieve PETSC_ARCH=arch-linux2-cxx-debug >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: PetscObjectReference() line 378 in >> /home/user/Desktop/LIBRARIES/petsc-dev/src/sys/objects/inherit.c >> application called MPI_Abort(MPI_COMM_WORLD, 64) - process 0 >> [unset]: aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 64) - process 0 >> >> >> On Thu, Aug 30, 2012 at 4:32 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: >> > On Thu, Aug 30, 2012 at 5:30 PM, Chris Eldred <chris.eldred(a)gmail.com> >> > wrote: >> >> >> >> How would I use PetscObjectReference((PetscObject)section) in Fortran? >> > >> > >> > call PetscObjectReference(section,ierr) >> > >> >> >> >> >> >> On Thu, Aug 30, 2012 at 4:16 PM, Jed Brown <jedbrown(a)mcs.anl.gov> >> >> wrote: >> >> > On Thu, Aug 30, 2012 at 5:12 PM, Matthew Knepley <knepley(a)gmail.com> >> >> > wrote: >> >> >> >> >> >> Its not wrong. This is the way I wanted it. You set the PetscSection >> >> >> and >> >> >> give up control. If you >> >> >> do not want to give up control, then call PetscObjectReference() >> >> >> before >> >> >> passing it in. >> >> > >> >> > >> >> > Since this is (a) different from everything else in PETSc and (b) not >> >> > explicitly documented, it is doubly wrong. >> >> >> >> >> >> >> >> -- >> >> Chris Eldred >> >> DOE Computational Science Graduate Fellow >> >> Graduate Student, Atmospheric Science, Colorado State University >> >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 >> >> chris.eldred(a)gmail.com >> > >> > >> >> >> >> -- >> Chris Eldred >> DOE Computational Science Graduate Fellow >> Graduate Student, Atmospheric Science, Colorado State University >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009 >> chris.eldred(a)gmail.com > > -- Chris Eldred DOE Computational Science Graduate Fellow Graduate Student, Atmospheric Science, Colorado State University B.S. Applied Computational Physics, Carnegie Mellon University, 2009 chris.eldred(a)gmail.com

1 0