January 2014 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] PetscMalloc for size zero
by Jed Brown 31 Jan '14

31 Jan '14

Barry Smith <bsmith(a)mcs.anl.gov> writes: > Note that currently when running optimized with valgrind; valgrind > only sees the one_big_malloc so can miss lots of overwrite and > underwrites within the chunks malloced. By making everything runtime > we can let valgrind have more information and manage correctness of > each sub chunk. I agree that it's good to defer as much as possible to runtime. I sorely wish I knew of a good way to implement a -debug option (short of LD_PRELOADing the debug library). I'm not happy about Matt's increasingly complicated macros, but I'm not going to fight it any more.

1 0

Re: [petsc-dev] Cell overlap in DMPlexDistribute
by Matthew Knepley 31 Jan '14

31 Jan '14

On Tue, Jan 28, 2014 at 8:57 AM, Michael Lange <michael.lange(a)imperial.ac.uk > wrote: > Hi, > > I noticed that the cell overlap created during DMPlexDistribute does not > include cells that only share a vertex but no edge with an owned cell. This > causes problems when performing local assembly > (MAT_IGNORE_OFF_PROC_ENTRIES) based on information from the plex, because > one contribution to the shared vertex is missing. > > As an example consider the 2x2 square with 8 cells (attached). When > partitioned across two ranks with overlap=1 each rank owns 4 cells and in > the current version knows about 2 halo cells, giving a total of 6 cells. > The central vertex, however, touches 3 owned and 3 non-owned cells, one of > which doesn't share an edge with any owned cells. So, in order to correctly > assemble the central vertex locally, the rank owning it needs to know about > 7 cells in total. > > I have attached a patch that fixes this problem by going over the inverse > closure of all vertices associated with a given cell instead of using the > provided edge graph. Please tell me what you think and whether there might > be an easier way to fix this. > This is true, but unfortunately not what everyone wants. FVM people want just what is there now. This same choice comes up in preallocation. There I have put in the "center dimension" which says what kind of point do you use for the center, vertex or face? I guess we need that here as well. Matt > Kind regards > Michael Lange > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] loop structure in PetscFEIntegrateJacobian_Basic
by Matthew Knepley 31 Jan '14

31 Jan '14

On Tue, Jan 28, 2014 at 3:45 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > On Tue, Jan 28, 2014 at 3:31 PM, Geoffrey Irving <irving(a)naml.us> wrote: > >> PetscFEIntegrateJacobian_Basic has four layers of loops: >> >> e - element >> f - component of field being differentiated >> g - component of field we're differentiating against >> q - quadrature point >> >> It looks like g3_func and friends are being called once for each >> combination of (e,f,g,q), even though the calls to g3_func are >> independent of (f,g). If g3_func is expensive, this duplication seems >> a bit slow. >> >> It seems like the loops should be flipped to be (e,q,f,g) major order, >> with the user functions hoisted up to the (e,q) level. Is there a >> reason not to do this? > > > Yes, it is stupid. I will fix it. > I did not forget. Pushed today and verified with the SNES ex62 tests. Matt > Matt > > >> >> Geoffrey >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] What happened to -log_summary_python?
by Matthew Knepley 31 Jan '14

31 Jan '14

On Thu, Jan 30, 2014 at 6:50 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Matt, > > Ok, I took a look at that and see why I deleted it. > > I had hoped my new viewer with PetscLogView_Detailed() which can be > accessed with -log_view :filename:ascii_info_detail could be used instead > of PetscLogViewPython() > I like your new organization. I was trying to output in a format that did not require us to write another parser. I will reintegrate the code into your new setup. Thanks, Matt > It, in theory, provides more information since it provides the values > for each process so you can do any kind of cross process statistics you > want. Or you can just add up the values from each process. > > Anyways, if you can't use it let me know and we can iterate on > PetscLogView_Detailed() to make it useful. > > Barry > > > On Jan 30, 2014, at 4:31 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > > > I was using that. > > > > Matt > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] configure failed after update of OSX
by Sean Farley 30 Jan '14

30 Jan '14

mfadams(a)lbl.gov writes: >> >> >> $ sudo port install mpich-default >> $ sudo port select --set mpi mpich-mp-fortran >> >> Then try to build PETSc (minus the --download-mpich and CC, CXX, FC >> options). > > > That worked Awesome >> You could also try to install most of the external packages: >> >> $ sudo port install hypre ml parmetis metis hdf5-18 netcdf triangle \ >> superlu superlu_dist mumps >> > > This instal seemed to work but how would I configure PETSc to use them? I have a bash function to help me configure but it's basically this: $ ./configure --with-{hdf5,netcdf,ml,hypre,metis,parmetis,scalapack,mumps,superlu,superlu_dist}-dir=/opt/local

1 0

Re: [petsc-dev] configure failed after update of OSX
by Mark Adams 30 Jan '14

30 Jan '14

> > > $ sudo port install mpich-default > $ sudo port select --set mpi mpich-mp-fortran > > Then try to build PETSc (minus the --download-mpich and CC, CXX, FC > options). That worked > You could also try to install most of the external packages: > > $ sudo port install hypre ml parmetis metis hdf5-18 netcdf triangle \ > superlu superlu_dist mumps > This instal seemed to work but how would I configure PETSc to use them?

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Barry Smith 30 Jan '14

30 Jan '14

On Jan 30, 2014, at 6:02 PM, Jed Brown <jed(a)jedbrown.org> wrote: > Barry Smith <bsmith(a)mcs.anl.gov> writes: > >> On Jan 30, 2014, at 3:00 PM, Jed Brown <jed(a)jedbrown.org> wrote: >> >> Yes, but in the debugger (or with a printf %p there is no portable >> way for a user to know that the pointer is invalid) whereas with >> NULL they would immediately know. A null pointer that leads to a >> segv conveys something different to someone debugging than some >> address where the user doesn’t know if that address came from a >> malloc(0) or some completely different way (like crazy pointer >> arithmetic). > > True, but an array length is needed anyway and any references to that > pointer can still be detected by the memory checker. Also, if someone > really wants to get NULL (or a special value), we can use a malloc > implementation that does so. (We can add a runtime option for this if > someone wants it for debugging.) > >> If you are trying to debug some code that you didn’t personally >> write (that may have been written by an idiot) it is rarely >> possible for you to know where this out-of-bound information is. > > Fair, but you'll have as much trouble doing anything with that code. > >> We would obviously turn off all of this completely when running >> under valgrind to let valgrind do its thing. >> >> Side note; PetscMallocn() needs much more error checking: for >> example PetscMalloc2(-8,&x1,10,&x2) would not be detected as a >> problem but is problematic. > > Is this a problem actually encountered in practice? > >> Since we have to add all this error checking (likely by making >> these beasts functions) adding a return NULL on 0 won’t muck up >> the code much. > > We can't make them functions because we use the type sizes and pass > __FILE__, __LINE__. I'm not motivated to change this unless someone > demonstrates that this is a non-theoretical problem. No one is asking YOU to change this. I would be happy if it was reworked using function calls (obviously with a macro wrapper) that made all the decisions (debug or not etc) runtime rather than compile time and had much more robust error checking, then adding a NULL on 0 feature would be a trivial addition. Note that currently when running optimized with valgrind; valgrind only sees the one_big_malloc so can miss lots of overwrite and underwrites within the chunks malloced. By making everything runtime we can let valgrind have more information and manage correctness of each sub chunk. Barry

1 0

Re: [petsc-dev] What happened to -log_summary_python?
by Barry Smith 30 Jan '14

30 Jan '14

Matt, Ok, I took a look at that and see why I deleted it. I had hoped my new viewer with PetscLogView_Detailed() which can be accessed with -log_view :filename:ascii_info_detail could be used instead of PetscLogViewPython() It, in theory, provides more information since it provides the values for each process so you can do any kind of cross process statistics you want. Or you can just add up the values from each process. Anyways, if you can’t use it let me know and we can iterate on PetscLogView_Detailed() to make it useful. Barry On Jan 30, 2014, at 4:31 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > I was using that. > > Matt > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Jed Brown 30 Jan '14

30 Jan '14

Barry Smith <bsmith(a)mcs.anl.gov> writes: > On Jan 30, 2014, at 3:00 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > Yes, but in the debugger (or with a printf %p there is no portable > way for a user to know that the pointer is invalid) whereas with > NULL they would immediately know. A null pointer that leads to a > segv conveys something different to someone debugging than some > address where the user doesn’t know if that address came from a > malloc(0) or some completely different way (like crazy pointer > arithmetic). True, but an array length is needed anyway and any references to that pointer can still be detected by the memory checker. Also, if someone really wants to get NULL (or a special value), we can use a malloc implementation that does so. (We can add a runtime option for this if someone wants it for debugging.) > If you are trying to debug some code that you didn’t personally > write (that may have been written by an idiot) it is rarely > possible for you to know where this out-of-bound information is. Fair, but you'll have as much trouble doing anything with that code. > We would obviously turn off all of this completely when running > under valgrind to let valgrind do its thing. > > Side note; PetscMallocn() needs much more error checking: for > example PetscMalloc2(-8,&x1,10,&x2) would not be detected as a > problem but is problematic. Is this a problem actually encountered in practice? > Since we have to add all this error checking (likely by making > these beasts functions) adding a return NULL on 0 won’t muck up > the code much. We can't make them functions because we use the type sizes and pass __FILE__, __LINE__. I'm not motivated to change this unless someone demonstrates that this is a non-theoretical problem.

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Barry Smith 30 Jan '14

30 Jan '14

On Jan 30, 2014, at 3:00 PM, Jed Brown <jed(a)jedbrown.org> wrote: > Barry Smith <bsmith(a)mcs.anl.gov> writes: >>> Why do we want this behavior? >> >> Matt feels that having zero space allocated items always as NULL is >> useful. Perhaps as a debugging/error determining issue since no one >> can “accidentally” use the location returned by malloc(0) and in >> the debugger one won’t mistakenly think that a pointer which >> contains something returned by malloc(0) has a legitimate value in >> it. > > The pointer returned by malloc(0) [e.g., optimized mode] is actually > invalid. Yes, but in the debugger (or with a printf %p there is no portable way for a user to know that the pointer is invalid) whereas with NULL they would immediately know. A null pointer that leads to a segv conveys something different to someone debugging than some address where the user doesn’t know if that address came from a malloc(0) or some completely different way (like crazy pointer arithmetic). > PETSc checks in debug mode; valgrind tracks zero-size > allocations. > >>> Can we see some concrete code that is affected by this behavior? We >>> needed PetscMalloc[2-7] to work in optimized mode, so nothing in >>> PETSc should be depending on it. >> >> What does “it” here refer to? And what do you mean by “depending >> on”. Certainly we could make PetscMallocn() and PetscFreen() >> behave in a way Matt suggests even in optimized code (if we choose >> to). > > I'm assuming that current code is correct, thus I'm asking where > significant simplifications would occur if the caller knew that > zero-sized arrays were NULL. There would no know simplifications in the code; it would allegedly make debugging sometimes easier > >> Some notes: >> >> We currently mark freed pointers with a NULL so they cannot be >> accidentally used at a later time. >> >> In Jed’s model something a “zero length space” will have the value >> NULL and something not. There is, AFAIK, no tool to check the >> address and see if it is a real pointer or a pointer to an invalid >> location. > > There's no tool to determine whether it can hold more than one entry. > What arrays do we have that (a) do not store size and (b) do not use a > mandatory end marker like '\0' (in which case you had to allocate the > end marker)? > >> With Matt’s scheme we could also mark “zero length space” with a >> NULL. In an ideal world I could see having a different specific >> value for “zero length space” such as INVALIDPOINTER then one could >> distinguish the two values and their different meanings. Alias I >> don’t control the ideal C world. > > Why don't you have this information out-of-band anyway? If you are trying to debug some code that you didn’t personally write (that may have been written by an idiot) it is rarely possible for you to know where this out-of-bound information is. > >> I agree with Jed that we don’t want any PETSc code to be dependent >> on PetscMallocn(0) always being NULL in all cases; but I understand >> Matt’s argument that having PetscMallocn() always being NULL makes >> debugging easier. > > I'm not convinced of this. Memory checkers will notice accesses (and be > able to track back to where the zero-length array was "allocated”). We would obviously turn off all of this completely when running under valgrind to let valgrind do its thing. Side note; PetscMallocn() needs much more error checking: for example PetscMalloc2(-8,&x1,10,&x2) would not be detected as a problem but is problematic. Since we have to add all this error checking (likely by making these beasts functions) adding a return NULL on 0 won’t muck up the code much. Barry

1 0