January 2014 - petsc-dev - mailman3.cels.anl.gov

What happened to -log_summary_python?
by Matthew Knepley 30 Jan '14

30 Jan '14

I was using that. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Jed Brown 30 Jan '14

30 Jan '14

Barry Smith <bsmith(a)mcs.anl.gov> writes: >> Why do we want this behavior? > > Matt feels that having zero space allocated items always as NULL is > useful. Perhaps as a debugging/error determining issue since no one > can “accidentally” use the location returned by malloc(0) and in > the debugger one won’t mistakenly think that a pointer which > contains something returned by malloc(0) has a legitimate value in > it. The pointer returned by malloc(0) [e.g., optimized mode] is actually invalid. PETSc checks in debug mode; valgrind tracks zero-size allocations. >> Can we see some concrete code that is affected by this behavior? We >> needed PetscMalloc[2-7] to work in optimized mode, so nothing in >> PETSc should be depending on it. > > What does “it” here refer to? And what do you mean by “depending > on”. Certainly we could make PetscMallocn() and PetscFreen() > behave in a way Matt suggests even in optimized code (if we choose > to). I'm assuming that current code is correct, thus I'm asking where significant simplifications would occur if the caller knew that zero-sized arrays were NULL. > Some notes: > > We currently mark freed pointers with a NULL so they cannot be > accidentally used at a later time. > > In Jed’s model something a “zero length space” will have the value > NULL and something not. There is, AFAIK, no tool to check the > address and see if it is a real pointer or a pointer to an invalid > location. There's no tool to determine whether it can hold more than one entry. What arrays do we have that (a) do not store size and (b) do not use a mandatory end marker like '\0' (in which case you had to allocate the end marker)? > With Matt’s scheme we could also mark “zero length space” with a > NULL. In an ideal world I could see having a different specific > value for “zero length space” such as INVALIDPOINTER then one could > distinguish the two values and their different meanings. Alias I > don’t control the ideal C world. Why don't you have this information out-of-band anyway? > I agree with Jed that we don’t want any PETSc code to be dependent > on PetscMallocn(0) always being NULL in all cases; but I understand > Matt’s argument that having PetscMallocn() always being NULL makes > debugging easier. I'm not convinced of this. Memory checkers will notice accesses (and be able to track back to where the zero-length array was "allocated").

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Barry Smith 30 Jan '14

30 Jan '14

On Jan 30, 2014, at 2:15 PM, Jed Brown <jed(a)jedbrown.org> wrote: > Barry Smith <bsmith(a)mcs.anl.gov> writes: >>> Here is what I suggestion. Someone suggest (i.e.. write) a refactorization of PetscMalloc(), PetscMallocn() that handles correctly any of the sizes being zero correctly in a branch and see how it goes. >>> >>> I have pushed it. It looks simple to me >> >> I don’t see how this can possibly work >> >> #if defined(PETSC_USE_DEBUG) >> #define PetscFree2(m1,m2) (PetscFree(m2) || PetscFree(m1)) >> #else >> #define PetscFree2(m1,m2) ((m2)=0, PetscFree(m1)) >> #endif >> >> You need to free the first m[i] that is not null. If m1 was null in the optimized form you never free anything. >> >> These macros are getting horrible, can they become something cleaner? > > Why do we want this behavior? Matt feels that having zero space allocated items always as NULL is useful. Perhaps as a debugging/error determining issue since no one can “accidentally” use the location returned by malloc(0) and in the debugger one won’t mistakenly think that a pointer which contains something returned by malloc(0) has a legitimate value in it. > Can we see some concrete code that is > affected by this behavior? We needed PetscMalloc[2-7] to work in > optimized mode, so nothing in PETSc should be depending on it. What does “it” here refer to? And what do you mean by “depending on”. Certainly we could make PetscMallocn() and PetscFreen() behave in a way Matt suggests even in optimized code (if we choose to). Some notes: We currently mark freed pointers with a NULL so they cannot be accidentally used at a later time. In Jed’s model something a “zero length space” will have the value NULL and something not. There is, AFAIK, no tool to check the address and see if it is a real pointer or a pointer to an invalid location. With Matt’s scheme we could also mark “zero length space” with a NULL. In an ideal world I could see having a different specific value for “zero length space” such as INVALIDPOINTER then one could distinguish the two values and their different meanings. Alias I don’t control the ideal C world. I agree with Jed that we don’t want any PETSc code to be dependent on PetscMallocn(0) always being NULL in all cases; but I understand Matt’s argument that having PetscMallocn() always being NULL makes debugging easier. Barry

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Jed Brown 30 Jan '14

30 Jan '14

Barry Smith <bsmith(a)mcs.anl.gov> writes: >> Here is what I suggestion. Someone suggest (i.e.. write) a refactorization of PetscMalloc(), PetscMallocn() that handles correctly any of the sizes being zero correctly in a branch and see how it goes. >> >> I have pushed it. It looks simple to me > > I don’t see how this can possibly work > > #if defined(PETSC_USE_DEBUG) > #define PetscFree2(m1,m2) (PetscFree(m2) || PetscFree(m1)) > #else > #define PetscFree2(m1,m2) ((m2)=0, PetscFree(m1)) > #endif > > You need to free the first m[i] that is not null. If m1 was null in the optimized form you never free anything. > > These macros are getting horrible, can they become something cleaner? Why do we want this behavior? Can we see some concrete code that is affected by this behavior? We needed PetscMalloc[2-7] to work in optimized mode, so nothing in PETSc should be depending on it.

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Barry Smith 30 Jan '14

30 Jan '14

On Jan 30, 2014, at 1:55 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > On Thu, Jan 30, 2014 at 1:53 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Jan 30, 2014, at 12:46 PM, Brad Aagaard <baagaard(a)usgs.gov> wrote: > > > On 01/30/2014 10:34 AM, Jed Brown wrote: > >> Brad Aagaard <baagaard(a)usgs.gov> writes: > >> > >>> Matt and Jed, > >>> > >>> I see that Jed pushed some changes (jed/malloc-zero) for PetscMalloc to > >>> deal with memory alignment and a zero size. It looks like the pointer > >>> will NOT be NULL for a size of 0. Is this true? > >> > >> Yes, just like malloc(), it can be either a unique pointer or NULL. You > >> need the size anyway to know how many elements are in the array. > > > > I thought it was a nice feature that PETSc improved on malloc() and free() by returning NULL for zero sized allocation (although this wasn't true for --with-debugging=0 due to memory alignment) and set pointers to NULL after freeing. > > With PetscMalloc() we could certainly return NULL on 0 mallocs but the code is a bit more involved for the PetscMallocn() case. > > Here is what I suggestion. Someone suggest (i.e.. write) a refactorization of PetscMalloc(), PetscMallocn() that handles correctly any of the sizes being zero correctly in a branch and see how it goes. > > I have pushed it. It looks simple to me I don’t see how this can possibly work #if defined(PETSC_USE_DEBUG) #define PetscFree2(m1,m2) (PetscFree(m2) || PetscFree(m1)) #else #define PetscFree2(m1,m2) ((m2)=0, PetscFree(m1)) #endif You need to free the first m[i] that is not null. If m1 was null in the optimized form you never free anything. These macros are getting horrible, can they become something cleaner? Barry > > Matt > > > Barry > > > > > > What is the rationale for not returning NULL for mallocs of size zero other than conforming to C malloc behavior? > > > > Brad > > > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Matthew Knepley 30 Jan '14

30 Jan '14

On Thu, Jan 30, 2014 at 1:53 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Jan 30, 2014, at 12:46 PM, Brad Aagaard <baagaard(a)usgs.gov> wrote: > > > On 01/30/2014 10:34 AM, Jed Brown wrote: > >> Brad Aagaard <baagaard(a)usgs.gov> writes: > >> > >>> Matt and Jed, > >>> > >>> I see that Jed pushed some changes (jed/malloc-zero) for PetscMalloc to > >>> deal with memory alignment and a zero size. It looks like the pointer > >>> will NOT be NULL for a size of 0. Is this true? > >> > >> Yes, just like malloc(), it can be either a unique pointer or NULL. You > >> need the size anyway to know how many elements are in the array. > > > > I thought it was a nice feature that PETSc improved on malloc() and > free() by returning NULL for zero sized allocation (although this wasn't > true for --with-debugging=0 due to memory alignment) and set pointers to > NULL after freeing. > > With PetscMalloc() we could certainly return NULL on 0 mallocs but the > code is a bit more involved for the PetscMallocn() case. > > Here is what I suggestion. Someone suggest (i.e.. write) a > refactorization of PetscMalloc(), PetscMallocn() that handles correctly any > of the sizes being zero correctly in a branch and see how it goes. I have pushed it. It looks simple to me Matt > > Barry > > > > > > What is the rationale for not returning NULL for mallocs of size zero > other than conforming to C malloc behavior? > > > > Brad > > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Barry Smith 30 Jan '14

30 Jan '14

On Jan 30, 2014, at 12:46 PM, Brad Aagaard <baagaard(a)usgs.gov> wrote: > On 01/30/2014 10:34 AM, Jed Brown wrote: >> Brad Aagaard <baagaard(a)usgs.gov> writes: >> >>> Matt and Jed, >>> >>> I see that Jed pushed some changes (jed/malloc-zero) for PetscMalloc to >>> deal with memory alignment and a zero size. It looks like the pointer >>> will NOT be NULL for a size of 0. Is this true? >> >> Yes, just like malloc(), it can be either a unique pointer or NULL. You >> need the size anyway to know how many elements are in the array. > > I thought it was a nice feature that PETSc improved on malloc() and free() by returning NULL for zero sized allocation (although this wasn't true for --with-debugging=0 due to memory alignment) and set pointers to NULL after freeing. With PetscMalloc() we could certainly return NULL on 0 mallocs but the code is a bit more involved for the PetscMallocn() case. Here is what I suggestion. Someone suggest (i.e.. write) a refactorization of PetscMalloc(), PetscMallocn() that handles correctly any of the sizes being zero correctly in a branch and see how it goes. Barry > > What is the rationale for not returning NULL for mallocs of size zero other than conforming to C malloc behavior? > > Brad > >

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Jed Brown 30 Jan '14

30 Jan '14

Matthew Knepley <knepley(a)gmail.com> writes: > Actually, we are violating the malloc standard that Jed sent out. It says > that for 0 size, malloc must return > > NULL or a unique pointer > > However, if we have > > PetscMalloc2(0,&r1,1,&r2) > > then r1 == r2 with the optimized implementation, and the pointers are not > unique. They are still correct to pass to PetscFree2(r1,r2). I thought the point of PetscMalloc[1-7] was to emulate doing a single big malloc and calculating offsets, not to be semantically identical to calling malloc several times (which would then allow separate calls to PetscFree).

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Jed Brown 30 Jan '14

30 Jan '14

Brad Aagaard <baagaard(a)usgs.gov> writes: > I thought it was a nice feature that PETSc improved on malloc() and > free() by returning NULL for zero sized allocation (although this wasn't > true for --with-debugging=0 due to memory alignment) and set pointers to > NULL after freeing. > > What is the rationale for not returning NULL for mallocs of size zero > other than conforming to C malloc behavior? The standard could have easily specified that malloc(0) returns NULL. It didn't because leaving it unspecified allows better memory-checking tools (e.g., you can check for matching free even when your tests happen to allocate 0 bytes). As you mention, PETSc did not guarantee it either --with-debugging=0, so correct code cannot depend on it. We are just now being explicit that we only make the same guarantees as malloc(). We still set pointers to NULL after freeing.

1 0

Re: [petsc-dev] PetscMalloc for size zero
by Matthew Knepley 30 Jan '14

30 Jan '14

On Thu, Jan 30, 2014 at 12:50 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > On Thu, Jan 30, 2014 at 12:46 PM, Brad Aagaard <baagaard(a)usgs.gov> wrote: > >> On 01/30/2014 10:34 AM, Jed Brown wrote: >> >>> Brad Aagaard <baagaard(a)usgs.gov> writes: >>> >>> Matt and Jed, >>>> >>>> I see that Jed pushed some changes (jed/malloc-zero) for PetscMalloc to >>>> deal with memory alignment and a zero size. It looks like the pointer >>>> will NOT be NULL for a size of 0. Is this true? >>>> >>> >>> Yes, just like malloc(), it can be either a unique pointer or NULL. You >>> need the size anyway to know how many elements are in the array. >>> >> >> I thought it was a nice feature that PETSc improved on malloc() and >> free() by returning NULL for zero sized allocation (although this wasn't >> true for --with-debugging=0 due to memory alignment) and set pointers to >> NULL after freeing. >> >> What is the rationale for not returning NULL for mallocs of size zero >> other than conforming to C malloc behavior? > > > There is no such thing as "conforming to C malloc", since that WOULD allow > returning NULL in this case. > Actually, we are violating the malloc standard that Jed sent out. It says that for 0 size, malloc must return NULL or a unique pointer However, if we have PetscMalloc2(0,&r1,1,&r2) then r1 == r2 with the optimized implementation, and the pointers are not unique. Matt > I think we are doing users a disservice by not enforcing this. > > Matt > > >> >> Brad >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0