April 2016 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Make PetscHash publicly accessible
by Matthew Knepley 22 Apr '16

22 Apr '16

On Fri, Apr 22, 2016 at 9:52 AM, Jed Brown <jed(a)jedbrown.org> wrote: > Matthew Knepley <knepley(a)gmail.com> writes: > > You are wrong in this case. The HashIJ is not trivial, and cannot be done > > easily inline. What is the problem here? > > $ git grep -l 'PetscHashIJ[^K]' > src/sys/examples/tests/ex26.c > src/sys/utils/hash.h > > $ git grep -l 'PetscHashJK' > src/dm/impls/plex/plexfem.c > src/mat/impls/preallocator/matpreallocator.c > src/sys/utils/hash.h > > But matpreallocator.c doesn't actually use the linked list aspects of > PetscHashJK, so it's memory overhead (two unused pointers plus padding > per entry) for nothing. > > $ git grep -l 'PetscHashIJKL' > src/dm/impls/plex/plexinterpolate.c > src/sys/utils/hash.h > > This is actually used, but I think the implementation (below) accurately > fits my description of wrapping khash primitives with imaginary error > checking. Lawrence, do you just want a general-purpose hash table or do > you specifically want Matt's hash/linked-list structure? > I assume you would just stick this stuff in source files? /* Linked list of values in a bucket. */ struct _IJKLNode { PetscInt k; struct _IJKLNode *next; }; typedef struct _IJKLNode IJKLNode; /* Value (holds a linked list of nodes) in the bucket. */ struct _IJKLVal { PetscInt n; IJKLNode *head, *tail; }; typedef struct _IJKLVal IJKLVal; /* Key (a quartet of integers). */ struct _PetscHashIJKLKey { PetscInt i, j, k, l; }; typedef struct _PetscHashIJKLKey PetscHashIJKLKey; /* Hash function: mix two integers into one. Shift by a quarter the number of bits in PetscInt to the left and then XOR. If the indices fit into the lowest quarter part of PetscInt, this is a bijection. We should shift by (8/4)*sizeof(PetscInt): sizeof(PetscInt) is the number of bytes in PetscInt, with 8 bits per byte. */ #define IJKLKeyHash(key) ( (((key).i) << (4*sizeof(PetscInt)))^((key).j)^(((key).k) << (2*sizeof(PetscInt)))^(((key).l) << (6*sizeof(PetscInt))) ) /* Compare two keys (integer pairs). */ #define IJKLKeyEqual(k1,k2) (((k1).i==(k2).i) ? ((k1).j==(k2).j) ? ((k1).k==(k2).k) ? ((k1).l==(k2).l) : 0 : 0 : 0) KHASH_INIT(HASHIJKL,PetscHashIJKLKey,IJKLVal,1,IJKLKeyHash,IJKLKeyEqual) struct _PetscHashIJKL { khash_t(HASHIJKL) *ht; }; typedef struct _PetscHashIJKL *PetscHashIJKL; typedef khiter_t PetscHashIJKLIter; typedef IJKLNode *PetscHashIJKLValIter; This is a silly argument. It is argument for the sake of arguing rather than to improve what we are doing. Matt > #undef __FUNCT__ > #define __FUNCT__ "PetscHashIJKLCreate" > PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLCreate(PetscHashIJKL *h) > { > PetscErrorCode ierr; > > PetscFunctionBegin; > PetscValidPointer(h, 1); > ierr = PetscNew((h));CHKERRQ(ierr); > (*h)->ht = kh_init(HASHIJKL); > PetscFunctionReturn(0); > } > > #undef __FUNCT__ > #define __FUNCT__ "PetscHashIJKLResize" > PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLResize(PetscHashIJKL h, > PetscInt n) > { > PetscFunctionBegin; > (kh_resize(HASHIJKL, (h)->ht, (n))); > PetscFunctionReturn(0); > } > > #undef __FUNCT__ > #define __FUNCT__ "PetscHashIJKLKeySize" > PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLKeySize(PetscHashIJKL h, > PetscInt *n) > { > PetscFunctionBegin; > ((*n) = kh_size((h)->ht)); > PetscFunctionReturn(0); > } > > #undef __FUNCT__ > #define __FUNCT__ "PetscHashIJKLPut" > /* > PetscHashIJKLPut - Insert key in the hash table > > Input Parameters: > + h - The hash table > - key - The key to insert > > Output Parameter: > + missing - 0 if the key is present in the hash table, 1 if the bucket is > empty (never used), 2 if the element in the bucket has been deleted > - iter - Iterator into table > > Level: developer > > .seealso: PetscHashIJKLCreate(), PetscHashIJKLSet() > */ > PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLPut(PetscHashIJKL h, > PetscHashIJKLKey key, PetscHashIJKLIter *missing, PetscHashIJKLIter *iter) > { > PetscFunctionBeginHot; > *iter = kh_put(HASHIJKL, (h)->ht, (key), missing); > PetscFunctionReturn(0); > } > > #undef __FUNCT__ > #define __FUNCT__ "PetscHashIJKLSet" > /* > PetscHashIJKLSet - Set the value for an iterator in the hash table > > Input Parameters: > + h - The hash table > . iter - An iterator into the table > - value - The value to set > > Level: developer > > .seealso: PetscHashIJKLCreate(), PetscHashIJKLPut(), PetscHashIJKLGet() > */ > PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLSet(PetscHashIJKL h, > PetscHashIJKLIter iter, PetscInt value) > { > PetscFunctionBeginHot; > kh_val((h)->ht, iter).n = value; > PetscFunctionReturn(0); > } > > #undef __FUNCT__ > #define __FUNCT__ "PetscHashIJKLGet" > /* > PetscHashIJKLGet - Get the value for an iterator in the hash table > > Input Parameters: > + h - The hash table > . iter - An iterator into the table > > Output Parameters: > . value - The value to get > > Level: developer > > .seealso: PetscHashIJKLCreate(), PetscHashIJKLPut(), PetscHashIJKLSet() > */ > PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLGet(PetscHashIJKL h, > PetscHashIJKLIter iter, PetscInt *value) > { > PetscFunctionBeginHot; > *value = kh_val((h)->ht, iter).n; > PetscFunctionReturn(0); > } > > #undef __FUNCT__ > #define __FUNCT__ "PetscHashIJKLClear" > PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLClear(PetscHashIJKL h) > { > PetscFunctionBegin; > kh_clear(HASHIJKL, (h)->ht); > PetscFunctionReturn(0); > } > > #undef __FUNCT__ > #define __FUNCT__ "PetscHashIJKLDestroy" > PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLDestroy(PetscHashIJKL *h) > { > PetscFunctionBegin; > PetscValidPointer(h, 1); > if ((*h)) { > PetscErrorCode ierr; > > if ((*h)->ht) { > kh_destroy(HASHIJKL, (*h)->ht); > (*h)->ht = NULL; > } > ierr = PetscFree((*h));CHKERRQ(ierr); > } > PetscFunctionReturn(0); > } > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Make PetscHash publicly accessible
by Jed Brown 22 Apr '16

22 Apr '16

Matthew Knepley <knepley(a)gmail.com> writes: > You are wrong in this case. The HashIJ is not trivial, and cannot be done > easily inline. What is the problem here? $ git grep -l 'PetscHashIJ[^K]' src/sys/examples/tests/ex26.c src/sys/utils/hash.h $ git grep -l 'PetscHashJK' src/dm/impls/plex/plexfem.c src/mat/impls/preallocator/matpreallocator.c src/sys/utils/hash.h But matpreallocator.c doesn't actually use the linked list aspects of PetscHashJK, so it's memory overhead (two unused pointers plus padding per entry) for nothing. $ git grep -l 'PetscHashIJKL' src/dm/impls/plex/plexinterpolate.c src/sys/utils/hash.h This is actually used, but I think the implementation (below) accurately fits my description of wrapping khash primitives with imaginary error checking. Lawrence, do you just want a general-purpose hash table or do you specifically want Matt's hash/linked-list structure? #undef __FUNCT__ #define __FUNCT__ "PetscHashIJKLCreate" PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLCreate(PetscHashIJKL *h) { PetscErrorCode ierr; PetscFunctionBegin; PetscValidPointer(h, 1); ierr = PetscNew((h));CHKERRQ(ierr); (*h)->ht = kh_init(HASHIJKL); PetscFunctionReturn(0); } #undef __FUNCT__ #define __FUNCT__ "PetscHashIJKLResize" PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLResize(PetscHashIJKL h, PetscInt n) { PetscFunctionBegin; (kh_resize(HASHIJKL, (h)->ht, (n))); PetscFunctionReturn(0); } #undef __FUNCT__ #define __FUNCT__ "PetscHashIJKLKeySize" PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLKeySize(PetscHashIJKL h, PetscInt *n) { PetscFunctionBegin; ((*n) = kh_size((h)->ht)); PetscFunctionReturn(0); } #undef __FUNCT__ #define __FUNCT__ "PetscHashIJKLPut" /* PetscHashIJKLPut - Insert key in the hash table Input Parameters: + h - The hash table - key - The key to insert Output Parameter: + missing - 0 if the key is present in the hash table, 1 if the bucket is empty (never used), 2 if the element in the bucket has been deleted - iter - Iterator into table Level: developer .seealso: PetscHashIJKLCreate(), PetscHashIJKLSet() */ PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLPut(PetscHashIJKL h, PetscHashIJKLKey key, PetscHashIJKLIter *missing, PetscHashIJKLIter *iter) { PetscFunctionBeginHot; *iter = kh_put(HASHIJKL, (h)->ht, (key), missing); PetscFunctionReturn(0); } #undef __FUNCT__ #define __FUNCT__ "PetscHashIJKLSet" /* PetscHashIJKLSet - Set the value for an iterator in the hash table Input Parameters: + h - The hash table . iter - An iterator into the table - value - The value to set Level: developer .seealso: PetscHashIJKLCreate(), PetscHashIJKLPut(), PetscHashIJKLGet() */ PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLSet(PetscHashIJKL h, PetscHashIJKLIter iter, PetscInt value) { PetscFunctionBeginHot; kh_val((h)->ht, iter).n = value; PetscFunctionReturn(0); } #undef __FUNCT__ #define __FUNCT__ "PetscHashIJKLGet" /* PetscHashIJKLGet - Get the value for an iterator in the hash table Input Parameters: + h - The hash table . iter - An iterator into the table Output Parameters: . value - The value to get Level: developer .seealso: PetscHashIJKLCreate(), PetscHashIJKLPut(), PetscHashIJKLSet() */ PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLGet(PetscHashIJKL h, PetscHashIJKLIter iter, PetscInt *value) { PetscFunctionBeginHot; *value = kh_val((h)->ht, iter).n; PetscFunctionReturn(0); } #undef __FUNCT__ #define __FUNCT__ "PetscHashIJKLClear" PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLClear(PetscHashIJKL h) { PetscFunctionBegin; kh_clear(HASHIJKL, (h)->ht); PetscFunctionReturn(0); } #undef __FUNCT__ #define __FUNCT__ "PetscHashIJKLDestroy" PETSC_STATIC_INLINE PetscErrorCode PetscHashIJKLDestroy(PetscHashIJKL *h) { PetscFunctionBegin; PetscValidPointer(h, 1); if ((*h)) { PetscErrorCode ierr; if ((*h)->ht) { kh_destroy(HASHIJKL, (*h)->ht); (*h)->ht = NULL; } ierr = PetscFree((*h));CHKERRQ(ierr); } PetscFunctionReturn(0); }

1 0

Re: [petsc-dev] Make PetscHash publicly accessible
by Matthew Knepley 22 Apr '16

22 Apr '16

On Fri, Apr 22, 2016 at 9:25 AM, Jed Brown <jed(a)jedbrown.org> wrote: > Matthew Knepley <knepley(a)gmail.com> writes: > > Sweeping generalization not actually true. The wrappers do setup the > > particular > > keys we want inside PETSc, > > All 3 invocations of KHASH_INIT amid 500 lines. You are wrong in this case. The HashIJ is not trivial, and cannot be done easily inline. What is the problem here? Matt > > > and do bundle some multiple operation calls. > > Some, yes, though some of that bundling produces demonstrably less > efficient interfaces. > > commit da33aca0a8f0fe1631697dcf59d191f75b34b5c6 > Author: Matthew G. Knepley <knepley(a)gmail.com> > Date: Fri Nov 8 15:40:59 2013 -0600 > > Sys ex26: Jed is right that this is much faster > > Modified src/sys/examples/tests/ex26.c > diff --git a/src/sys/examples/tests/ex26.c b/src/sys/examples/tests/ex26.c > index e8cc50a..30ed523 100644 > --- a/src/sys/examples/tests/ex26.c > +++ b/src/sys/examples/tests/ex26.c > @@ -22,8 +22,13 @@ int main(int argc, char **argv) > > key.i = PetscMin(i, j); > key.j = PetscMax(i, j); > +#if 1 > + khint_t ret, idx = kh_put(HASHIJ, table->ht, key, &ret); > + if (ret == 1) kh_val(table->ht, idx).n = newp++; > +#else > ierr = PetscHashIJGet(table, key, &p);CHKERRQ(ierr); > if (p < 0) {ierr = PetscHashIJAdd(table, key, > newp++);CHKERRQ(ierr);} > +#endif > } > } > ierr = PetscHashIJDestroy(&table);CHKERRQ(ierr); > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Make PetscHash publicly accessible
by Lawrence Mitchell 22 Apr '16

22 Apr '16

> On 22 Apr 2016, at 15:04, Jed Brown <jed(a)jedbrown.org> wrote: > > Matthew Knepley <knepley(a)gmail.com> writes: > >> On Fri, Apr 22, 2016 at 4:32 AM, Lawrence Mitchell < >> lawrence.mitchell(a)imperial.ac.uk> wrote: >> >>> Hi all, >>> >>> I would like to use the PetscHash functionality in my code. At the moment >>> this is not possible, since the header file hash.h is not public (it's in >>> src/sys/utils/hash.h). Would there be any objections to moving it to (say) >>> include/petschash.h so that PETSc applications can use it as well as PETSc >>> itself. >>> >> >> No, go ahead and do it. > > If you do, then you need to namespace everything so it doesn't conflict > with khash.h from: Sure. > https://github.com/attractivechaos/klib > (or https://github.com/clibs/hash) > Now personally, if I was writing code that was not intended for eventual > inclusion in PETSc, I would just use upstream khash.h directly. (PETSc > should probably upgrade to the latest upstream khash.h.) That is certainly an option, I was potentially being lazy (oh petsc already has a hash implementation, I'll just use that). If I'm the only person coding outside of petsc who's ever going to use this, it may not be worth it. Lawrence

1 0

Re: [petsc-dev] Make PetscHash publicly accessible
by Jed Brown 22 Apr '16

22 Apr '16

Matthew Knepley <knepley(a)gmail.com> writes: > Sweeping generalization not actually true. The wrappers do setup the > particular > keys we want inside PETSc, All 3 invocations of KHASH_INIT amid 500 lines. > and do bundle some multiple operation calls. Some, yes, though some of that bundling produces demonstrably less efficient interfaces. commit da33aca0a8f0fe1631697dcf59d191f75b34b5c6 Author: Matthew G. Knepley <knepley(a)gmail.com> Date: Fri Nov 8 15:40:59 2013 -0600 Sys ex26: Jed is right that this is much faster Modified src/sys/examples/tests/ex26.c diff --git a/src/sys/examples/tests/ex26.c b/src/sys/examples/tests/ex26.c index e8cc50a..30ed523 100644 --- a/src/sys/examples/tests/ex26.c +++ b/src/sys/examples/tests/ex26.c @@ -22,8 +22,13 @@ int main(int argc, char **argv) key.i = PetscMin(i, j); key.j = PetscMax(i, j); +#if 1 + khint_t ret, idx = kh_put(HASHIJ, table->ht, key, &ret); + if (ret == 1) kh_val(table->ht, idx).n = newp++; +#else ierr = PetscHashIJGet(table, key, &p);CHKERRQ(ierr); if (p < 0) {ierr = PetscHashIJAdd(table, key, newp++);CHKERRQ(ierr);} +#endif } } ierr = PetscHashIJDestroy(&table);CHKERRQ(ierr);

1 0

Re: [petsc-dev] Make PetscHash publicly accessible
by Matthew Knepley 22 Apr '16

22 Apr '16

On Fri, Apr 22, 2016 at 9:04 AM, Jed Brown <jed(a)jedbrown.org> wrote: > Matthew Knepley <knepley(a)gmail.com> writes: > > > On Fri, Apr 22, 2016 at 4:32 AM, Lawrence Mitchell < > > lawrence.mitchell(a)imperial.ac.uk> wrote: > > > >> Hi all, > >> > >> I would like to use the PetscHash functionality in my code. At the > moment > >> this is not possible, since the header file hash.h is not public (it's > in > >> src/sys/utils/hash.h). Would there be any objections to moving it to > (say) > >> include/petschash.h so that PETSc applications can use it as well as > PETSc > >> itself. > >> > > > > No, go ahead and do it. > > If you do, then you need to namespace everything so it doesn't conflict > with khash.h from: > > https://github.com/attractivechaos/klib > (or https://github.com/clibs/hash) > > Now personally, if I was writing code that was not intended for eventual > inclusion in PETSc, I would just use upstream khash.h directly. (PETSc > should probably upgrade to the latest upstream khash.h.) > > The PetscHash* wrappers don't add value in my opinion, just consistency > with PETSc style via lots of lines of code that add PetscErrorCode > returns that cannot be anything but 0 because the underlying operations > cannot fail. > Sweeping generalization not actually true. The wrappers do setup the particular keys we want inside PETSc, and do bundle some multiple operation calls. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Make PetscHash publicly accessible
by Jed Brown 22 Apr '16

22 Apr '16

Matthew Knepley <knepley(a)gmail.com> writes: > On Fri, Apr 22, 2016 at 4:32 AM, Lawrence Mitchell < > lawrence.mitchell(a)imperial.ac.uk> wrote: > >> Hi all, >> >> I would like to use the PetscHash functionality in my code. At the moment >> this is not possible, since the header file hash.h is not public (it's in >> src/sys/utils/hash.h). Would there be any objections to moving it to (say) >> include/petschash.h so that PETSc applications can use it as well as PETSc >> itself. >> > > No, go ahead and do it. If you do, then you need to namespace everything so it doesn't conflict with khash.h from: https://github.com/attractivechaos/klib (or https://github.com/clibs/hash) Now personally, if I was writing code that was not intended for eventual inclusion in PETSc, I would just use upstream khash.h directly. (PETSc should probably upgrade to the latest upstream khash.h.) The PetscHash* wrappers don't add value in my opinion, just consistency with PETSc style via lots of lines of code that add PetscErrorCode returns that cannot be anything but 0 because the underlying operations cannot fail.

1 0

Re: [petsc-dev] Make PetscHash publicly accessible
by Matthew Knepley 22 Apr '16

22 Apr '16

On Fri, Apr 22, 2016 at 4:32 AM, Lawrence Mitchell < lawrence.mitchell(a)imperial.ac.uk> wrote: > Hi all, > > I would like to use the PetscHash functionality in my code. At the moment > this is not possible, since the header file hash.h is not public (it's in > src/sys/utils/hash.h). Would there be any objections to moving it to (say) > include/petschash.h so that PETSc applications can use it as well as PETSc > itself. > No, go ahead and do it. Thanks, Matt > Thanks, > > Lawrence > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Make PetscHash publicly accessible
by Lawrence Mitchell 22 Apr '16

22 Apr '16

Hi all, I would like to use the PetscHash functionality in my code. At the moment this is not possible, since the header file hash.h is not public (it's in src/sys/utils/hash.h). Would there be any objections to moving it to (say) include/petschash.h so that PETSc applications can use it as well as PETSc itself. Thanks, Lawrence

1 0

Re: [petsc-dev] --with-shared-libraries=1 is a happy camper if it ends up using a static linker
by Satish Balay 22 Apr '16

22 Apr '16

Pushed https://bitbucket.org/petsc/petsc/commits/016c41562d0c66f2a2a067d47ae42d9f8… Satish On Thu, 21 Apr 2016, Barry Smith wrote: > > Hmm, when someone explicitly sets --with-shared-libraries=1 I think they would be rather upset if configure ends up using ar and static libraries because it could not find options to build shared libraries. > > Yet that is what happens now! > > Could someone add support so that if a user specifically requests --with-shared-libraries=1 configure barfs with a nice error message if it cannot find any? > > I think it is a trivial code change but does require the snippet go in exactly the right place which I cannot determine > > Thanks > > Barry > >

1 0