On Nov 17, 2009, at 5:08 PM, Jed Brown wrote:
Lisandro Dalcin wrote:
On Mon, Nov 16, 2009 at 4:04 PM, Barry Smith <[email protected]> wrote:
Ok, we could just put the "counter" into the MPI attribute and then just have uniqueness of the names within each communicator. This would solve Jed's problem cleanly.
I agree. +1 on this.
If I understand correctly, every communicator would start at 0 with no attempt to distinguish communicators. PetscObjectName would add an attribute to each communicator it sees. Nope, don't want to do it this way, it increases the complexity. See below.
Browsing tagm.c, MPI_Keyval_free is never called for the other attributes. This explains some valgrind warnings under Open MPI but not under MPICH (presumably MPICH frees zombie keyvals in MPI_Finalize() but OMPI does not).
We should refactor the code to move the body of the code block below into PetscIntialize() if (Petsc_Tag_keyval == MPI_KEYVAL_INVALID) { ierr = MPI_Keyval_create(MPI_NULL_COPY_FN,Petsc_DelTag,&Petsc_Tag_keyval, (void*)0);CHKERRQ(ierr); ierr = MPI_Keyval_create (MPI_NULL_COPY_FN,Petsc_DelComm,&Petsc_InnerComm_keyval, (void*)0);CHKERRQ(ierr); ierr = MPI_Keyval_create (MPI_NULL_COPY_FN,Petsc_DelComm,&Petsc_OuterComm_keyval, (void*)0);CHKERRQ(ierr); } and put a corresponding set of frees() into PetscFinalize().
I see that Petsc_InnerComm_keyval and Petsc_OuterComm_keyval just uses the pointer to hold the value, since the counter changes we need to actually malloc.
Do we want the counter to hold a reference count (and be shared across MPI_Comm_dup)?
No, we don't want to share across MPI_Comm_dup(). They are different communicators so should have their own name counters. Note that it would be rare that there are MPI_Comm_dup()s lying around anyways unless the user explicitly calls it.
This is slightly tricky because it's not normally obvious to a user when a duplication occurs, but the numbering would be different if the comm was duped before or after the attribute was set (unpredictable if it's set in PetscObjectName()). OTOH, PetscCheckSameComm will say duped communicators are equivalent so we really have to share the counter. I'm talking myself into starting a reference-counted counter the first time PETSc touches a communicator.
I am not sure what reference counting you are talking about but PETSc currently reference counts how many objects reference a communicator attribute (this is tagvalp[1]). When it gets down to zero it deletes the attribute and the inner communicator. To get a counter for object names you can increase the size of tagvalp[2] to tagvalp[3], then each time you get a name crank the value up by 1, much like PetscCommGetNewTag() No need to add new attributes. Barry
Jed