https://bitbucket.org/petsc/petsc/issue/46/replace-petscsubcomm-with-the-inf... On Jul 31, 2013, at 10:59 AM, Jed Brown <[email protected]> wrote:

Barry Smith <[email protected]> writes:

...

Jed,

Hong and I were discussing MatGetRedundantMatrix() etc and she mention that you don't like PetscSubcomm. I suggested removing the PetscSubcomm object from PETSc but retaining the functionality of PetscSubcomm by putting the subcomm information into an MPI attribute inside an MPI_Comm (so this new comm behaves like a MPI_Comm subclass with extra data over a regular MPI_Comm) and having something like PetscCommSublimate(comm, …., &subcomm with attribute). Then in all the places in PETSc that take PetscSubcomm they would instead take a MPI_Comm and check for the attribute and use the additional information if it is there.

I would be inclined to remove dependence on this extra information whenever possible. For example, I think it is counter-productive in MatGetRedundantMatrix because it imposes a distribution that has poor locality. But if we need that extra information, or can do something better if it is there, I prefer attaching it as an attribute to having a wrapper (PetscSubcomm).

...

Managing destruction will be tricky due to possible reference loops but presumably we can figure out a way to do it.

What is missing in the MPI attribute callbacks to implement reference counting? (We already use it for this purpose.)