On Mon, Jul 16, 2012 at 10:44 PM, Barry Smith <[email protected]> wrote:

/* PETSC_COMM_SELF = PETSC_COMM_WORLD for MPIUNI */ #if !defined(PETSC_HAVE_MPIUNI) ierr = PetscCommDuplicate(PETSC_COMM_WORLD,&icomm,PETSC_NULL);CHKERRQ(ierr); ierr = MPI_Attr_put(icomm,Petsc_ThreadComm_keyval,(void*)tcomm);CHKERRQ(ierr); tcomm->refct++; /* Share the threadcomm with PETSC_COMM_SELF */ #endif

ierr = PetscCommDuplicate(PETSC_COMM_SELF,&icomm,PETSC_NULL);CHKERRQ(ierr); ierr = MPI_Attr_put(icomm,Petsc_ThreadComm_keyval,(void*)tcomm);CHKERRQ(ierr);

I would not do it this way. Instead I would write a general routine that attached a threadcomm to a MPI_Comm; this routine would get the threadcomm_keyval and if it did NOT find it then would be put the attribute, otherwise it would know one was already there. Say it is called PetscThreadCommAttach(MPI_Comm, threadcomm); then in this routine you would just write

PetscThreadCommAttach(PETSC_COMM_WORLD, tcomm); PetscThreadCommAttach(PETSC_COMM_SELF,tcomm); /* won't attr it again for MPIUni because it is already there */

This looks good, but there is also a ref-counting check needed in PetscThreadCommDetach/Destroy because the thread pool (presumably) needs to be closed before PetscFinalize returns.