Mark Miller wrote:

Most HPC codes I have worked on have parallel-specific code blocks of one form or another.

Are these runtime or compile-time branches? Which branch is executed when you have built in parallel and run with mpirun -n 1 ?

That isn't so much to address MPI-linkage issues in serial as it is to address situations in which the actual logic and work to be done is different in serial and parallel.

PETSc's stubs must have been getting included inadvertently, there are no symbols because they are just macros. An alternative is to namespace the stubs appropriately.

In fact PETSC is the ONLY HPC code I am aware of that does this. And, I recall many years ago running into MPI name collisions when using it in Ale3d. For some reason, Ale3d's MPI_Init() call wound up resolving to PETSC's dummy implementation.

Others that I use daily include include Hypre, MUMPS, and ML (and I think other Trilinos packages). There is also this package http://people.sc.fsu.edu/~burkardt/c_src/mpi_stubs/mpi_stubs.html

I think it is a problem in general for different MPI implementations to 'live' in the same executable, even if one of them is a 'serial dummy'. And, there isn't necessarily any reason to expect that a parallel application is always going to use a given parallel-capable library, in parallel as well. HDF5 is a great example. We at LLNL don't use is it in parallel though it is used from parallel apps. If HDF5 had taken the dummy MPI approach, that would present a serious problem.

The only problem here is if MPI types are part of the serial interface. The MPI standard does not define an ABI so, e.g. sizeof(MPI_Comm) == sizeof(int) with MPICH2 but sizeof(MPI_Comm) == sizeof(void*) with Open MPI. As long as any symbols provided by the stubs (if they even define any) don't use the MPI namespace, there should be no problem. Jed