The current code just charges along if H5Pset_fapl_mpio is not available, but then H5Fcreate fails because ranks are fighting for a new file. #000: H5F.c line 491 in H5Fcreate(): unable to create file major: File accessibilty minor: Unable to open file #001: H5Fint.c line 1305 in H5F_open(): unable to lock the file major: File accessibilty minor: Unable to open file #002: H5FD.c line 1839 in H5FD_lock(): driver lock request failed major: Virtual File Layer minor: Can't update object #003: H5FDsec2.c line 940 in H5FD_sec2_lock(): unable to lock file, errno = 11, error message = 'Resource temporarily unavailable' major: File accessibilty minor: Bad file ID accessed Seems to me that if the size of the communicator is 1, we shouldn't bother trying H5Pset_fapl_mpio and if the comm is larger than 1, we need to *fail* with a sensible error message if H5Pset_fapl_mpio is not available. Does that sound right to the people that use HDF5 more heavily? #if defined(PETSC_HAVE_H5PSET_FAPL_MPIO) PetscStackCallHDF5(H5Pset_fapl_mpio,(plist_id, PetscObjectComm((PetscObject)viewer), info)); #endif /* Create or open the file collectively */ switch (hdf5->btype) { case FILE_MODE_READ: PetscStackCallHDF5Return(hdf5->file_id,H5Fopen,(name, H5F_ACC_RDONLY, plist_id)); break; case FILE_MODE_APPEND: PetscStackCallHDF5Return(hdf5->file_id,H5Fopen,(name, H5F_ACC_RDWR, plist_id)); break; case FILE_MODE_WRITE: PetscStackCallHDF5Return(hdf5->file_id,H5Fcreate,(name, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id)); break; default: SETERRQ(PETSC_COMM_SELF,PETSC_ERR_ORDER, "Must call PetscViewerFileSetMode() before PetscViewerFileSetName()"); }