I got the following error trying to slurp in a large file (slice-17.pentago) with MPI_File_read_ordered: rank 0: pentago/mpi/io.cpp:read_sections:397: MPI_File_read_ordered(file,raw.data(),raw.size(),MPI_BYTE,MPI_STATUS_IGNO ADIOI_CRAY_OPEN(102): Access denied to file all-1/.slice-17.pentago.shfp.670064 The directory all-1 was read only due to paranoia over accidentally deleting expensively obtained data. Looking at ADIOI_Shfp_fname in the mpich source, it looks like this error is intentional: the shared file pointer routines generate temporary files in the same directory as the read-from file. I couldn't find any attempts at recovery if the file cannot be written in that place. This behavior doesn't seem to have changed since the beginning of the git repository. Is my reading of the code correct: MPI_File_read_ordered can't be used on a file in a read only directory? I can see the motivation for this in the case of noncollective shared routines: since the other processes aren't necessarily doing any MPI at the moment, the only way to synchronize is through the file system. And because there might be all sorts of different filesystems in operation, the easiest way to ensure that we're touching the right one is use the same directory. I can't imagine any reasonable use of the noncollective shared routines, but maybe that's a different discussion. Is this an unfortunate leak between broken routines which need questionable trickery and perfectly good routines like MPI_File_read_ordered? Thanks, Geoffrey
participants (1)
-
Geoffrey Irving