On Wed, Aug 12, 2015 at 9:27 AM, Dorier, Matthieu <[email protected]> wrote:
Hi,
I'm trying to refactor an MPI code using MPI one-sided communications.
The initial version of the code reads its data from a file containing a 3D array of floats. Each process has a series of subdomains (blocks) to load from the file, so they all open the file and then issue a series of MPI_File_set_view and MPI_File_read. The type passed to MPI_File_set_view is constructed using MPI_Type_create_subarray to match the block that needs to be loaded.
This code performs very poorly even at small scale: the file is 7GB but the blocks are a few hundreds of bytes, and each process has many blocks to load.
What is the root cause for the poor performance here?
Instead, I would like to have process rank 0 load the entire file, then expose it over RMA. I'm not
I don't understand why you want to serialize your I/O through rank 0, limit your code to loading files that fit into the memory of one process, and then force every single process to obtain data via a point-to-point operation with one process. This seems triply unscalable. Perhaps you can address your performance issues without compromising scalability. Do you have a MCVE (http://stackoverflow.com/help/mcve) for this? Jeff
familiar at all with MPI one-sided operations, since I never used them before, but I guess there should be a simple way to reuse the subarray datatype of my MPI_File_set_view and use it in the context of an MPI_Get. I'm just not sure what the arguments of this MPI_Get would be. My guess is: origin_count would be the number of floats in a single block, origin_datatype would be MPI_FLOAT, target_rank = 0, target_disp = 0, target_count = 1, target_datatype = my subarray datatype. Would that be correct?
Thanks,
Matthieu
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
-- Jeff Hammond [email protected] http://jeffhammond.github.io/ _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss