Hi Matthieu, If you are referring to the requests used by RMA operations, there is a CVAR (MPIR_CVAR_CH3_RMA_ACTIVE_REQ_THRESHOLD) in MPICH which can control number of active RMA requests. Basically what it does is if number of active RMA requests reaches that value, RMA operation will be blockingly waiting until number of active requests is reduced. If you want to tune that CVAR, I suggest you using the latest MPICH (http://www.mpich.org/static/downloads/nightly/master/mpich/), because we recently modify the default value of it. Thanks, Xin ________________________________________ From: Dorier, Matthieu [[email protected]] Sent: Wednesday, August 12, 2015 12:36 PM To: [email protected] Subject: Re: [mpich-discuss] From file reading to memory sharing Hi, I'm gessing the cause of the performance issue is the large number of small requests, because when I read the full 7GB using a single process issuing one single MPI_File_read, it gets much better. Matthieu ________________________________________ From: Wei-keng Liao [[email protected]] Sent: Wednesday, August 12, 2015 11:56 AM To: [email protected] Subject: Re: [mpich-discuss] From file reading to memory sharing Hi, Matthieu If you have "a series of subdomains (blocks) to load from the file", one way to get a better I/O performance is to concatenate the filetypes into a single one and then use only one MPI_File_set_view and one MPI_File_read_all to read the data. FYI. This request aggregation strategy is also used in PnetCDF. Wei-keng On Aug 12, 2015, at 11:27 AM, Dorier, Matthieu wrote:
Hi,
I'm trying to refactor an MPI code using MPI one-sided communications.
The initial version of the code reads its data from a file containing a 3D array of floats. Each process has a series of subdomains (blocks) to load from the file, so they all open the file and then issue a series of MPI_File_set_view and MPI_File_read. The type passed to MPI_File_set_view is constructed using MPI_Type_create_subarray to match the block that needs to be loaded.
This code performs very poorly even at small scale: the file is 7GB but the blocks are a few hundreds of bytes, and each process has many blocks to load.
Instead, I would like to have process rank 0 load the entire file, then expose it over RMA. I'm not familiar at all with MPI one-sided operations, since I never used them before, but I guess there should be a simple way to reuse the subarray datatype of my MPI_File_set_view and use it in the context of an MPI_Get. I'm just not sure what the arguments of this MPI_Get would be. My guess is: origin_count would be the number of floats in a single block, origin_datatype would be MPI_FLOAT, target_rank = 0, target_disp = 0, target_count = 1, target_datatype = my subarray datatype. Would that be correct?
Thanks,
Matthieu _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss