On Mon, Feb 25, 2013 at 9:22 AM, Jim Dinan <[email protected]> wrote:

On 2/25/13 9:08 AM, Jeff Hammond wrote:

...

Is it the case that entering into MPI will lead to processing of all packets in an incoming RMA op? Does the receipt of the first packet from an MPI_Accumulate cause the target to sit inside of the progress engine until all packages in the message have arrived or does the passive target merely process the packets that arrive while it is inside of MPI and then return.

Hi Jeff,

It depends on the op. We start processing the op as soon as the packet header arrives. For ops that send data to the target, if the data (payload and datatype) has not all arrived at the time we enter the progress engine, the op will be enqueued to finish later using an internal/device-level request. Ops that send a response to the origin process, always enqueue a request that completes when we finish sending the response message. Some ops (notably, CAS and FOP and also Acc in some cases) embed the data payload in the packet header, allowing us to handle them completely with a single iteration through the progress engine.

Is it possible to add an option to force the progress engine to finish RMA ops once they start? That might help NWChem since it does larger transfers, particularly Acc. Could this be an MPICH-specific info key on the window? Thanks, Jeff -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute [email protected] / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond