Attached is my test code. I've been trying to emulate the gather-scatter semantics in Global Arrays using MPI RMA calls. The top of the file has a bunch of parameters that can be used to configure different protocols for implementing GA-style puts and gets. The failure I'm seeing seems to be independent of the choice of protocol. The parameter USE_STRUCTS can be used to switch between structs and subarrays. Let me know if you have more questions. Bruce -----Original Message----- From: Rob Latham [mailto:[email protected]] Sent: Wednesday, May 25, 2016 9:07 AM To: [email protected] Subject: Re: [mpich-discuss] RMA calls with MPI Datatypes using structs On 05/25/2016 10:40 AM, Palmer, Bruce J wrote:

Hi,

I've been trying to implement code using MPI_Get and MPI_Put that move around data using MPI_Datatypes. If I create the data type using MPI_Type_create_subarray, everything works fine. If I create the data type using MPI_Type_create_struct then I get a failure if the amount of data I'm moving around becomes large. At the moment the crossover point is moving a total of a million doubles on 8 cores (125000 doubles per core), which doesn't seem to be excessively large. The error I'm seeing on a Linux cluster using and Infiniband interconnect is

Assertion failed in file ./src/mpid/ch3/channels/nemesis/include/mpid_nem_inline.h at line 406

: MPID_NEM_MPICH_DATA_LEN - buf_offset >= ext_hdr_sz

internal ABORT - process 1

Note that the same code runs fine if the data type is created using the subarray call. It looks like some internal limit is being exceeded, but if so, I think the threshold may be set a bit on the low side. Does anyone know more about this?

Doesn't look like something we've seen before. Can you send us the working subarray and non-working struct datatype? _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss