Yes, displacements for the filetype must be in “monotonically nondecreasing order”. Rajeev

On Aug 22, 2017, at 3:05 PM, pramod kumbhar <[email protected]> wrote:

Hi Rob,

Thanks! Below is not exactly same issue/error but related :

While constructing derived datatype (filetype used for set_view), do we need displacements / offsets to be in ascending order? I mean, suppose I am creating derived datatype using MPI_Type_create_hindexed (or mpi struct) with length/displacements as:

blocklengths[0] = 8; blocklengths[1] = 231670; blocklengths[2] = 116606;

displacements[0] = 0; displacements[1] = 8; displacements[2] = 231678;

Above displacements are in ascending order. Suppose I shuffle order bit:

blocklengths[0] = 8; blocklengths[1] = 116606; blocklengths[2] = 231670;

displacements[0] = 0; displacements[1] = 231678; displacements[2] = 8;

It's still the same but while specifying block-lengths/offsets I changed the order. (resultant file will have data in different oder but that's ignored here) Isn't this a valid specification? This second example results in a segfault (in ADIO_GEN_WriteStrided / Coll).

I quickly wrote attached program, let me know if I have missed anything obvious here.

Regards, Pramod

p.s. you can compile & run as:

Not working => mpicxx test.cpp && mpirun -n 2 ./a.out Working =>. mpicxx test.cpp -DUSE_ORDER && mpirun -n 2 ./a.out

On Tue, Aug 22, 2017 at 5:25 PM, Latham, Robert J. <[email protected]> wrote: On Mon, 2017-08-21 at 17:45 +0200, pramod kumbhar wrote:

...

Dear All,

In one of our application I am seeing following error while using collective call MPI_File_write_all :

Error in ADIOI_Calc_aggregator(): rank_index(1) >= fd->hints-

...

cb_nodes (1) fd_size=102486061 off=102486469

Non collective version works fine.

While looking at callstack I came across below comment in mpich- 3.2/src/mpi/romio/adio/common/ad_aggregate.c :

/* we index into fd_end with rank_index, and fd_end was allocated to be no * bigger than fd->hins->cb_nodes. If we ever violate that, we're * overrunning arrays. Obviously, we should never ever hit this abort */ if (rank_index >= fd->hints->cb_nodes || rank_index < 0) { FPRINTF(stderr, "Error in ADIOI_Calc_aggregator(): rank_index(%d) >= fd->hints->cb_nodes (%d) fd_size=%lld off=%lld\n", rank_index,fd->hints->cb_nodes,fd_size,off); MPI_Abort(MPI_COMM_WORLD, 1); }

I am going to look into application and see if there is an issue with offset overflow. But looking at above comment ("Obviously, we should never ever hit this abort ") I thought should ask if there is any obvious thing I am missing.

that's my comment. The 'rank_index' array is allocated based on the 'cb_nodes' hint. I definitely would like to know more about how the code is manipulating rank_index, cb_nodes, and fd_end .

If there is a reduced test case you can send me, that will be a huge help.

==rob

...

Regards, Pramod

p.s. I will provide reproducer after looking into this more carefully. _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

<test.cpp>_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss