Hi, Huiwei,
I think the best solution for your
question is cuting the data buffers into smaller sizes. However,
This may be not easy.
According to my
experience, the suggestions are
1. Do
not use MPI_Alltoallv, and use nonblocking communication insted, such
as MPI_send and MPI_irecv.
2. In
solution 1, each data buffer can not exceeds 2GB too, If
the data is too large, I think we should develop a
class encapsulated with the data buffer cuting method.
However, using int as the
parameters' type in MPI communication primitives is not reasonable. As we
know, in x86_64 machine, sizeof(int)=4, however sizeof(long) and
sizeof(long long int)=8, we should support the new parameter type of long
and long long int.
jt.meng
Date: 2013-12-14 07:18
Subject: [mpich-discuss] The max amount of data each process
can send in MPI_Alltoallv
Hi, folks,
As I understand, the input parameter of sendcnts is an array of int,
so the amount of data each process can send is limited to MAX_INT (approximately
2G).
What if I want to use MPI_Alltoallv-like MPI routines to send a buffer of
data that exceeds 2GB? Does it need to cut the data buffers to smaller sizes? Is
there an easy way to do this? The reason I am asking is one of my distributed
sort algorithm uses MPI_Alltoallv to distribute data after partitioning; it
crashed when I tried to send in each process more than 2GB data.
Thanks.
--