The way you’re doing things is theoretically fine, but unnecessary. In MPI, it’s fine to not know the size of the message before you receive it (as long as you know the type). Instead of sending around “pre-messages” with all of the meta-data, you can use MPI_Probe (http://www.mpich.org/static/docs/latest/www3/MPI_Probe.html) which will give you information about the next message that you’ll receive (there are other variants that you might prefer to use like MPI_Iprobe, MPI_Mprobe, etc.). Have a look at the documentation at the link above, but generally, you’ll probe MPI to find out information about the next message such as the source and the size. Then you can create your buffers appropriately and avoid lots of extra buffer creation / destruction calls. Thanks, Wesley On Tuesday, May 27, 2014 at 11:15 AM, Grindeanu, Iulian R. wrote:

Hello, We are trying to track down a memory leak, and we are not sure if we are using correctly MPI_ISend and MPI_Irecv

We would like to use non-blocking communication, between pairs of processes.

(it happens during a computation, at every time step; every processor has a list of processors it needs to communicate with; what we do not know, is the size of the messages, in advance)

Let's assume A needs to send to B Because we do not know the size in advance, we send first a fixed size message from A to B; this first message has info about how big the total message needs to be; A knows it has to send more. When B receives the first part, it sends an ack (small size, int 4) to A, acknowledging, and also resizing the local buffer, to fill more data; then A sends the rest.

All these sends / receives use Isend and Irecv, and we try to match the messages, using proper tags. We use different tags for fixed size, ack, and rest of the message.

Before sending the code, I would like to know is what we try is feasible and doable?

We notice that the memory use on processor A keeps increasing, when we do this in a loop, on the order of about 1000-2000 bytes per iteration.

I think we are matching correctly isends and ireceives, and that our buffers are not leaking (we may be wrong)

So again, an iteration is like this: A sends a fixed size message to B; B sends back an ack, and when A receives it, sends the rest, because B has now the proper buffer to receive.

Next iteration, the size of the message is again unknown, so we do again the dance.

Do you have an example like that we can use? Should we use other types of sends/receives?

Thanks, Iulian

Attachments: - ATT00001.txt