I think it is because MPICH just crosses over the eager to rendezvous mode threshold, when  n goes from 9999 to 99999.  OpenMPI certainly uses a different threshold than MPICH.

When you install MPICH, a utility program mpivars is also installed. Type 'mpivars | grep EAGER',  you will get default values for various eager thresholds. 

In your case, export MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE=5000000 and you will get the same result as OpenMPI. 

--Junchao Zhang

On Wed, Nov 5, 2014 at 11:20 AM, Zhen Wang <toddwz@gmail.com> wrote:

Hi MPIers,

I have some questions regarding MPI_Isend() and MPI_Recv(). MPICH-3.1.3 is used to compile and run the attached code on Red Hat Enterprise Linux 6.3 (a shared memory machine). While n = 9999, the MPI_Recv() finishes immediately after MPI_Isend(): (This is what I understand and expect)

MPI 1: Recv started at 09:53:53.
MPI 0: Isend started at 09:53:53.
MPI 1: Recv finished at 09:53:53.
MPI 0: Isend finished at 09:53:58.

When n = 99999, I get the following. The MPI_Recv() finishes after MPI_Wait():

MPI 1: Recv started at 09:47:56.
MPI 0: Isend started at 09:47:56.
MPI 0: Isend finished at 09:48:01.
MPI 1: Recv finished at 09:48:01.

But with OpenMPI 1.8 and n = 99999, MPI_Recv() finishes immediately after MPI_Isend():

MPI 0: Isend started at 09:55:28.
MPI 1: Recv started at 09:55:28.
MPI 1: Recv finished at 09:55:28.
MPI 0: Isend finished at 09:55:33.

Am I misunderstanding something here? In case the attached code is dropped, the code is included. Thanks in advance.

#include "mpi.h"

#include <unistd.h>

#include <stdio.h>

#include "vector"

#include <time.h>

int main(int argc, char* argv[])

{

  MPI_Init(&argc, &argv);

  int rank;

  MPI_Comm_rank(MPI_COMM_WORLD, &rank);

  int n = 9999;

  std::vector<int> vec(n);

  MPI_Request mpiRequest;

  MPI_Status mpiStatus;

  char tt[9] = {0};

  MPI_Barrier(MPI_COMM_WORLD);

  if (rank == 0)

  {

    MPI_Isend(&vec[0], n, MPI_INT, 1, 0, MPI_COMM_WORLD, &mpiRequest);

    time_t t = time(0);

    strftime(tt, 9, "%H:%M:%S", localtime(&t));

    printf("MPI %d: Isend started at %s.\n", rank, tt);

    //int done = 0;

    //while (done == 0)

    //{

    //  MPI_Test(&mpiRequest, &done, &mpiStatus);

    //}

    sleep(5);

    MPI_Wait(&mpiRequest, &mpiStatus);

    t = time(0);

    strftime(tt, 9, "%H:%M:%S", localtime(&t));

    printf("MPI %d: Isend finished at %s.\n", rank, tt);

  }

  else

  {

    time_t t = time(0);

    strftime(tt, 9, "%H:%M:%S", localtime(&t));

    printf("MPI %d: Recv started at %s.\n", rank, tt);

    MPI_Recv(&vec[0], n, MPI_INT, 0, 0, MPI_COMM_WORLD, &mpiStatus);

    t = time(0);

    strftime(tt, 9, "%H:%M:%S", localtime(&t));

    printf("MPI %d: Recv finished at %s.\n", rank, tt);

  }

  MPI_Finalize();

  return 0;

}

Best regards,
Zhen

_______________________________________________
discuss mailing list     discuss@mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________
discuss mailing list     discuss@mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss