How is what you see not consistent with "The usual barrier semantics are enforced at the corresponding completion operation (test or wait)..."? I don't know what you mean by "expected output and timing" since I don't share your interpretation of the MPI standard, but I believe that the n=3 case is absolutely consistent with the semantics of MPI_Ibarrier+MPI_Wait. Jeff On Wed, Jun 26, 2013 at 2:54 PM, Jiri Simsa <[email protected]> wrote:
Hi,
I have a question about the semantics of non-blocking collective communication. For example, let's consider MPI_Ibarrier. The MPI 3.0 standard specifies that:
"MPI_IBARRIER is a nonblocking version of MPI_BARRIER. By calling MPI_IBARRIER, a process notifies that it has reached the barrier. The call returns immediately, indepen- dent of whether other processes have called MPI_IBARRIER. The usual barrier semantics are enforced at the corresponding completion operation (test or wait), which in the intra- communicator case will complete only after all other processes in the communicator have called MPI_IBARRIER. In the intercommunicator case, it will complete when all processes in the remote group have called MPI_IBARRIER."
My understanding of the standard is that that MPI_Wait(&request, &status), where request has been previously passed into MPI_Ibarrier, returns after all processes in the respective intra-communicator called MPI_Ibarrier. However, the mpich-3.0.4 library, seems to in some cases wait for all processes in the respective intra-communicator to call MPI_Wait. Here is an example that demonstrates this behavior:
#include <mpi.h> #include <unistd.h>
int main( int argc, char *argv[]) { MPI_Request request; MPI_Status status; MPI_Init(&argc, &argv ); int myrank; MPI_Comm_rank(MPI_COMM_WORLD, &myrank); if (myrank == 0) { MPI_Ibarrier(MPI_COMM_WORLD, &request); MPI_Wait(&request, &status); printf("%d, Completed barrier.\n", myrank); } else { MPI_Ibarrier(MPI_COMM_WORLD, &request); sleep(1); MPI_Wait(&request, &status); printf("%d, Completed barrier.\n", myrank); } MPI_Finalize(); return 0; }
When executed with "mpiexec -n 2 ./example", I see the expected output and timing. However, when executed with "mpiexec -n 3 ./example", the call to MPI_Wait in process 0 returns only after the other processes wake up from sleep() and call MPI_Wait.
Isn't this a violation of the standard?
Best,
--Jiri Simsa
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
-- Jeff Hammond [email protected]