Thanks. That solves my issue (both with the code and the reason I wrote it). It means that the statement on 486 about no restrictions is not exactly correct, but and the standard is not a user guide, one can only reason about it with the entire contents in cache. Jeff On Thu, Jan 15, 2015 at 10:18 PM, Thakur, Rajeev <[email protected]> wrote:

pg 483, ln 44-45: "Multiple threads completing the same request. A program in which two threads block, waiting on the same request, is erroneous."

Rajeev

On Jan 16, 2015, at 12:03 AM, Jeff Hammond <[email protected]> wrote:

MPI-3 page 483 says: "two concurrently running threads may make MPI calls and the outcome will be as if the calls executed in some order, even if their execution is interleaved."

The outcome of my program where the calls execute in some order is well-defined: the first call completes the request and the second call returns immediately since req=MPI_REQUEST_NULL.

Furthermore, MPI-3 page 486 says that MPI_THREAD_MULTIPLE is defined to mean "Multiple threads may call MPI, with no restrictions." Without restriction implies that data races are valid. The prior statement ensures that the result of a program with MPI-related races is well-defined.

Do you disagree with my interpretation of MPI-3?

Jeff

On Thu, Jan 15, 2015 at 9:41 PM, Thakur, Rajeev <[email protected]> wrote:

I would say the program is erroneous. It is calling MPI_Wait concurrently on the same request, i.e., you are completing one operation twice. You can call MPI_Wait concurrently, but with different requests.

Rajeev

On Jan 15, 2015, at 11:32 PM, Jeff Hammond <[email protected]> wrote:

I am trying to run the following program. It's obviously a silly program but I wrote it to make an argument about the MPI standard. I don't believe it violates the MPI standard, but please let me know if I am mistaken.

smakramu-mobl:MPI jrhammon$ cat wait-race.c #include <stdio.h> #include <omp.h> #include <mpi.h>

int main(int argc, char * argv[]) { int provided; MPI_Init_thread(&argc, &argv, MPI_THREAD_MULTIPLE, &provided); if (provided<MPI_THREAD_MULTIPLE) { printf("inadequate thread support\n"); MPI_Finalize(); }

MPI_Request req; MPI_Ibarrier(MPI_COMM_WORLD, &req); //req = MPI_REQUEST_NULL; #pragma omp parallel num_threads(2) shared(req) { MPI_Wait(&req, MPI_STATUS_IGNORE); }

printf("test finished\n"); MPI_Finalize(); return 0; }

I compile the program with MPICH git master and GCC 4.9.2 from about a week ago:

smakramu-mobl:MPI jrhammon$ mpicc -fopenmp -g -Wall wait-race.c

smakramu-mobl:MPI jrhammon$ mpichversion MPICH Version: 3.2a2 MPICH Release date: unreleased development copy MPICH Device: ch3:nemesis MPICH configure: CC=gcc-4.9 CXX=g++-4.9 FC=gfortran-4.9 F77=gfortran-4.9 --disable-cxx --enable-fortran --enable-threads=runtime --enable-g=dbg --with-pm=hydra --prefix=/opt/mpich/dev/gcc/default --enable-wrapper-rpath --enable-static --enable-shared MPICH CC: gcc-4.9 -g -O2 MPICH CXX: no -g MPICH F77: gfortran-4.9 -g -O2 MPICH FC: gfortran-4.9 -g -O2

Sometimes the program runs fine:

smakramu-mobl:MPI jrhammon$ mpiexec -n 2 ./a.out test finished test finished

And sometimes it fails like this:

smakramu-mobl:MPI jrhammon$ mpiexec -n 2 ./a.out test finished Fatal error in MPI_Wait: Internal MPI error!, error stack: MPI_Wait(187).............: MPI_Wait(request=0x7fff5b51ee88, status=0x1) failed MPIR_Wait_impl(84)........: MPIR_Request_complete(232): INTERNAL ERROR: unexpected value in case statement (value=78309520)

So either my program violates the MPI standard or there's a bug in MPICH. Please let me know which it is.

Thanks,

Jeff

-- Jeff Hammond [email protected] http://jeffhammond.github.io/ _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

-- Jeff Hammond [email protected] http://jeffhammond.github.io/ _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

-- Jeff Hammond [email protected] http://jeffhammond.github.io/ _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss