Hi,
I have multithreaded MPI program (MPI_THREAD_MULTIPLE mode) with RMA calls.This is a simplified example:
#include <stdio.h>
#include <mpi.h>
#include <pthread.h>
MPI_Win win;
pthread_mutex_t lock;
void *thread(void *arg)
{
MPI_Win_lock_all(0, win);
MPI_Win_unlock_all(win);
return NULL;
}
int main(int argc, char *argv[])
{
MPI_Init_thread(&argc, &argv, MPI_THREAD_MULTIPLE, NULL);
pthread_mutex_init(&lock, NULL);
int *buf = NULL;
const int bufsize = 1;
MPI_Alloc_mem(bufsize, MPI_INFO_NULL, &buf);
MPI_Win_create(buf, 1, 1, MPI_INFO_NULL, MPI_COMM_WORLD, &win);
pthread_t tid;
pthread_create(&tid, NULL, thread, NULL);
MPI_Win_lock_all(0, win);
MPI_Win_unlock_all(win);
pthread_join(tid, NULL);
MPI_Win_free(&win);
MPI_Finalize();
return 0;
}
If I run such program, it crashes with an error message:
Fatal error in PMPI_Win_lock_all: Wrong synchronization of RMA calls , error stack:
PMPI_Win_lock_all(149).: MPI_Win_lock_all(assert=0, win=0xa0000000) failed
MPID_Win_lock_all(1522): Wrong synchronization of RMA calls
If I replace MPI_Win_lock_all with MPI_Win_lock the problem remains:
Fatal error in PMPI_Win_lock: Wrong synchronization of RMA calls , error stack:
PMPI_Win_lock(157).: MPI_Win_lock(lock_type=234, rank=0, assert=0, win=0xa0000000) failed
MPID_Win_lock(1163): Wrong synchronization of RMA calls
(this message is repeated many times)
If I protect RMA operations with a mutex, the problem disappears.
MPICH version 3.3. Similar problem is also in MVAPICH2 2.3.1.
Are the RMA operations not thread safe at the moment? Could you tell me how to deal with this problem?