mpi_shared memory
Hello all my name is Arash Hajisharifi and I am a phd student in computational fluid dynamics, Iam working on a code which has been parallelizaed with MPI. I need to improve the parallelisation to reach better performance. I have a 3D domain which has decomposed as 2D domain decomposition and each rank has a stencil of the domain. In my case I have a large 3D array for example Array_total(nx,ny,nz), after decomposition each rank has a part this array:#0 : array_partial(nx,ny/4,nz/4) in rank 0#1 :array_partial(nx,ny/4,nz/4) in rank 1 ... #15 :array_partial(nx,ny/4,nz/4) in rank 15 After some operation, each rank should see Array_partial of every rank, I could not use allgather for each rank because Array_total is so large (about 10 GIG) to gather it inside the local buffer of each rank and can not fit inside and the only way is mpi_shared memory by which each rank shared its part . I know the idea of mpi_shared memory to creat a window among the processes, this idea could be feasible for the processes in one node which the ranks have a shared memory, but it is also possible to creat a window for the processes between 2 nodes? I mean creating one window among all the processes in node 1 and node 2? Unfortunately I am searching for an easy example or a good documentation specially in Fortran for 2 weeks but there is nothing,i don't know how is it possible to not find any thing for one mpi functinlaity which has released 3 years ago and it is not new anymore. May I ask you kindly to guid me or send me some simple example to understand the idea? looking forward to hearing from you, Best Regards,Arash Hajisharifi
participants (1)
-
Arash sharify