On Tue, May 5, 2020, at 9:57 AM, Benson Muite wrote:

On Tue, May 5, 2020, at 8:38 AM, hritikesh semwal via discuss wrote:

...

Hello all,

I am working on the development of a parallel CFD solver and I am using MPI_Allreduce for the global summation of the local errors calculated on all processes of a group and the summation is to be used by all the processes. My concern is that MPI_Allreduce is taking almost 27-30% of the total time used, which is a significant amount. So, I want to ask if anyone can suggest me better alternative/s to replace MPI_Allreduce which can reduce the time consumption.

Thank you. _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

Hi Hitesh,

What hardware are you running on and what is the interconnect? Have you tried changing any of the MPI settings? Can the reduction be done asynchronously?

Regards, Benson

Also, is your work load balanced? One way to check this might be to place a barrier just before the all-reduce call. If the barrier ends up taking most of your time, then it is likely you will need to determine a better way to distribute the computational work.