Jeff, thanks a lot for your detailed answer. It makes much more sense now. Best regards, Khalid On Tue, Mar 3, 2015 at 4:11 AM, Jeff Hammond <[email protected]> wrote:

Those operations were never specially optimized on Blue Gene. The operations that were optimized heavily on Blue Gene were/are Barrier, Bcast, Allreduce, and Alltoall(v). Most of the other collective optimizations were derivatives of that. For example, one can implement Allgather as a series of Bcasts. And Reduce was optimized as a side effect of Allreduce; in some cases, Allreduce was faster than Reduce, which is counter-intuitive.

One optimization that Blue Gene had for Gather was to use Reduce with BOR (on BG/P) and SUM (on BG/Q, for float types at least). This turned Gather of count=1 into Reduce of count=nproc (size of communicator), but it was very faster for short messages.

There aren't many good optimizations for Scatter. MPICH has most of the generic ones, as you might expect. It is possible that Scatter as Alltoallv with only one non-zero in the count vector was faster than MPICH, but this would surprise me.

In any case, all MPI collectives on BG benefit from very good network-to-processor balance, minimal rendezvous (connectionless HW and SW), trivial virtual-to-physical translation, good bisection bandwidth of _electrically isolated_ torus networks, etc.

If you have a more specific question, I might have a better answer.

Best,

Jeff

On Sun, Mar 1, 2015 at 4:36 PM, Khalid Hasanov <[email protected]> wrote:

...

Hello,

First of all, I am not sure if this group is the right place for this question. If not I apologize for asking unrelated question.

I read two papers about optimizing MPI collective communications on BG/L and BG/P. (Optimization of MPI Collective Communication on BlueGene/L Systems and MPI Collective Communications on The Blue Gene/P Supercomputer: Algorithms and Optimizations respectively). However, these two papers do not mention anything about MPI scatter and gather operations, I wonder if these two collective operations have been optimized for BlueGene or they use exactly the same algorithms from MPICH. Any reference appreciated.

Best regards, Khalid

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

-- Jeff Hammond [email protected] http://jeffhammond.github.io/ _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss