However, the standard does contain the following (MPI-3.0 page 175)

Advice to implementors.
It is strongly recommended that MPI_REDUCE be im-
plemented so that the same result be obtained whenever the function is applied on
the same arguments, appearing in the same order. Note that this may prevent op-
timizations that take advantage of the physical location of ranks. (End of advice to
implementors.)

It would be useful to have a table of which implementations adhere to this advice.

On Mon, Nov 12, 2012 at 9:35 PM, Pavan Balaji <balaji@mcs.anl.gov> wrote:
The MPI standard does not guarantee this.  For example, an
implementation could use topology-aware reductions or an MPI_ANY_SOURCE
while waiting for messages from peers making the order in which the
reduction operations are applied different for each run.