I'm currently working on some performance numbers of MPICH 3.1.3 on the Tilera TILE-Gx platform. While direct support isn't indicated, I wanted to see what the performance profile would be for some of the core MPI functions. The barrier times, however, exhibit poor performance and high variance as the number of PEs scale up on this 36-core SMP device.

PEs    Trial 1     Trial 2 (times in microseconds)
2         7.28        7.54
3        11.63    13010.92
4        15.01      268
5     21073.14       18.11
6        19.63       18.97
7        17.83    22698.7
8        23.82     4099.2
9        24.72     9334.79
10    39859.54     1591.71
11    39586.55    38599.86
12    30229.56    39799.5
13    28860.67     5684.71
14    38742.38    39956.55
15    43117.49    39952.83
16    26762.15    26230.78
17    39458.78    59911.38
18    46232.86    45327.22
19    45462.67    54762.76
20    59744.57    39657.49
21    54023.2     72209.07
22    78213.27    49108.57
23    69221.4     79534.28
24    71599.46    81920.28
25    45443.61    76059.46
26    50649.56    72402.17
27    39936.37    66906.83
28    64324.64    44803.16
29    59913.22    53071.9
30    44569.65    70844.15
31    55428.5     47399.51
32    54708.82    64102.45
33    34413.99    52384.35
34    66390.67    97134.79
35    68839.46    62719.52
36    43243.98    59132.83 (59.1 ms)
Each entry is an average of 100 iterations. Benchmark used is from OSU MPI microbenchmarks (osu_barrier.c).

These are the configure options I used to cross-compile and build MPICH for TILE-Gx:
mkdir build && cd build
PREFIXROOT=/opt/tilegx
../configure CC=tile-gcc CXX=tile-g++ FC=tile-gfortran F77=tile-gfortran \
    --prefix=$PREFIXROOT/mpich --host=tile \
    --disable-fortran \
    --enable-fast=yes --with-thread-package=posix

Are there any build options that I'm not using that I should be? I'm trying to focus this build specifically for SMP performance.

Thanks,

Bryant Lam