On Sat, Feb 23, 2013 at 6:49 PM, Jeff Hammond <jhammond@alcf.anl.gov> wrote:

My understanding, having looked at the code for this, is that no
thread binding is set. The OS will hopefully do the right thing when
the cores are undersubscribed, but I see no thread affinity code in
the MPICH source for the comm threads. MVAPICH has some additional
(i.e. non-MPICH) affinity code but I think it is mostly for
process-binding.

I have a modified version of MPICH that allows the user to set the
affinity of the comm threads explicitly. I was interested in pinning
all of the comm threads to one core and letting them fight for time.
For example, on an 8-core node, I was hoping to get async progress on
7 processes by pinning 7 comm threads to the 8th core.

Did this work at all?

My patches for setting comm thread affinity are pretty simple. I
assume you want me to share them?

Sure, but I'm more interested in whether there will be an API for determining this information. That is, suppose I have a node running 4 MPI ranks consisting of 15 user threads each . Those 15 threads are split between two dies and will be running in two groups: one group of 8 doing entirely local work and one group of 7 that communicates frequently. If I could detect the affinity of the comm thread, I could choose how to set affinity for the application threads to get the layout I wanted.