On Sat, Feb 23, 2013 at 6:49 PM, Jeff Hammond <[email protected]> wrote:
My understanding, having looked at the code for this, is that no thread binding is set. The OS will hopefully do the right thing when the cores are undersubscribed, but I see no thread affinity code in the MPICH source for the comm threads. MVAPICH has some additional (i.e. non-MPICH) affinity code but I think it is mostly for process-binding.
I have a modified version of MPICH that allows the user to set the affinity of the comm threads explicitly. I was interested in pinning all of the comm threads to one core and letting them fight for time. For example, on an 8-core node, I was hoping to get async progress on 7 processes by pinning 7 comm threads to the 8th core.
Did this work at all?
My patches for setting comm thread affinity are pretty simple. I assume you want me to share them?
Sure, but I'm more interested in whether there will be an API for determining this information. That is, suppose I have a node running 4 MPI ranks consisting of 15 user threads each . Those 15 threads are split between two dies and will be running in two groups: one group of 8 doing entirely local work and one group of 7 that communicates frequently. If I could detect the affinity of the comm thread, I could choose how to set affinity for the application threads to get the layout I wanted.