No, something simpler.  Just ignore the nest count code entirely; we only use it for error handling and perhaps for choosing whether to get the lock in nested MPI calls, of which these examples have none (or maybe one, if there's a Barrier somewhere).  That will tell us how much the get thread specific etc costs.  Its probably not much, though it might be a non-trivial part of the overall latency.

Bill

On Jan 31, 2008, at 1:03 PM, Pavan Balaji wrote:

Bill,

We know that holding the global lock serializes the MPI code.  The only other questions are (1) how much does the nest count cost, (2) how much would fine grain locking/atomic updates cost (e.g., for the communicator reference count update in an uncompleted Irecv), and (3) how narrow could a global lock be for the necessary updates instead of fine-grain?
We can answer (1) by temporarily changing the definition of the nest count macros and rerunning these tests.  

Do you mean acquire the global lock in the nest count increment?

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji


William Gropp
Paul and Cynthia Saylor Professor of Computer Science
University of Illinois Urbana-Champaign