No, something simpler. Just ignore the nest count code entirely; we only use it for error handling and perhaps for choosing whether to get the lock in nested MPI calls, of which these examples have none (or maybe one, if there's a Barrier somewhere). That will tell us how much the get thread specific etc costs. Its probably not much, though it might be a non-trivial part of the overall latency. Bill On Jan 31, 2008, at 1:03 PM, Pavan Balaji wrote:
Bill,
We know that holding the global lock serializes the MPI code. The only other questions are (1) how much does the nest count cost, (2) how much would fine grain locking/atomic updates cost (e.g., for the communicator reference count update in an uncompleted Irecv), and (3) how narrow could a global lock be for the necessary updates instead of fine-grain? We can answer (1) by temporarily changing the definition of the nest count macros and rerunning these tests.
Do you mean acquire the global lock in the nest count increment?
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign