Actually, you can do something like the halo test I've added to mpptest to check that a single process can use multiple links concurrently. In that case, you really need a system with 2t cores, where t processes can run 1 thread and 1 process runs t threads; each of those t threads communicates with a uniquely chosen process. That will test sending on t different VCs; you only time the process with t threads. Along the lines of the experiments that Rajeev and I did, users will expect the overhead to be small, so in the current test, the messaging rate should stay relatively constant (actually increasing slightly as the non-communication part can be run concurrently, assuming we avoid the one big lock - that, by the way, is anther test - just use the per VC lock, and disable to global lock. This is also wrong, but does help set a limit on the potential performance with the current code. A harder question is can the lock on the VC also be narrowed so that threads communicating on the same VC spend as little time as possible coordinating access to the VC. Eg, if there was a separate thread performing at actions on the VC, then the only time involved for the compute/user threads would be enqueueing or dequeueing an operation. Bill On Jan 30, 2008, at 1:07 PM, Darius Buntinas wrote:
I just want to check what the benchmark should do.
My understanding from the conference calls and Sameer's slides is that they're interested in the case where multiple threads of a process are sending/receiving to/from different processes. This allows the process to use multiple DMAs concurrently.
Right now, we've been testing the case where one process is sending/ receiving with itself. So moving the locks around VCs may decrease the size of the CS, but will still serialize.
So what's the test we want to run?
I imagine something like P processes each with P-1 threads where each thread is communicating with a different process. But then we need a machine with P(P-1) cores.
On the 8-core nodes we have, we can run 3 processes with 2 threads each. But I think we need more threads to really stress things.
It seems that what we can see from the one or two processes is the just the effect of the size of the CS.
Comments?
-d
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign