I see. Sounds like #3 can be handled by optimization in the leaf function, then? That one should be easy. -- Rob On Aug 2, 2012, at 9:01 AM, William Gropp wrote:
Its #1 (this is the simplest combination). #3 is different. The code for packing from block indexed for single word data should look roughly like
while (count--) *outbuf++ = inbuf[*id++];
with the appropriate variations for size, aligned types, size of outbuf (but all such checks hoisted out of the loop). With const and restrict, a really good compiler can generate good code for this; for other compilers, the usual tricks can be applied.
Bill
William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
On Aug 1, 2012, at 5:17 PM, Rob Ross wrote:
We never implemented optimizations that required combining multiple dataloops into one. This sounds like (1) and (3)? -- Rob
On Aug 1, 2012, at 5:06 PM, William Gropp wrote:
This is just a heads up - inspired by a paper that Torsten will present at EuroMPI, I created some new datatype performance tests (in Torsten's paper, Open MPI does much better than MPICH2). These are drawn from applications and are cases that users expect to perform well. I've also identified some of the reasons:
1) contiguous blocks aren't merged into the parent datatype (nestvec) 2) struct always flattens types, even when this is not appropriate (nestvec2) 3) block index has high overhead for small (e.g., one word) blocks. (indexperf)
These should really be compiled with optimization (not currently done, so you'll need to change the Makefile in perf by hand), where 10x performance problems become apparent.
In general, the data loop code, except for the simple vector case, has high overhead and is likely to be slow for small blocks, which are common in applications. The data loop concept permits optimizations for this case (see the vector case), but it looks like those were never done.
I have a partial fix for the first case, and its pretty easy to turn on or off the struct flattening. Harder is to flatten structs when it makes sense. Fixing #3 requires doing the same sort of code that was used for vectors; that should also be applied to indexed types. Even better is to generate good code based on the platform; I have a student working on that.
Bill
William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign