Hi, Let's separate collective I/O the concept from collective I/O the implementation in ROMIO. Lots of things we *could* do: - switch to a different internal representation of noncontiguous regions, perhaps dramatically decreasing memory allocation when exchanging regions - switch to a different collective I/O algorithm that uses some hierarchy to reduce number of processes for which space must be allocated - don't globally coordinate, but rather coordinate more loosely to reduce communication and memory allocation requirements - … I'm sure we can come up with more ideas, some of which would be new research and some of which would be mostly engineering. All relatively major activities. Thus my question :). Rob On May 6, 2013, at 2:30 PM, Bob Cernohous wrote:
From: Rob Ross <[email protected]>
Should we consider this as interest in working on this problem on the IBM side :)? -- Rob
Say what?! ;)
Meaning we can get rid of all o(p) allocations? Not sure how you do internal collectives on behalf of the app/mpi-io without at least some of those. I was looking more for agreement that collective i/o is 'what it is'... and maybe some idea if we just have some known limitations on scaling it. Yes, that BG alltoallv is a bigger problem that we can avoid with an env var -- is that just going to have to be 'good enough'? (I think that Jeff P wrote that on BG/P and got good performance with that alltoallv. Trading memory for performance, not unusual, and at least it's selectable.)