Since Hydra already has a "--binding" option (used in the same sense as "map") would it be better if we have two options, "--node-mapping" & "--core-mapping", to replace the "--binding" option ? Then we can have "--node-mapping=rr" (round robin) & "--node-mapping=pack" (consequtive)... -Jayesh -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of William Gropp Sent: Tuesday, August 18, 2009 9:08 AM To: [email protected] Subject: Re: [mpich2-core] mpiexec extensions How about this for the mapping --mapping <key> --mapfile <filename> With the following values for <key> rr, roundrobin conseq, consequtive On systems with higher-level hierarchies (like BG), this could pick an order, much like BG does now: txyz xyzt etc. The mapfile option allows the use to specify the specific mapping of ranks to resources. The format will depend on the system, but for typical smp clusters could be an ordered list of nodes. OpenMPI apparently already has -bynode and -byslot ; this maps in round robin or successive ranks on each node respectively. We could support these ase -bynode == -mapping rr -byslot == --mapping conseq Thoughts? Bill On Aug 14, 2009, at 3:11 PM, Rajeev Thakur wrote:
It would be good to have something like this. By default, it should place successive ranks on the same node (that's what users expect), and another option to place them on consecutive nodes.
Rajeev
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of William Gropp Sent: Friday, August 14, 2009 1:44 PM To: [email protected] Subject: [mpich2-core] mpiexec extensions
I'd like to discuss two extensions for mpiexec to better support SMP clusters. These aren't things I'd propose for the standard. I'm using the first one, and it has made life much easier for me.
The first option specifies the number of processes per node. I'm using -pn <n>, as in -pn 8. On our POWER5 system, with 16 processors be node, I can run tests with
bpmpiexec.pl -n 128 pn 4 ./halocompare
(bpmpiexec.pl is a perl script that submits a LoadLeveler job using the IBM MPI that I put together).
The other option would control whether successive ranks are on the same node of the "next" node. This would be nice to have, though not all systems might support it. That option would be similar in spirit to the BG mapping option, though perhaps only for simple layouts.
Comments?
Bill
William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign