Hello, Is there a reason for you to disable the default mpiexec? You could just do: ./configure --prefix=/home/biddisco/apps/mpich2-1.5 --enable-shared --disable-static This will build the default mpiexec in mpich (it'll internally automatically detect slurm and use srun, but with better PMI communication with the processes). Once you have mpiexec, MPMD and other features will work correctly. FWIW, I'd recommend upgrading to mpich-3.0.2. -- Pavan On 02/04/2013 09:51 AM US Central Time, Biddiscombe, John A. wrote:

Hello,

I�m working on someone else�s cluster and the mpi is not well setup (i.e. not working).

I compiled my own mpich2 using

./configure --with-pmi=slurm --with-pm=no --prefix=/home/biddisco/apps/mpich2-1.5 --enable-shared --disable-static

And a simple hello world executed using srun, works as expected and I get the ranks from all nodes responding correctly.

When I used mpiexec previously, I ran an MPMD command like this

mpiexec -n 4 -env DISPLAY :0.0 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111 : -n 4 -env DISPLAY :0.1 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111 : -n 4 -env DISPLAY :0.2 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111

I need each of the 3 sets of mpi ranks to have a different DISPLAY environment variable to correctly utilise the GPUs.

Unfortunately, the mpiexec on the machine does not work with slurm and I�d like to use

srun �n XXX �env XXX /program : �n XXX �env YYY /program ...

etc etc

but srun doesn�t have an option to pass command line environment vars to the mpi processes. I just can�t find a way of launching an MPMD job using srun with different env vars.

Is there another way?

Thanks

JB

--

John Biddiscombe, email:biddisco @.at.@ cscs.ch

http://www.cscs.ch/

CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07

Via Trevano 131, 6900 Lugano, Switzerland | Fax: +41 (91) 610.82.82

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

-- Pavan Balaji http://www.mcs.anl.gov/~balaji