On 07/27/2016 12:31 AM, Ryan Lewis wrote:
However, assuming that this is all I need, it seems that essentially each YARN container needs to execute these command lines:
/usr/lib64/mpich/bin/hydra_pmi_proxy --control-port skynet03:58584 --rmk user --launcher manual --demux poll --pgid 0 --retries 10 --usize -2 --proxy-id 0
Which they can get from starting the mpi control process on the machine which runs the YARN Application Master.
And then they will all just work. Is this accurate? Is this a "supported" mode of operation? this certainly is an extremely easy way to get MPI to run on top of YARN, with zero code change necessary to the MPICH codebase. I'm not sure how portable (across MPI implementations) this is though, but, for now I don't care.
This is correct. In a typical launch mode, the PMI server (mpiexec) takes care of starting the proxies for you, but the proxy startup commands are the same as the manual ones you discovered. Ken