July 2016 - devel - mailman3.cels.anl.gov

Re: [mpich-devel] broken link to user guide
by Balaji, Pavan 30 Jul '16

30 Jul '16

Sorry, something seems to have gone wrong after our power outage two days ago. We'll get it fixed. -- Pavan > On Jul 30, 2016, at 9:28 AM, Dan Ibanez <dan.a.ibanez(a)gmail.com> wrote: > > Hello, > > On this page: > > https://www.mpich.org/documentation/guides/ > > The link called "MPICH User's Guide" points to > this location, which returns 404: > > https://www.mpich.org/static/downloads/3.2/mpich-3.2-userguide.pdf > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel

1 0

broken link to user guide
by Dan Ibanez 30 Jul '16

30 Jul '16

Hello, On this page: https://www.mpich.org/documentation/guides/ The link called "MPICH User's Guide" points to this location, which returns 404: https://www.mpich.org/static/downloads/3.2/mpich-3.2-userguide.pdf

1 0

Re: [mpich-devel] Fix for documentation ready for review
by William Gropp 29 Jul '16

29 Jul '16

Thanks, Rob. I’ve pushed it as described in the workflow as ticket-2340. Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Jul 28, 2016, at 10:41 AM, Rob Latham <robl(a)mcs.anl.gov> wrote: > > > On 07/28/2016 09:43 AM, Kenneth Raffenetti wrote: >> I'm backed up reviewing changes for the upcoming 3.3a1 release. If >> nobody else jumps to review this, I can do it in a couple weeks. > > I can review this, sure. Bill: your choice. You can push to a branch, attach to trac ticket, or send me the patches. > > ==rob > >> >> Ken >> >> On 07/27/2016 02:36 PM, William Gropp wrote: >>> I have a fix for #2340, missing documentation. This restores >>> incorrectly deleted files and updates them to include MPI 3.1 >>> constants. Who can review this? >>> >>> Bill >>> >>> William Gropp >>> Director, Parallel Computing Institute >>> Thomas M. Siebel Chair in Computer Science >>> Chief Scientist, NCSA >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> >>> >>> _______________________________________________ >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/devel >>> >> _______________________________________________ >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/devel

1 0

Re: [mpich-devel] Fix for documentation ready for review
by Rob Latham 28 Jul '16

28 Jul '16

On 07/28/2016 09:43 AM, Kenneth Raffenetti wrote: > I'm backed up reviewing changes for the upcoming 3.3a1 release. If > nobody else jumps to review this, I can do it in a couple weeks. I can review this, sure. Bill: your choice. You can push to a branch, attach to trac ticket, or send me the patches. ==rob > > Ken > > On 07/27/2016 02:36 PM, William Gropp wrote: >> I have a fix for #2340, missing documentation. This restores >> incorrectly deleted files and updates them to include MPI 3.1 >> constants. Who can review this? >> >> Bill >> >> William Gropp >> Director, Parallel Computing Institute >> Thomas M. Siebel Chair in Computer Science >> Chief Scientist, NCSA >> University of Illinois Urbana-Champaign >> >> >> >> >> >> _______________________________________________ >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/devel >> > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel

1 0

Re: [mpich-devel] Fix for documentation ready for review
by Kenneth Raffenetti 28 Jul '16

28 Jul '16

I'm backed up reviewing changes for the upcoming 3.3a1 release. If nobody else jumps to review this, I can do it in a couple weeks. Ken On 07/27/2016 02:36 PM, William Gropp wrote: > I have a fix for #2340, missing documentation. This restores incorrectly deleted files and updates them to include MPI 3.1 constants. Who can review this? > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel >

1 0

Re: [mpich-devel] mpi process wireup and apache yarn
by Kenneth Raffenetti 28 Jul '16

28 Jul '16

On 07/27/2016 12:31 AM, Ryan Lewis wrote: > However, assuming that this is all I need, it seems that essentially > each YARN container needs to execute these command lines: > > /usr/lib64/mpich/bin/hydra_pmi_proxy --control-port skynet03:58584 > --rmk user --launcher manual --demux poll --pgid 0 --retries 10 --usize > -2 --proxy-id 0 > > Which they can get from starting the mpi control process on the machine > which runs the YARN Application Master. > > And then they will all just work. Is this accurate? Is this a > "supported" mode of operation? this certainly is an extremely easy way > to get MPI to run on top of YARN, with zero code change necessary to > the MPICH codebase. I'm not sure how portable (across MPI > implementations) this is though, but, for now I don't care. This is correct. In a typical launch mode, the PMI server (mpiexec) takes care of starting the proxies for you, but the proxy startup commands are the same as the manual ones you discovered. Ken

1 0

Fix for documentation ready for review
by William Gropp 27 Jul '16

27 Jul '16

I have a fix for #2340, missing documentation. This restores incorrectly deleted files and updates them to include MPI 3.1 constants. Who can review this? Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign

1 0

mpi process wireup and apache yarn
by Ryan Lewis 27 Jul '16

27 Jul '16

Hi, I am doing a bit of experimentation with the goal of getting MPI to run on top of Apache YARN. I know that a few others have written here looking for help with mpich2-yarn, and the strangely unreleased hamster project on the hadoop JIRA. I'm not interested in those things. I am writing this note to document my progress so far, and get some confirmation that what I am doing is considered a "supported" mode of operation. For context, within YARN, a java based process called a "YARN Application Master: submits requests for resources to the YARN ResourceManager and launches "YARN Containers" via its own process launcher. There can be many AppMasters and each of them may do different things. As a proof of concept, I want to make a given Application Master request N containers, and within each of them start the individual mpi processes. I'm used to well, honestly, never dealing with any of this, and showing up at some cluster where SLURM (or whatever) already exists, and all I need to do is just write code compile it and 'qsub.' So all of this is a learning experience. Looking at Hydra, it seems that the intention is for Hydra to start processes, and oddly (and surprising to me) it is designed to need to _ask_ a RM for resources. With different logic for different RM. There is not a huge amount of documentation here, and so I was largely flying blind. I was expecting that a RM just starts processes on machines, and wire up just happens via some set of environment variable commands and shell commands, and perhaps black magic. After some googling, and private discussion with Jeff Hammond, he pointed me and the -launcher manual flag for mpirun. By issuing: 1. rlewis@skynet03 build]$ mpirun -np 2 -launcher manual -hosts skynet01,skynet02 a.out I was able to get these two hydra_pmi_proxy command lines, which after running both on the two machines, roughly seem to make my mpi program execute normally. 1. HYDRA_LAUNCH: /usr/lib64/mpich/bin/hydra_pmi_proxy --control-port skynet03:58584 --rmk user --launcher manual --demux poll --pgid 0 --retries 10 --usize -2 --proxy-id 0 2. HYDRA_LAUNCH: /usr/lib64/mpich/bin/hydra_pmi_proxy --control-port skynet03:58584 --rmk user --launcher manual --demux poll --pgid 0 --retries 10 --usize -2 --proxy-id 1 3. HYDRA_LAUNCH_END However, using MPI_Send, I would then see this occur: 1. Fatal error in MPI_Send: A process has failed, error stack: 2. MPI_Send(171)..............: MPI_Send(buf=0x7fff2e38b044, count=1, MPI_INT, dest=1, tag=0, MPI_COMM_WORLD) failed 3. MPID_nem_tcp_connpoll(1833): Communication error with rank 1: Connection refused 4. 5. ===================================================================== It seems that when I add the option: `-disable-hostname-propagation` the underlying code seems to work. I'm not exactly sure if this is an accident. However, assuming that this is all I need, it seems that essentially each YARN container needs to execute these command lines: /usr/lib64/mpich/bin/hydra_pmi_proxy --control-port skynet03:58584 --rmk user --launcher manual --demux poll --pgid 0 --retries 10 --usize -2 --proxy-id 0 Which they can get from starting the mpi control process on the machine which runs the YARN Application Master. And then they will all just work. Is this accurate? Is this a "supported" mode of operation? this certainly is an extremely easy way to get MPI to run on top of YARN, with zero code change necessary to the MPICH codebase. I'm not sure how portable (across MPI implementations) this is though, but, for now I don't care. Best, -rhl

1 0