October 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] running parallel job issue!
by Reuti 23 Oct '13

23 Oct '13

Am 23.10.2013 um 16:51 schrieb Ricardo Román Brenes: > Hi. I'm not sure how it's done on that version of mpich, but you need to be sure that the mpd daemon is running on both hosts (mpdboot or maybe mpiexec starts it ). No. The `mpdboot` is gone for some time - nowadays Hydra is used (since MPICH2 v1.3). -- Reuti > Does cpi runs in hosts 2 locally? > > On Oct 23, 2013 8:47 AM, "Alexandra Betouni" <alexandra_99(a)windowslive.com> wrote: > > > > > Hey there, I am trying to set up a parallel invironment with 14 machines, running Linux XUbuntu, connected via ethernet. > They all have same IP's and same hostnames. Well I started installing mpich-3.0.4 on a single machine, I run the cpi example on localhost by giving mpiexec -host localhost -n 4 ./examples/cpi and everything worked fine! > So I continued changing the hostnames of 2 pc's for a start, and setting up the ssh in these two, also I installed the mpich-3.0.4 on the other machine too. > By giving the ssh <othermachine> date commant , I get the date of the other host without giving a password, so I think I passed that step too. > Next step was to check if the mpich-3.0.4 runs parallel, so I created a machine file (I made a text file giving the hostnames of the two computers , host1 and host2), and save it in my mpich-3.0.4 build directory. Though when I am trying to parallel run the cpi code by giving mpiexec -n 4 -f machinefile ./examples/cpi on my working directory, I get NO errors but neither parallel job... > All processes still running on host1 which is my work station. > What am I doing wrong? > Thanks > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] running parallel job issue!
by Ken Raffenetti 23 Oct '13

23 Oct '13

mpich 3.0.4 uses hydra as the default process manager, not mpd. ----- Original Message ----- > From: "Ricardo Román Brenes" <roman.ricardo(a)gmail.com> > To: discuss(a)mpich.org > Sent: Wednesday, October 23, 2013 9:51:51 AM > Subject: Re: [mpich-discuss] running parallel job issue! > > > > > Hi. I'm not sure how it's done on that version of mpich, but you need > to be sure that the mpd daemon is running on both hosts (mpdboot or > maybe mpiexec starts it ). > > Does cpi runs in hosts 2 locally? > On Oct 23, 2013 8:47 AM, "Alexandra Betouni" < > alexandra_99(a)windowslive.com > wrote: > > > > > > > > > > > > Hey there, I am trying to set up a parallel invironment with 14 > machines, running Linux XUbuntu, connected via ethernet. > They all have same IP's and same hostnames. Well I started installing > mpich-3.0.4 on a single machine, I run the cpi example on localhost > by giving mpiexec -host localhost -n 4 ./examples/cpi and everything > worked fine! > So I continued changing the hostnames of 2 pc's for a start, and > setting up the ssh in these two, also I installed the mpich-3.0.4 on > the other machine too. > By giving the ssh <othermachine> date commant , I get the date of the > other host without giving a password, so I think I passed that step > too. > Next step was to check if the mpich-3.0.4 runs parallel, so I created > a machine file (I made a text file giving the hostnames of the two > computers , host1 and host2), and save it in my mpich-3.0.4 build > directory. Though when I am trying to parallel run the cpi code by > giving mpiexec -n 4 -f machinefile ./examples/cpi on my working > directory, I get NO errors but neither parallel job... > All processes still running on host1 which is my work station. > What am I doing wrong? > Thanks > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] running parallel job issue!
by Reuti 23 Oct '13

23 Oct '13

Am 23.10.2013 um 16:46 schrieb Alexandra Betouni: > Hey there, I am trying to set up a parallel invironment with 14 machines, running Linux XUbuntu, connected via ethernet. > They all have same IP's and same hostnames. I assume you mean "same entries for TCP/IP address and hostnames" in /etc/hosts - if all have the same TCP/IP address it won't work. > Well I started installing mpich-3.0.4 on a single machine, I run the cpi example on localhost by giving mpiexec -host localhost -n 4 ./examples/cpi and everything worked fine! > So I continued changing the hostnames of 2 pc's for a start, and setting up the ssh in these two, also I installed the mpich-3.0.4 on the other machine too. > By giving the ssh <othermachine> date commant , I get the date of the other host without giving a password, Is it really the other host, or just the local machine accessed by a local `ssh` command? > so I think I passed that step too. > Next step was to check if the mpich-3.0.4 runs parallel, so I created a machine file (I made a text file giving the hostnames of the two computers , host1 and host2), and save it in my mpich-3.0.4 build directory. Though when I am trying to parallel run the cpi code by giving mpiexec -n 4 -f machinefile ./examples/cpi on my working directory, I get NO errors but neither parallel job... Do you share the /home directory or transferred the MPI installation to all machines (like the ~/.ssh/authorized_keys" file)? > All processes still running on host1 which is my work station. > What am I doing wrong? Whether it's some kind of loopback you can check with: $ ps -e f (f w/o -) to check whether all processes of the MPI application are kids of the local shell or via a local `ssh`. -- Reuti > Thanks > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] running parallel job issue!
by Ricardo Román Brenes 23 Oct '13

23 Oct '13

Hi. I'm not sure how it's done on that version of mpich, but you need to be sure that the mpd daemon is running on both hosts (mpdboot or maybe mpiexec starts it ). Does cpi runs in hosts 2 locally? On Oct 23, 2013 8:47 AM, "Alexandra Betouni" <alexandra_99(a)windowslive.com> wrote: > > > ------------------------------ > > > Hey there, I am trying to set up a parallel invironment with 14 machines, > running Linux XUbuntu, connected via ethernet. > They all have same IP's and same hostnames. Well I started installing > mpich-3.0.4 on a single machine, I run the cpi example on localhost by > giving mpiexec -host localhost -n 4 ./examples/cpi and everything worked > fine! > So I continued changing the hostnames of 2 pc's for a start, and setting > up the ssh in these two, also I installed the mpich-3.0.4 on the other > machine too. > By giving the ssh <othermachine> date commant , I get the date of the > other host without giving a password, so I think I passed that step too. > Next step was to check if the mpich-3.0.4 runs parallel, so I created a > machine file (I made a text file giving the hostnames of the two computers > , host1 and host2), and save it in my mpich-3.0.4 build directory. Though > when I am trying to parallel run the cpi code by giving mpiexec -n 4 -f > machinefile ./examples/cpi on my working directory, I get NO errors but > neither parallel job... > All processes still running on host1 which is my work station. > What am I doing wrong? > Thanks > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

running parallel job issue!
by Alexandra Betouni 23 Oct '13

23 Oct '13

Hey there, I am trying to set up a parallel invironment with 14 machines, running Linux XUbuntu, connected via ethernet. They all have same IP's and same hostnames. Well I started installing mpich-3.0.4 on a single machine, I run the cpi example on localhost by giving mpiexec -host localhost -n 4 ./examples/cpi and everything worked fine! So I continued changing the hostnames of 2 pc's for a start, and setting up the ssh in these two, also I installed the mpich-3.0.4 on the other machine too. By giving the ssh <othermachine> date commant , I get the date of the other host without giving a password, so I think I passed that step too. Next step was to check if the mpich-3.0.4 runs parallel, so I created a machine file (I made a text file giving the hostnames of the two computers , host1 and host2), and save it in my mpich-3.0.4 build directory. Though when I am trying to parallel run the cpi code by giving mpiexec -n 4 -f machinefile ./examples/cpi on my working directory, I get NO errors but neither parallel job... All processes still running on host1 which is my work station. What am I doing wrong? Thanks

1 0

Re: [mpich-discuss] mpich2 installation fails
by Ken Raffenetti 22 Oct '13

22 Oct '13

This output is incomplete. Were you sure to pipe all output from configure into a separate file? e.g. ./configure | tee c.txt The config.log file that is generated in your MPICH directory during configure would also be helpful. ----- Original Message ----- > From: "bhimashankar dhappadhule" <dhappabn(a)gmail.com> > To: discuss(a)mpich.org > Sent: Tuesday, October 22, 2013 11:55:24 AM > Subject: Re: [mpich-discuss] mpich2 installation fails > > > > sir,i am sending my configuration file.plz help us urgently. > > > > On Tue, Oct 22, 2013 at 7:43 PM, Ken Raffenetti < > raffenet(a)mcs.anl.gov > wrote: > > > Hi, > > Can you send us the output of your configure command, as well as the > config.log file that was generated? Additionally, if you do not need > the Fortran bindings for MPICH, you can try adding --disable-f77 and > --disable-fc to your configure and see if your build completes then. > > Ken > > > > ----- Original Message ----- > > From: "bhimashankar dhappadhule" < dhappabn(a)gmail.com > > > To: discuss(a)mpich.org > > Sent: Tuesday, October 22, 2013 6:59:06 AM > > Subject: [mpich-discuss] mpich2 installation fails > > > > > > > > > > > > sir , > > while installing mpich2 on ubuntu 13.04 ,configuration successfully > > done but at time of command '' make ",i get error like shown in > > attachment.I urgently required your help sir. > > Thank you...! waiting for reply. > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Affinity mapping with MPICH progress thread
by Jeff Hammond 22 Oct '13

22 Oct '13

See https://github.com/jeffhammond/mpich/commit/1f4f6b1ecef8c6ea727ea671cec9a92…. Please note that this is a hack not a proper implementation. It should use whatever internal affinity API MPICH uses. It won't work on Mac because I don't think Mac OS supports affinity. I'm very curious to know your results. I never got around to experimenting with this implementation as I had hoped. Best, Jeff On Tue, Oct 22, 2013 at 11:58 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > I already implemented this. Let me do a merge with the latest trunk > and then put it as a branch on https://github.com/jeffhammond/mpich. > You may need to assign an comm thread in order to observe passive > progress on this operation :-) > > I seriously doubt the OS will migrate all the busy-polling progress > threads to the same core. Pinning them manually is going to be > required. > > Jeff > > On Tue, Oct 22, 2013 at 8:01 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >> Tareq, >> >> On Oct 22, 2013, at 7:02 AM, Tareq Malas wrote: >>> I am looking for a relatively simple way to control the affinity mapping of the MPICH progress thread and the other threads of my application. In particular, I am using a cluster that has dual 8-core Intel Sandy Bridge sockets per node. I would like to assign the progress thread to a singe core and use the other cores to run my application. Is there a way to pin or detect which core is assigned to the progress thread? I am using MVAPICH2 version 1.9a2 built with icc version 13.1.0 (gcc version 4.4.7 compatibility). I am already using MPICH_ASYNC_PROGRESS variable, but I am not sure how can I combine this with thread affinity mapping library. >> >> Hydra provides a way to bind all threads of the process to a set of cores (e.g., you can assign all cores of the socket to all threads of your process), but no way to specifically bind each thread to a particular core. You'll need to do this assignment from inside your application. >> >> I understand that this is inconvenient for the MPICH progress thread, since that's within MPI, and that should be fixed within MPICH. We'll do that. But in reality, it's best to leave that thread not bound to a particular core, IMO. >> >> -- Pavan >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > Jeff Hammond > jeff.science(a)gmail.com -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] Affinity mapping with MPICH progress thread
by Jeff Hammond 22 Oct '13

22 Oct '13

I already implemented this. Let me do a merge with the latest trunk and then put it as a branch on https://github.com/jeffhammond/mpich. You may need to assign an comm thread in order to observe passive progress on this operation :-) I seriously doubt the OS will migrate all the busy-polling progress threads to the same core. Pinning them manually is going to be required. Jeff On Tue, Oct 22, 2013 at 8:01 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > Tareq, > > On Oct 22, 2013, at 7:02 AM, Tareq Malas wrote: >> I am looking for a relatively simple way to control the affinity mapping of the MPICH progress thread and the other threads of my application. In particular, I am using a cluster that has dual 8-core Intel Sandy Bridge sockets per node. I would like to assign the progress thread to a singe core and use the other cores to run my application. Is there a way to pin or detect which core is assigned to the progress thread? I am using MVAPICH2 version 1.9a2 built with icc version 13.1.0 (gcc version 4.4.7 compatibility). I am already using MPICH_ASYNC_PROGRESS variable, but I am not sure how can I combine this with thread affinity mapping library. > > Hydra provides a way to bind all threads of the process to a set of cores (e.g., you can assign all cores of the socket to all threads of your process), but no way to specifically bind each thread to a particular core. You'll need to do this assignment from inside your application. > > I understand that this is inconvenient for the MPICH progress thread, since that's within MPI, and that should be fixed within MPICH. We'll do that. But in reality, it's best to leave that thread not bound to a particular core, IMO. > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] mpich2 installation fails
by bhimashankar dhappadhule 22 Oct '13

22 Oct '13

sir,i am sending my configuration file.plz help us urgently. On Tue, Oct 22, 2013 at 7:43 PM, Ken Raffenetti <raffenet(a)mcs.anl.gov>wrote: > Hi, > > Can you send us the output of your configure command, as well as the > config.log file that was generated? Additionally, if you do not need the > Fortran bindings for MPICH, you can try adding --disable-f77 and > --disable-fc to your configure and see if your build completes then. > > Ken > > ----- Original Message ----- > > From: "bhimashankar dhappadhule" <dhappabn(a)gmail.com> > > To: discuss(a)mpich.org > > Sent: Tuesday, October 22, 2013 6:59:06 AM > > Subject: [mpich-discuss] mpich2 installation fails > > > > > > > > > > > > sir , > > while installing mpich2 on ubuntu 13.04 ,configuration successfully > > done but at time of command '' make ",i get error like shown in > > attachment.I urgently required your help sir. > > Thank you...! waiting for reply. > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] MPICH2 on LAN with Ubuntu 12.04
by Jeff Hammond 22 Oct '13

22 Oct '13

See http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager for how to launch MPI across multiple machines. You do not need to use any special hardware adapter; any Ethernet card should provide sufficient TCP/IP support. Jeff On Tue, Oct 22, 2013 at 9:23 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > Hello, > > You'll need to either install mpich on a shared file system (such as NFS) or individually on each of the nodes. The README file in the mpich tarball should have most of this information. > > -- Pavan > > On Oct 22, 2013, at 8:26 AM, heshsham basit wrote: > >> Hi, >> >> I did some experimentation with MPICH on windows, and recently came to know that there is no support for Windows now. >> I am now shifting to Ubuntu. I was successful in installing MPICH2 and running my first MPI program on a single machine. >> >> Now I want to run my code on multiple PCs. How do I install MPI in a LAN ? What are the additional requirement if any? Do I need any specific hardware -network adapter My card details are as under: >> >> root@ubuntu:/home# lspci -nnk | grep -i net -A2 >> 01:07.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ [10ec:8139] (rev 10) >> Subsystem: Hewlett-Packard Company Device [103c:301d] >> Kernel driver in use: 8139too >> >> >> Most of my machines are having this type of card and are connected in LAN. >> Really appreciate for any pointers to appropriate document or suggestions. >> >> >> Thanks and regards, >> >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com

1 0