March 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPID_nem_tcp_init(377)... gethostbyname failed, mac (errno 1)
by Reuti 04 Mar '14

04 Mar '14

Am 04.03.2014 um 12:10 schrieb Luis Cebamanos: > It also works with en0: > > lcebaman@mac~/$ mpiexec -iface en0 -n 4 ./hello > Hello world from process 0 of 4 > Hello world from process 3 of 4 > Hello world from process 1 of 4 > Hello world from process 2 of 4 Interesting, something I never tried before as my name resolution works as expected. -- Reuti > And that is what my ifconfig says: > > lcebaman@mac~/$ ifconfig > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 > options=3<RXCSUM,TXCSUM> > inet6 ::1 prefixlen 128 > inet 127.0.0.1 netmask 0xff000000 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > nd6 options=1<PERFORMNUD> > gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280 > stf0: flags=0<> mtu 1280 > en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > ether 28:cf:e9:1f:cf:13 > inet6 fe80::2acf:e9ff:fe1f:cf13%en0 prefixlen 64 scopeid 0x4 > inet 172.20.170.189 netmask 0xfffff000 broadcast 172.20.175.255 > nd6 options=1<PERFORMNUD> > media: autoselect > status: active > en4: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 > options=60<TSO4,TSO6> > ether 32:00:17:4d:80:20 > media: autoselect <full-duplex> > status: inactive > en5: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 > options=60<TSO4,TSO6> > ether 32:00:17:4d:80:21 > media: autoselect <full-duplex> > status: inactive > p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304 > ether 0a:cf:e9:1f:cf:13 > media: autoselect > status: inactive > bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > options=63<RXCSUM,TXCSUM,TSO4,TSO6> > ether 2a:cf:e9:f1:ef:00 > Configuration: > id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0 > maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200 > root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0 > ipfilter disabled flags 0x2 > member: en4 flags=3<LEARNING,DISCOVER> > ifmaxaddr 0 port 5 priority 0 path cost 0 > member: en5 flags=3<LEARNING,DISCOVER> > ifmaxaddr 0 port 6 priority 0 path cost 0 > nd6 options=1<PERFORMNUD> > media: <unknown type> > status: inactive > > Cheers, > Luis > On 3 Mar 2014, at 22:24, Luis Cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: > >> >> Hi Reuti, >> >> I have done what you suggested and it WORKS with the wireless off >> >> lcebaman@mac~/$ mpiexec -iface lo0 -n 4 ./hello >> Hello world from process 0 of 4 >> Hello world from process 3 of 4 >> Hello world from process 1 of 4 >> Hello world from process 2 of 4 >> >> What is the problem here then? >> >> Cheers, >> Luis >> On 3 Mar 2014, at 14:54, Reuti <reuti(a)staff.uni-marburg.de> wrote: >> >>> Am 03.03.2014 um 11:29 schrieb Luis Cebamanos: >>> >>>> I have also noticed that working without any internet connection does not work either. >>> >>> Just for curiosity: >>> >>> $ mpiexec -iface lo0 -n 4 ./foo >>> >>> -- Reuti >>> >>> >>>> On 3 Mar 2014, at 09:53, Luis Cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: >>>> >>>>> Hi there, >>>>> >>>>> I’ve got more news. The last IP that I showed in my last email it turns out that works OKAY, however now that I am back to the office and working on the wireless, it does not work. And that is the result of nslookup: >>>>> >>>>> lcebaman@mac~/Projects/SPRINT$ nslookup localhost >>>>> Server: 129.215.205.191 >>>>> Address: 129.215.205.191#53 >>>>> >>>>> Name: localhost.ed.ac.uk >>>>> Address: 127.0.0.1 >>>>> >>>>> lcebaman@mac~/Projects/SPRINT$ nslookup mac >>>>> Server: 129.215.205.191 >>>>> Address: 129.215.205.191#53 >>>>> >>>>> ** server can't find mac: NXDOMAIN >>>>> >>>>> Any thoughts? >>>>> >>>>> Cheers, >>>>> Luis >>>>> >>>>> On 1 Mar 2014, at 19:40, l.cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: >>>>> >>>>>> Well, is not fixed. Its just the one I got here, but it changes. Any ideas of how to fix this? >>>>>> >>>>>> -------- Original message -------- >>>>>> From: Reuti <reuti(a)staff.uni-marburg.de> >>>>>> Date:01/03/2014 17:38 (GMT+00:00) >>>>>> To: discuss(a)mpich.org >>>>>> Subject: Re: [mpich-discuss] MPID_nem_tcp_init(377)... gethostbyname failed, >>>>>> mac (errno 1) >>>>>> >>>>>> Am 01.03.2014 um 17:58 schrieb Luis Cebamanos: >>>>>> >>>>>>> Here it is the output of those commands: >>>>>>> >>>>>>> lcebaman@mac~/$ nslookup localhost >>>>>>> Server: 192.168.1.1 >>>>>>> Address: 192.168.1.1#53 >>>>>>> >>>>>>> Name: localhost >>>>>>> Address: 127.0.0.1 >>>>>>> >>>>>>> lcebaman@mac~/$ nslookup mac >>>>>>> Server: 192.168.1.1 >>>>>>> Address: 192.168.1.1#53 >>>>>>> >>>>>>> Name: mac >>>>>>> Address: 192.168.1.127 >>>>>> >>>>>> Is this a fixed address you get everytime? You could add it to /etc/hosts. >>>>>> >>>>>> Interesting that MPICH throws an error about resolving "mac", although DNS is working. >>>>>> >>>>>> -- Reuti >>>>>> >>>>>> >>>>>>> lcebaman@mac~/$ hostname >>>>>>> mac >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 28 Feb 2014, at 23:35, Reuti <reuti(a)staff.uni-marburg.de> wrote: >>>>>>> >>>>>>>> Am 28.02.2014 um 14:39 schrieb Luis Cebamanos: >>>>>>>> >>>>>>>>> Hi Pavan, >>>>>>>>> >>>>>>>>> Yes, that is the output: >>>>>>>>> >>>>>>>>> $ cat /etc/hosts >>>>>>>>> ## >>>>>>>>> # Host Database >>>>>>>>> # >>>>>>>>> # localhost is used to configure the loopback interface >>>>>>>>> # when the system is booting. Do not change this entry. >>>>>>>>> ## >>>>>>>>> 127.0.0.1 localhost >>>>>>>>> 255.255.255.255 broadcasthost >>>>>>>>> ::1 localhost >>>>>>>>> fe80::1%lo0 localhost >>>>>>>> >>>>>>>> What is the output of: >>>>>>>> >>>>>>>> $ nslookup localhost >>>>>>>> $ nslookup mac >>>>>>>> $ hostname >>>>>>>> >>>>>>>> AFAICS your machine has no domainname, otherwise it would show up in the MPICH error message after "mac". >>>>>>>> >>>>>>>> -- Reuti >>>>>>>> >>>>>>>> >>>>>>>>> On 28/02/14 13:35, Balaji, Pavan wrote: >>>>>>>>>> $ cat /etc/hosts >>>>>>>>>>>> ## >>>>>>>>>>>> # Host Database >>>>>>>>>>>> # >>>>>>>>>>>> # localhost is used to configure the loopback interface >>>>>>>>>>>> # when the system is booting. Do not change this entry. >>>>>>>>>>>> ## >>>>>>>>>>>> 127.0.0.1 localhost >>>>>>>>>>>> 255.255.255.255 broadcasthost >>>>>>>>>>>> ::1 localhost >>>>>>>>>>>> fe80::1%lo0 localhost >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> The University of Edinburgh is a charitable body, registered in >>>>>>>>> Scotland, with registration number SC005336. >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>>> To manage subscription options or unsubscribe: >>>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>> To manage subscription options or unsubscribe: >>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>> >>>>>>> The University of Edinburgh is a charitable body, registered in >>>>>>> Scotland, with registration number SC005336. >>>>>>> _______________________________________________ >>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>> To manage subscription options or unsubscribe: >>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> The University of Edinburgh is a charitable body, registered in >>>>> Scotland, with registration number SC005336. >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Problems with running make
by Reuti 04 Mar '14

04 Mar '14

Hi, Am 04.03.2014 um 00:49 schrieb Gus Correa: > On 03/03/2014 06:19 PM, Ron Palmer wrote: >> Reuti, good comment about primary interface for mpi and secondary for >> the rest of the world. >> >> Yes, setting it up from scratch, and I would love a queuing system as >> well, I just had no idea such existed (learning by the minute!). These >> computers are dedicated to this parallelisation jobs I am running and >> will not run anything else. Would you be able to point my nose in a >> direction of a simple queuing system? >> > > In the open source / free software world: > > I use Torque+Maui: > > http://www.adaptivecomputing.com/products/open-source/torque/ > http://www.adaptivecomputing.com/products/open-source/maui/ Personally I don't like the split of resource manager and scheduler. I struggle between `qstat` and `showq`. > Another very popular queuing system is the Sun Grid Engine (maybe this > is what Reuti uses?), Yep. > which I think became the Open Grid Scheduler. After Oracle bought SUN there were first two open source forks besides Oracle's commercial one. The one you mentioned and https://arc.liv.ac.uk/trac/SGE Eventually Univa hired several members of the original development team and bought all assets from Oracle, so that there is one commercial version of GridEngine from Univa. -- Reuti > Better ask somebody else (Reuti maybe) about this one. > > http://gridscheduler.sourceforge.net/ > > Yet a third one is LLNL's Slurm: > > https://computing.llnl.gov/linux/slurm/ > > >> All my parallel processing are controlled by my own perl scripts and >> commandline commands (provided binaries), nothing GUI at all (which is >> kinda nice). >> >> Do you think shutting down the xserver would make any difference to the >> performance of these computers? I am pretty sure that they al have a >> windows manager running by default... I only got them going last week so >> I have not (yet) gone in to this level of detail. >> > > If you are sure nobody will ever want to run Matlab, > or to show a graph with gnuplot, or perhaps more likely, use a graphical debugger such as DDD (or the equivalent from Intel or PGI), you can live without X-windows. > > I don't see it as a big burden, though, unless there is somebody doing > heavy GUI stuff while other computation is going on concurrently. > > You could simply start the computers at runlevel 3. > > ** > > OK, this is now more about cluster then about MPICH specifically. :) > > I hope this helps, > Gus Correa > >> Thanks, >> Ron >> >> On 4/03/2014 09:06, Reuti wrote: >>> Am 03.03.2014 um 23:54 schrieb Gus Correa: >>> >>>> On 03/03/2014 04:36 PM, Ron Palmer wrote: >>>>> Thanks Reuti for your comments. I will peruse that FAQ detail. >>>>> >>>>> I just thought of the fact that each of these rack computers have 4 >>>>> ethernet sockets, eth0 - eth3... I could connect the cluster on a >>>>> separate ethernet sockets via an extra switch not connected to the >>>>> internet or any other computers, and accept all communication among >>>>> them, and keep iptables up on the ethx connected to the outside >>>>> world. I >>>>> guess I would have to set up routing tables or something. Ah, more >>>>> reading :-) >>>>> >>>>> Thanks for your help. >>>>> Ron >>>>> >>>> Hi Ron >>>> >>>> If those extra interfaces are not in use, >>>> and if you have a spare switch, >>>> you can setup a separate private subnet exclusively for MPI. >>>> You need to configure the interfaces consistently (IP, subnet mask, >>>> perhaps a gateway). Configuring them statically is easy: >>>> >>>> https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… >>>> >>>> >>>> Use a subnet that doesn't intersect the existent/original IP range. >>>> >>>> http://en.wikipedia.org/wiki/Private_network >>>> >>>> You could also create host names associated to those IPs (say >>>> node01, node02, node02), resolve them via /etc/hosts on each computer, >>>> set passwordless ssh across these newly named "hosts". >>>> This may simpler/safer than messing with the iptables. >>>> >>>> [Actually, the IP addresses you showed 192.168.X.Y, sound as a private >>>> subnet already, not Internet, but that may be the subnet for your >>>> organization/school/department already. So, you may set up a different >>>> one on these three computers for MPI and very-local access.] >>> Yep, maybe in the 10.0.0.0/8 range. >>> >>> BTW: are you setting up a cluster from scratch - will you also add any >>> queuing system later on? >>> >>> >>>> OpenMPI allows you to choose the interface that it will use, >>>> so you can direct it to your very-local subnet: >>>> >>>> http://www.open-mpi.org/faq/?category=tcp#tcp-selection >>> MPICH: >>> >>> http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Hydra… >>> >>> >>> It should work for other Ethernet interfaces too. >>> >>> Nevertheless: in any case it might be easier for the applications to >>> use the primary interface solely for MPI, and any other one for the >>> external access (this is usually my setup, as I don't have to provide >>> the assignment of other interfaces this way). >>> >>> -- Reuti >>> >>> >>> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_init(377)... gethostbyname failed, mac (errno 1)
by Luis Cebamanos 04 Mar '14

04 Mar '14

It also works with en0: lcebaman@mac~/$ mpiexec -iface en0 -n 4 ./hello Hello world from process 0 of 4 Hello world from process 3 of 4 Hello world from process 1 of 4 Hello world from process 2 of 4 And that is what my ifconfig says: lcebaman@mac~/$ ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 options=3<RXCSUM,TXCSUM> inet6 ::1 prefixlen 128 inet 127.0.0.1 netmask 0xff000000 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 nd6 options=1<PERFORMNUD> gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280 stf0: flags=0<> mtu 1280 en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ether 28:cf:e9:1f:cf:13 inet6 fe80::2acf:e9ff:fe1f:cf13%en0 prefixlen 64 scopeid 0x4 inet 172.20.170.189 netmask 0xfffff000 broadcast 172.20.175.255 nd6 options=1<PERFORMNUD> media: autoselect status: active en4: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 options=60<TSO4,TSO6> ether 32:00:17:4d:80:20 media: autoselect <full-duplex> status: inactive en5: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 options=60<TSO4,TSO6> ether 32:00:17:4d:80:21 media: autoselect <full-duplex> status: inactive p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304 ether 0a:cf:e9:1f:cf:13 media: autoselect status: inactive bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 options=63<RXCSUM,TXCSUM,TSO4,TSO6> ether 2a:cf:e9:f1:ef:00 Configuration: id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0 maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200 root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0 ipfilter disabled flags 0x2 member: en4 flags=3<LEARNING,DISCOVER> ifmaxaddr 0 port 5 priority 0 path cost 0 member: en5 flags=3<LEARNING,DISCOVER> ifmaxaddr 0 port 6 priority 0 path cost 0 nd6 options=1<PERFORMNUD> media: <unknown type> status: inactive Cheers, Luis On 3 Mar 2014, at 22:24, Luis Cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: > > Hi Reuti, > > I have done what you suggested and it WORKS with the wireless off > > lcebaman@mac~/$ mpiexec -iface lo0 -n 4 ./hello > Hello world from process 0 of 4 > Hello world from process 3 of 4 > Hello world from process 1 of 4 > Hello world from process 2 of 4 > > What is the problem here then? > > Cheers, > Luis > On 3 Mar 2014, at 14:54, Reuti <reuti(a)staff.uni-marburg.de> wrote: > >> Am 03.03.2014 um 11:29 schrieb Luis Cebamanos: >> >>> I have also noticed that working without any internet connection does not work either. >> >> Just for curiosity: >> >> $ mpiexec -iface lo0 -n 4 ./foo >> >> -- Reuti >> >> >>> On 3 Mar 2014, at 09:53, Luis Cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: >>> >>>> Hi there, >>>> >>>> I’ve got more news. The last IP that I showed in my last email it turns out that works OKAY, however now that I am back to the office and working on the wireless, it does not work. And that is the result of nslookup: >>>> >>>> lcebaman@mac~/Projects/SPRINT$ nslookup localhost >>>> Server: 129.215.205.191 >>>> Address: 129.215.205.191#53 >>>> >>>> Name: localhost.ed.ac.uk >>>> Address: 127.0.0.1 >>>> >>>> lcebaman@mac~/Projects/SPRINT$ nslookup mac >>>> Server: 129.215.205.191 >>>> Address: 129.215.205.191#53 >>>> >>>> ** server can't find mac: NXDOMAIN >>>> >>>> Any thoughts? >>>> >>>> Cheers, >>>> Luis >>>> >>>> On 1 Mar 2014, at 19:40, l.cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: >>>> >>>>> Well, is not fixed. Its just the one I got here, but it changes. Any ideas of how to fix this? >>>>> >>>>> -------- Original message -------- >>>>> From: Reuti <reuti(a)staff.uni-marburg.de> >>>>> Date:01/03/2014 17:38 (GMT+00:00) >>>>> To: discuss(a)mpich.org >>>>> Subject: Re: [mpich-discuss] MPID_nem_tcp_init(377)... gethostbyname failed, >>>>> mac (errno 1) >>>>> >>>>> Am 01.03.2014 um 17:58 schrieb Luis Cebamanos: >>>>> >>>>>> Here it is the output of those commands: >>>>>> >>>>>> lcebaman@mac~/$ nslookup localhost >>>>>> Server: 192.168.1.1 >>>>>> Address: 192.168.1.1#53 >>>>>> >>>>>> Name: localhost >>>>>> Address: 127.0.0.1 >>>>>> >>>>>> lcebaman@mac~/$ nslookup mac >>>>>> Server: 192.168.1.1 >>>>>> Address: 192.168.1.1#53 >>>>>> >>>>>> Name: mac >>>>>> Address: 192.168.1.127 >>>>> >>>>> Is this a fixed address you get everytime? You could add it to /etc/hosts. >>>>> >>>>> Interesting that MPICH throws an error about resolving "mac", although DNS is working. >>>>> >>>>> -- Reuti >>>>> >>>>> >>>>>> lcebaman@mac~/$ hostname >>>>>> mac >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 28 Feb 2014, at 23:35, Reuti <reuti(a)staff.uni-marburg.de> wrote: >>>>>> >>>>>>> Am 28.02.2014 um 14:39 schrieb Luis Cebamanos: >>>>>>> >>>>>>>> Hi Pavan, >>>>>>>> >>>>>>>> Yes, that is the output: >>>>>>>> >>>>>>>> $ cat /etc/hosts >>>>>>>> ## >>>>>>>> # Host Database >>>>>>>> # >>>>>>>> # localhost is used to configure the loopback interface >>>>>>>> # when the system is booting. Do not change this entry. >>>>>>>> ## >>>>>>>> 127.0.0.1 localhost >>>>>>>> 255.255.255.255 broadcasthost >>>>>>>> ::1 localhost >>>>>>>> fe80::1%lo0 localhost >>>>>>> >>>>>>> What is the output of: >>>>>>> >>>>>>> $ nslookup localhost >>>>>>> $ nslookup mac >>>>>>> $ hostname >>>>>>> >>>>>>> AFAICS your machine has no domainname, otherwise it would show up in the MPICH error message after "mac". >>>>>>> >>>>>>> -- Reuti >>>>>>> >>>>>>> >>>>>>>> On 28/02/14 13:35, Balaji, Pavan wrote: >>>>>>>>> $ cat /etc/hosts >>>>>>>>>>> ## >>>>>>>>>>> # Host Database >>>>>>>>>>> # >>>>>>>>>>> # localhost is used to configure the loopback interface >>>>>>>>>>> # when the system is booting. Do not change this entry. >>>>>>>>>>> ## >>>>>>>>>>> 127.0.0.1 localhost >>>>>>>>>>> 255.255.255.255 broadcasthost >>>>>>>>>>> ::1 localhost >>>>>>>>>>> fe80::1%lo0 localhost >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> The University of Edinburgh is a charitable body, registered in >>>>>>>> Scotland, with registration number SC005336. >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>>> To manage subscription options or unsubscribe: >>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>> >>>>>>> _______________________________________________ >>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>> To manage subscription options or unsubscribe: >>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> The University of Edinburgh is a charitable body, registered in >>>>> Scotland, with registration number SC005336. >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: [mpich-discuss] very slow file writes independent of file size
by Wei-keng Liao 04 Mar '14

04 Mar '14

project directory is a GPFS. lfs command is for Lustre only. You can run lfs command on the directory where you wrote your files under $SCRATCH. Files created in a Lustre directory inherit the striping settings of that directory. The default stripe_count on Edison appears to be one. So, if you have large amount of data to write, then changing stripe_count to a larger value should give a much higher performance. All you need to do is to first create a new directory and then set the stripe_count to say 96 using lfs command and write your files there. For example, mkdir $SCRATCH/new_directory lfs setstripe -s 1M -c 96 -o -1 $SCRATCH/new_directory Using independent file points will most likely give you a much better performance, as some implementations of shared file pointer MPI-IO functions use a temporary file to store the pointer (so one process's change to the pointer can be immediately visible to other processes). Because of this and other overheads, the use of shared-pointer functions is generally discouraged. However, I think the poor performance in your case is mainly because of file striping issue. Wei-keng On Mar 3, 2014, at 11:36 PM, Geoffrey Irving wrote: > Unfortunately, I have moved the files since I ran the job (from > scratch to project). Running lfs getstripe on files generated by a > different run in the project directory produces > > edison04:restart-5% lfs getstripe sparse-15.npy > can't find fs root for > '/global/project/projectdirs/pentago/restart-5/sparse-15.npy': -19 > 'sparse-15.npy' is not on a Lustre filesystem: No such device (19) > cb_getstripe: 'sparse-15.npy' not on a Lustre fs?: Inappropriate ioctl > for device (25) > error: getstripe failed for sparse-15.npy. > > I will certainly use MPI_File_write_at_all in future, though it's > unintuitive if accidentally using a different but still collective > routine results in >3 minutes hangs. > > Thanks, > Geoffrey > > > > On Mon, Mar 3, 2014 at 10:43 AM, Wei-keng Liao > <wkliao(a)eecs.northwestern.edu> wrote: >> Hi, Geoffrey >> >> Edison has a Lustre file system and I assume your program writes files >> there. If this is the case, please check the file striping setting of your >> output files, using command "lfs getstripe filename" >> >> Usually, a high-performance I/O can be obtained if you have configured >> the lustre striping setting to a higher stripe_count (96 or 144 max on Edison). >> >> In addition, MPI_File_write_ordered is for shared file pointers. >> It can have some impact to the performance. Using independent file pointers >> often results in a better performance. You might want to consider to >> change your program to use those functions (eg. MPI_File_write_all). >> >> >> Wei-keng >> >> On Mar 3, 2014, at 12:21 AM, Geoffrey Irving wrote: >> >>> I'm doing postmortem on a 2048 node (16384 rank) job on Edison, and >>> trying to understand why my I/O performance might have been slow. >>> >>> Here's the data: >>> >>> Measured I/O bandwidth: >>> slice 35 write sparse bandwidth = 6082640 / (3.52519e+06 s / 16384) = >>> 2.63287e-05 GB/s >>> slice 34 write sparse bandwidth = 13824080 / (3.66608e+06 s / 16384) = >>> 5.75379e-05 GB/s >>> slice 33 write sparse bandwidth = 24754256 / (2.83647e+06 s / 16384) = >>> 0.000133166 GB/s >>> slice 32 write sparse bandwidth = 39370832 / (3.47016e+06 s / 16384) = >>> 0.000173119 GB/s >>> slice 31 write sparse bandwidth = 55812176 / (2.53623e+06 s / 16384) = >>> 0.000335785 GB/s >>> slice 30 write sparse bandwidth = 74741840 / (2.5714e+06 s / 16384) = >>> 0.00044352 GB/s >>> slice 29 write sparse bandwidth = 93560912 / (2.67336e+06 s / 16384) = >>> 0.000534019 GB/s >>> slice 28 write sparse bandwidth = 112803920 / (2.74639e+06 s / 16384) >>> = 0.000626733 GB/s >>> slice 27 write sparse bandwidth = 128194640 / (3.1603e+06 s / 16384) = >>> 0.000618958 GB/s >>> slice 26 write sparse bandwidth = 141281360 / (3.12754e+06 s / 16384) >>> = 0.00068929 GB/s >>> slice 25 write sparse bandwidth = 148193360 / (2.62376e+06 s / 16384) >>> = 0.000861835 GB/s >>> slice 24 write sparse bandwidth = 151861328 / (3.2145e+06 s / 16384) = >>> 0.000720865 GB/s >>> slice 23 write sparse bandwidth = 148193360 / (2.44736e+06 s / 16384) >>> = 0.000923956 GB/s >>> slice 22 write sparse bandwidth = 142055504 / (3.15962e+06 s / 16384) >>> = 0.000686031 GB/s >>> slice 21 write sparse bandwidth = 130388048 / (3.09774e+06 s / 16384) >>> = 0.000642263 GB/s >>> slice 20 write sparse bandwidth = 117964880 / (3.02676e+06 s / 16384) >>> = 0.000594696 GB/s >>> slice 19 write sparse bandwidth = 101560400 / (2.97198e+06 s / 16384) >>> = 0.000521434 GB/s >>> slice 18 write sparse bandwidth = 86372432 / (2.96247e+06 s / 16384) = >>> 0.000444878 GB/s >>> slice 18 write sections bandwidth = 1954957518434 / (1.83937e+07 s / >>> 16384) = 1.62177 GB/s >>> slice 17 write sparse bandwidth = 70170704 / (2.88973e+06 s / 16384) = >>> 0.000370526 GB/s >>> slice 17 write sections bandwidth = 1475380615039 / (1.36018e+07 s / >>> 16384) = 1.65511 GB/s >>> slice 17 read bandwidth (192 nodes) = 1475380615039 / 3383.36 s = 0.406122 GB/s >>> per node: measured = 2.16598 MB/s, theoretical peak = 33.1341 MB/s >>> >>> Focusing on the "sparse" lines, the main point is that the time seems >>> to be roughly independent of file size (plot attached). Each timing >>> sample consists of (1) setup which I believe is negligible, (2) >>> MPI_File_open, (3) MPI_File_write_ordered, (4) MPI_File_close. >>> >>> What might have caused these file writes to take so long? >>> >>> Geoffrey >>> <2014-03-02-221950_1220x500.png>_______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] very slow file writes independent of file size
by Geoffrey Irving 03 Mar '14

03 Mar '14

Unfortunately, I have moved the files since I ran the job (from scratch to project). Running lfs getstripe on files generated by a different run in the project directory produces edison04:restart-5% lfs getstripe sparse-15.npy can't find fs root for '/global/project/projectdirs/pentago/restart-5/sparse-15.npy': -19 'sparse-15.npy' is not on a Lustre filesystem: No such device (19) cb_getstripe: 'sparse-15.npy' not on a Lustre fs?: Inappropriate ioctl for device (25) error: getstripe failed for sparse-15.npy. I will certainly use MPI_File_write_at_all in future, though it's unintuitive if accidentally using a different but still collective routine results in >3 minutes hangs. Thanks, Geoffrey On Mon, Mar 3, 2014 at 10:43 AM, Wei-keng Liao <wkliao(a)eecs.northwestern.edu> wrote: > Hi, Geoffrey > > Edison has a Lustre file system and I assume your program writes files > there. If this is the case, please check the file striping setting of your > output files, using command "lfs getstripe filename" > > Usually, a high-performance I/O can be obtained if you have configured > the lustre striping setting to a higher stripe_count (96 or 144 max on Edison). > > In addition, MPI_File_write_ordered is for shared file pointers. > It can have some impact to the performance. Using independent file pointers > often results in a better performance. You might want to consider to > change your program to use those functions (eg. MPI_File_write_all). > > > Wei-keng > > On Mar 3, 2014, at 12:21 AM, Geoffrey Irving wrote: > >> I'm doing postmortem on a 2048 node (16384 rank) job on Edison, and >> trying to understand why my I/O performance might have been slow. >> >> Here's the data: >> >> Measured I/O bandwidth: >> slice 35 write sparse bandwidth = 6082640 / (3.52519e+06 s / 16384) = >> 2.63287e-05 GB/s >> slice 34 write sparse bandwidth = 13824080 / (3.66608e+06 s / 16384) = >> 5.75379e-05 GB/s >> slice 33 write sparse bandwidth = 24754256 / (2.83647e+06 s / 16384) = >> 0.000133166 GB/s >> slice 32 write sparse bandwidth = 39370832 / (3.47016e+06 s / 16384) = >> 0.000173119 GB/s >> slice 31 write sparse bandwidth = 55812176 / (2.53623e+06 s / 16384) = >> 0.000335785 GB/s >> slice 30 write sparse bandwidth = 74741840 / (2.5714e+06 s / 16384) = >> 0.00044352 GB/s >> slice 29 write sparse bandwidth = 93560912 / (2.67336e+06 s / 16384) = >> 0.000534019 GB/s >> slice 28 write sparse bandwidth = 112803920 / (2.74639e+06 s / 16384) >> = 0.000626733 GB/s >> slice 27 write sparse bandwidth = 128194640 / (3.1603e+06 s / 16384) = >> 0.000618958 GB/s >> slice 26 write sparse bandwidth = 141281360 / (3.12754e+06 s / 16384) >> = 0.00068929 GB/s >> slice 25 write sparse bandwidth = 148193360 / (2.62376e+06 s / 16384) >> = 0.000861835 GB/s >> slice 24 write sparse bandwidth = 151861328 / (3.2145e+06 s / 16384) = >> 0.000720865 GB/s >> slice 23 write sparse bandwidth = 148193360 / (2.44736e+06 s / 16384) >> = 0.000923956 GB/s >> slice 22 write sparse bandwidth = 142055504 / (3.15962e+06 s / 16384) >> = 0.000686031 GB/s >> slice 21 write sparse bandwidth = 130388048 / (3.09774e+06 s / 16384) >> = 0.000642263 GB/s >> slice 20 write sparse bandwidth = 117964880 / (3.02676e+06 s / 16384) >> = 0.000594696 GB/s >> slice 19 write sparse bandwidth = 101560400 / (2.97198e+06 s / 16384) >> = 0.000521434 GB/s >> slice 18 write sparse bandwidth = 86372432 / (2.96247e+06 s / 16384) = >> 0.000444878 GB/s >> slice 18 write sections bandwidth = 1954957518434 / (1.83937e+07 s / >> 16384) = 1.62177 GB/s >> slice 17 write sparse bandwidth = 70170704 / (2.88973e+06 s / 16384) = >> 0.000370526 GB/s >> slice 17 write sections bandwidth = 1475380615039 / (1.36018e+07 s / >> 16384) = 1.65511 GB/s >> slice 17 read bandwidth (192 nodes) = 1475380615039 / 3383.36 s = 0.406122 GB/s >> per node: measured = 2.16598 MB/s, theoretical peak = 33.1341 MB/s >> >> Focusing on the "sparse" lines, the main point is that the time seems >> to be roughly independent of file size (plot attached). Each timing >> sample consists of (1) setup which I believe is negligible, (2) >> MPI_File_open, (3) MPI_File_write_ordered, (4) MPI_File_close. >> >> What might have caused these file writes to take so long? >> >> Geoffrey >> <2014-03-02-221950_1220x500.png>_______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Problems with running make
by Gus Correa 03 Mar '14

03 Mar '14

On 03/03/2014 06:57 PM, Ron Palmer wrote: > Thanks Gus, I will read up on those queuing systems. > > mathlab, not really, I do not know how to use it and I am the only user > (and I could always re-enable the windows manager later if required). I > do generate gnuplots in my perl scripts but email them to me as the > inversion progresses. All results get downloaded via ftp to a > workstation where are graphics (visualisation, interpretation ad > fly-throughs) are done. The cluster is really only horsepower, and all > processes and OS could quite comfortably run in RAM only, with a token > harddrive for intermediate files. If they crash, I just start them up > again, only the last few hours to a few days lost, nothing longer term. > So many things, so little time... > > Yes, not mpi related, so I appropriately apologise to all and sundry! > > :-) Hi Ron When it comes to the details of how to setup a queuing system, of course you will get better advice in their respective mailing lists. However, troubleshooting network setup, ssh authentication, integration of MPI with queuing system, etc, these are all related to MPI, and it is hard to trace a dividing line, as you may have noticed. IMHO this list is not strict in this regard. I for sure think that your postings were focused on how to get MPICH up and running, well taken, and very appropriate. Gus Correa > > On 4/03/2014 09:49, Gus Correa wrote: >> Hi Ron >> >> On 03/03/2014 06:19 PM, Ron Palmer wrote: >>> Reuti, good comment about primary interface for mpi and secondary for >>> the rest of the world. >>> >>> Yes, setting it up from scratch, and I would love a queuing system as >>> well, I just had no idea such existed (learning by the minute!). These >>> computers are dedicated to this parallelisation jobs I am running and >>> will not run anything else. Would you be able to point my nose in a >>> direction of a simple queuing system? >>> >> >> In the open source / free software world: >> >> I use Torque+Maui: >> >> http://www.adaptivecomputing.com/products/open-source/torque/ >> http://www.adaptivecomputing.com/products/open-source/maui/ >> >> Another very popular queuing system is the Sun Grid Engine (maybe this >> is what Reuti uses?), which I think became the Open Grid Scheduler. >> Better ask somebody else (Reuti maybe) about this one. >> >> http://gridscheduler.sourceforge.net/ >> >> Yet a third one is LLNL's Slurm: >> >> https://computing.llnl.gov/linux/slurm/ >> >> >>> All my parallel processing are controlled by my own perl scripts and >>> commandline commands (provided binaries), nothing GUI at all (which is >>> kinda nice). >>> >>> Do you think shutting down the xserver would make any difference to the >>> performance of these computers? I am pretty sure that they al have a >>> windows manager running by default... I only got them going last week so >>> I have not (yet) gone in to this level of detail. >>> >> >> If you are sure nobody will ever want to run Matlab, >> or to show a graph with gnuplot, or perhaps more likely, use a >> graphical debugger such as DDD (or the equivalent from Intel or PGI), >> you can live without X-windows. >> >> I don't see it as a big burden, though, unless there is somebody doing >> heavy GUI stuff while other computation is going on concurrently. >> >> You could simply start the computers at runlevel 3. >> >> ** >> >> OK, this is now more about cluster then about MPICH specifically. :) >> >> I hope this helps, >> Gus Correa >> >>> Thanks, >>> Ron >>> >>> On 4/03/2014 09:06, Reuti wrote: >>>> Am 03.03.2014 um 23:54 schrieb Gus Correa: >>>> >>>>> On 03/03/2014 04:36 PM, Ron Palmer wrote: >>>>>> Thanks Reuti for your comments. I will peruse that FAQ detail. >>>>>> >>>>>> I just thought of the fact that each of these rack computers have 4 >>>>>> ethernet sockets, eth0 - eth3... I could connect the cluster on a >>>>>> separate ethernet sockets via an extra switch not connected to the >>>>>> internet or any other computers, and accept all communication among >>>>>> them, and keep iptables up on the ethx connected to the outside >>>>>> world. I >>>>>> guess I would have to set up routing tables or something. Ah, more >>>>>> reading :-) >>>>>> >>>>>> Thanks for your help. >>>>>> Ron >>>>>> >>>>> Hi Ron >>>>> >>>>> If those extra interfaces are not in use, >>>>> and if you have a spare switch, >>>>> you can setup a separate private subnet exclusively for MPI. >>>>> You need to configure the interfaces consistently (IP, subnet mask, >>>>> perhaps a gateway). Configuring them statically is easy: >>>>> >>>>> https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… >>>>> >>>>> >>>>> >>>>> Use a subnet that doesn't intersect the existent/original IP range. >>>>> >>>>> http://en.wikipedia.org/wiki/Private_network >>>>> >>>>> You could also create host names associated to those IPs (say >>>>> node01, node02, node02), resolve them via /etc/hosts on each computer, >>>>> set passwordless ssh across these newly named "hosts". >>>>> This may simpler/safer than messing with the iptables. >>>>> >>>>> [Actually, the IP addresses you showed 192.168.X.Y, sound as a private >>>>> subnet already, not Internet, but that may be the subnet for your >>>>> organization/school/department already. So, you may set up a different >>>>> one on these three computers for MPI and very-local access.] >>>> Yep, maybe in the 10.0.0.0/8 range. >>>> >>>> BTW: are you setting up a cluster from scratch - will you also add any >>>> queuing system later on? >>>> >>>> >>>>> OpenMPI allows you to choose the interface that it will use, >>>>> so you can direct it to your very-local subnet: >>>>> >>>>> http://www.open-mpi.org/faq/?category=tcp#tcp-selection >>>> MPICH: >>>> >>>> http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Hydra… >>>> >>>> >>>> >>>> It should work for other Ethernet interfaces too. >>>> >>>> Nevertheless: in any case it might be easier for the applications to >>>> use the primary interface solely for MPI, and any other one for the >>>> external access (this is usually my setup, as I don't have to provide >>>> the assignment of other interfaces this way). >>>> >>>> -- Reuti >>>> >>>> >>>> >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> >> > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_Publish_name
by Balaji, Pavan 03 Mar '14

03 Mar '14

Yes. On Mar 3, 2014, at 3:48 PM, Ramesh Vinayagam <rvinayagam.85(a)gmail.com> wrote: > Is it possible to use MPI_Publish_name for a server/client style approach? > > Server: > MPI_Publish_name > > Client > MPI_Lookup_name > > > Thanks > Ramesh > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Problems with running make
by Ron Palmer 03 Mar '14

03 Mar '14

Thanks Gus, I will read up on those queuing systems. mathlab, not really, I do not know how to use it and I am the only user (and I could always re-enable the windows manager later if required). I do generate gnuplots in my perl scripts but email them to me as the inversion progresses. All results get downloaded via ftp to a workstation where are graphics (visualisation, interpretation ad fly-throughs) are done. The cluster is really only horsepower, and all processes and OS could quite comfortably run in RAM only, with a token harddrive for intermediate files. If they crash, I just start them up again, only the last few hours to a few days lost, nothing longer term. So many things, so little time... Yes, not mpi related, so I appropriately apologise to all and sundry! :-) On 4/03/2014 09:49, Gus Correa wrote: > Hi Ron > > On 03/03/2014 06:19 PM, Ron Palmer wrote: >> Reuti, good comment about primary interface for mpi and secondary for >> the rest of the world. >> >> Yes, setting it up from scratch, and I would love a queuing system as >> well, I just had no idea such existed (learning by the minute!). These >> computers are dedicated to this parallelisation jobs I am running and >> will not run anything else. Would you be able to point my nose in a >> direction of a simple queuing system? >> > > In the open source / free software world: > > I use Torque+Maui: > > http://www.adaptivecomputing.com/products/open-source/torque/ > http://www.adaptivecomputing.com/products/open-source/maui/ > > Another very popular queuing system is the Sun Grid Engine (maybe this > is what Reuti uses?), which I think became the Open Grid Scheduler. > Better ask somebody else (Reuti maybe) about this one. > > http://gridscheduler.sourceforge.net/ > > Yet a third one is LLNL's Slurm: > > https://computing.llnl.gov/linux/slurm/ > > >> All my parallel processing are controlled by my own perl scripts and >> commandline commands (provided binaries), nothing GUI at all (which is >> kinda nice). >> >> Do you think shutting down the xserver would make any difference to the >> performance of these computers? I am pretty sure that they al have a >> windows manager running by default... I only got them going last week so >> I have not (yet) gone in to this level of detail. >> > > If you are sure nobody will ever want to run Matlab, > or to show a graph with gnuplot, or perhaps more likely, use a > graphical debugger such as DDD (or the equivalent from Intel or PGI), > you can live without X-windows. > > I don't see it as a big burden, though, unless there is somebody doing > heavy GUI stuff while other computation is going on concurrently. > > You could simply start the computers at runlevel 3. > > ** > > OK, this is now more about cluster then about MPICH specifically. :) > > I hope this helps, > Gus Correa > >> Thanks, >> Ron >> >> On 4/03/2014 09:06, Reuti wrote: >>> Am 03.03.2014 um 23:54 schrieb Gus Correa: >>> >>>> On 03/03/2014 04:36 PM, Ron Palmer wrote: >>>>> Thanks Reuti for your comments. I will peruse that FAQ detail. >>>>> >>>>> I just thought of the fact that each of these rack computers have 4 >>>>> ethernet sockets, eth0 - eth3... I could connect the cluster on a >>>>> separate ethernet sockets via an extra switch not connected to the >>>>> internet or any other computers, and accept all communication among >>>>> them, and keep iptables up on the ethx connected to the outside >>>>> world. I >>>>> guess I would have to set up routing tables or something. Ah, more >>>>> reading :-) >>>>> >>>>> Thanks for your help. >>>>> Ron >>>>> >>>> Hi Ron >>>> >>>> If those extra interfaces are not in use, >>>> and if you have a spare switch, >>>> you can setup a separate private subnet exclusively for MPI. >>>> You need to configure the interfaces consistently (IP, subnet mask, >>>> perhaps a gateway). Configuring them statically is easy: >>>> >>>> https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… >>>> >>>> >>>> >>>> Use a subnet that doesn't intersect the existent/original IP range. >>>> >>>> http://en.wikipedia.org/wiki/Private_network >>>> >>>> You could also create host names associated to those IPs (say >>>> node01, node02, node02), resolve them via /etc/hosts on each computer, >>>> set passwordless ssh across these newly named "hosts". >>>> This may simpler/safer than messing with the iptables. >>>> >>>> [Actually, the IP addresses you showed 192.168.X.Y, sound as a private >>>> subnet already, not Internet, but that may be the subnet for your >>>> organization/school/department already. So, you may set up a different >>>> one on these three computers for MPI and very-local access.] >>> Yep, maybe in the 10.0.0.0/8 range. >>> >>> BTW: are you setting up a cluster from scratch - will you also add any >>> queuing system later on? >>> >>> >>>> OpenMPI allows you to choose the interface that it will use, >>>> so you can direct it to your very-local subnet: >>>> >>>> http://www.open-mpi.org/faq/?category=tcp#tcp-selection >>> MPICH: >>> >>> http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Hydra… >>> >>> >>> >>> It should work for other Ethernet interfaces too. >>> >>> Nevertheless: in any case it might be easier for the applications to >>> use the primary interface solely for MPI, and any other one for the >>> external access (this is usually my setup, as I don't have to provide >>> the assignment of other interfaces this way). >>> >>> -- Reuti >>> >>> >>> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > >

1 0

Re: [mpich-discuss] Problems with running make
by Gus Correa 03 Mar '14

03 Mar '14

Hi Ron On 03/03/2014 06:19 PM, Ron Palmer wrote: > Reuti, good comment about primary interface for mpi and secondary for > the rest of the world. > > Yes, setting it up from scratch, and I would love a queuing system as > well, I just had no idea such existed (learning by the minute!). These > computers are dedicated to this parallelisation jobs I am running and > will not run anything else. Would you be able to point my nose in a > direction of a simple queuing system? > In the open source / free software world: I use Torque+Maui: http://www.adaptivecomputing.com/products/open-source/torque/ http://www.adaptivecomputing.com/products/open-source/maui/ Another very popular queuing system is the Sun Grid Engine (maybe this is what Reuti uses?), which I think became the Open Grid Scheduler. Better ask somebody else (Reuti maybe) about this one. http://gridscheduler.sourceforge.net/ Yet a third one is LLNL's Slurm: https://computing.llnl.gov/linux/slurm/ > All my parallel processing are controlled by my own perl scripts and > commandline commands (provided binaries), nothing GUI at all (which is > kinda nice). > > Do you think shutting down the xserver would make any difference to the > performance of these computers? I am pretty sure that they al have a > windows manager running by default... I only got them going last week so > I have not (yet) gone in to this level of detail. > If you are sure nobody will ever want to run Matlab, or to show a graph with gnuplot, or perhaps more likely, use a graphical debugger such as DDD (or the equivalent from Intel or PGI), you can live without X-windows. I don't see it as a big burden, though, unless there is somebody doing heavy GUI stuff while other computation is going on concurrently. You could simply start the computers at runlevel 3. ** OK, this is now more about cluster then about MPICH specifically. :) I hope this helps, Gus Correa > Thanks, > Ron > > On 4/03/2014 09:06, Reuti wrote: >> Am 03.03.2014 um 23:54 schrieb Gus Correa: >> >>> On 03/03/2014 04:36 PM, Ron Palmer wrote: >>>> Thanks Reuti for your comments. I will peruse that FAQ detail. >>>> >>>> I just thought of the fact that each of these rack computers have 4 >>>> ethernet sockets, eth0 - eth3... I could connect the cluster on a >>>> separate ethernet sockets via an extra switch not connected to the >>>> internet or any other computers, and accept all communication among >>>> them, and keep iptables up on the ethx connected to the outside >>>> world. I >>>> guess I would have to set up routing tables or something. Ah, more >>>> reading :-) >>>> >>>> Thanks for your help. >>>> Ron >>>> >>> Hi Ron >>> >>> If those extra interfaces are not in use, >>> and if you have a spare switch, >>> you can setup a separate private subnet exclusively for MPI. >>> You need to configure the interfaces consistently (IP, subnet mask, >>> perhaps a gateway). Configuring them statically is easy: >>> >>> https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… >>> >>> >>> Use a subnet that doesn't intersect the existent/original IP range. >>> >>> http://en.wikipedia.org/wiki/Private_network >>> >>> You could also create host names associated to those IPs (say >>> node01, node02, node02), resolve them via /etc/hosts on each computer, >>> set passwordless ssh across these newly named "hosts". >>> This may simpler/safer than messing with the iptables. >>> >>> [Actually, the IP addresses you showed 192.168.X.Y, sound as a private >>> subnet already, not Internet, but that may be the subnet for your >>> organization/school/department already. So, you may set up a different >>> one on these three computers for MPI and very-local access.] >> Yep, maybe in the 10.0.0.0/8 range. >> >> BTW: are you setting up a cluster from scratch - will you also add any >> queuing system later on? >> >> >>> OpenMPI allows you to choose the interface that it will use, >>> so you can direct it to your very-local subnet: >>> >>> http://www.open-mpi.org/faq/?category=tcp#tcp-selection >> MPICH: >> >> http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Hydra… >> >> >> It should work for other Ethernet interfaces too. >> >> Nevertheless: in any case it might be easier for the applications to >> use the primary interface solely for MPI, and any other one for the >> external access (this is usually my setup, as I don't have to provide >> the assignment of other interfaces this way). >> >> -- Reuti >> >> >> > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Problems with running make
by Gus Correa 03 Mar '14

03 Mar '14

On 03/03/2014 06:06 PM, Reuti wrote: > Am 03.03.2014 um 23:54 schrieb Gus Correa: > >> OpenMPI allows you to choose the interface that it will use, >> so you can direct it to your very-local subnet: >> >> http://www.open-mpi.org/faq/?category=tcp#tcp-selection > > MPICH: > > http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Hydra… > > It should work for other Ethernet interfaces too. > > Nevertheless: in any case it might be easier for the applications to use the primary interface solely for MPI, and any other one for the external access (this is usually my setup, as I don't have to provide the assignment of other interfaces this way). > > -- Reuti > Oops! Apologies for the not so polite reference to another MPI implementation. I must have be missing some sleep. As Reuti points out, you can do the same with MPICH as well. Gus Correa

1 0