March 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Problems with running make
by Ron Palmer 03 Mar '14

03 Mar '14

Reuti, good comment about primary interface for mpi and secondary for the rest of the world. Yes, setting it up from scratch, and I would love a queuing system as well, I just had no idea such existed (learning by the minute!). These computers are dedicated to this parallelisation jobs I am running and will not run anything else. Would you be able to point my nose in a direction of a simple queuing system? All my parallel processing are controlled by my own perl scripts and commandline commands (provided binaries), nothing GUI at all (which is kinda nice). Do you think shutting down the xserver would make any difference to the performance of these computers? I am pretty sure that they al have a windows manager running by default... I only got them going last week so I have not (yet) gone in to this level of detail. Thanks, Ron On 4/03/2014 09:06, Reuti wrote: > Am 03.03.2014 um 23:54 schrieb Gus Correa: > >> On 03/03/2014 04:36 PM, Ron Palmer wrote: >>> Thanks Reuti for your comments. I will peruse that FAQ detail. >>> >>> I just thought of the fact that each of these rack computers have 4 >>> ethernet sockets, eth0 - eth3... I could connect the cluster on a >>> separate ethernet sockets via an extra switch not connected to the >>> internet or any other computers, and accept all communication among >>> them, and keep iptables up on the ethx connected to the outside world. I >>> guess I would have to set up routing tables or something. Ah, more >>> reading :-) >>> >>> Thanks for your help. >>> Ron >>> >> Hi Ron >> >> If those extra interfaces are not in use, >> and if you have a spare switch, >> you can setup a separate private subnet exclusively for MPI. >> You need to configure the interfaces consistently (IP, subnet mask, >> perhaps a gateway). Configuring them statically is easy: >> >> https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… >> >> Use a subnet that doesn't intersect the existent/original IP range. >> >> http://en.wikipedia.org/wiki/Private_network >> >> You could also create host names associated to those IPs (say >> node01, node02, node02), resolve them via /etc/hosts on each computer, >> set passwordless ssh across these newly named "hosts". >> This may simpler/safer than messing with the iptables. >> >> [Actually, the IP addresses you showed 192.168.X.Y, sound as a private >> subnet already, not Internet, but that may be the subnet for your >> organization/school/department already. So, you may set up a different >> one on these three computers for MPI and very-local access.] > Yep, maybe in the 10.0.0.0/8 range. > > BTW: are you setting up a cluster from scratch - will you also add any queuing system later on? > > >> OpenMPI allows you to choose the interface that it will use, >> so you can direct it to your very-local subnet: >> >> http://www.open-mpi.org/faq/?category=tcp#tcp-selection > MPICH: > > http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Hydra… > > It should work for other Ethernet interfaces too. > > Nevertheless: in any case it might be easier for the applications to use the primary interface solely for MPI, and any other one for the external access (this is usually my setup, as I don't have to provide the assignment of other interfaces this way). > > -- Reuti > > >

1 0

Re: [mpich-discuss] Problems with running make
by Ron Palmer 03 Mar '14

03 Mar '14

Gus, thanks for your suggestions and links to more info; I was contemplating using any of the three unused ethernet interfaces at the back of each of the three computers, and using a cluster-only subnet for the inversion. You comments about a separate hostname and so forth are great, and I can re-activate iptables on those interfaces connected to the outside world. I have just read up on 'screen', an approach that suits me like glove - all my parallel processing is via perl scripts/process control and command line commands, and I have no use for xwindows or other windows managers. Being able to remotely detach (-d), re-attach (-R) and even duplicate (-x) are real time savers for when I log in remote to check the progress. Thanks, Ron On 4/03/2014 08:54, Gus Correa wrote: > > On 03/03/2014 04:36 PM, Ron Palmer wrote: >> Thanks Reuti for your comments. I will peruse that FAQ detail. >> >> I just thought of the fact that each of these rack computers have 4 >> ethernet sockets, eth0 - eth3... I could connect the cluster on a >> separate ethernet sockets via an extra switch not connected to the >> internet or any other computers, and accept all communication among >> them, and keep iptables up on the ethx connected to the outside world. I >> guess I would have to set up routing tables or something. Ah, more >> reading :-) >> >> Thanks for your help. >> Ron >> > Hi Ron > > If those extra interfaces are not in use, > and if you have a spare switch, > you can setup a separate private subnet exclusively for MPI. > You need to configure the interfaces consistently (IP, subnet mask, > perhaps a gateway). Configuring them statically is easy: > > https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… > > > Use a subnet that doesn't intersect the existent/original IP range. > > http://en.wikipedia.org/wiki/Private_network > > You could also create host names associated to those IPs (say > node01, node02, node02), resolve them via /etc/hosts on each computer, > set passwordless ssh across these newly named "hosts". > This may simpler/safer than messing with the iptables. > > [Actually, the IP addresses you showed 192.168.X.Y, sound as a private > subnet already, not Internet, but that may be the subnet for your > organization/school/department already. So, you may set up a different > one on these three computers for MPI and very-local access.] > > OpenMPI allows you to choose the interface that it will use, > so you can direct it to your very-local subnet: > > http://www.open-mpi.org/faq/?category=tcp#tcp-selection > > I hope this helps, > Gus > > >> On 4/03/2014 00:13, Reuti wrote: >>> Am 03.03.2014 um 01:34 schrieb Ron Palmer: >>> >>>> Gus, >>>> I have just replied with the details of the success, but I will >>>> clarify your questions here, if it helps next time. >>>> >>>> Re the actual application to be run, 'inversion', I have only >>>> received binaries. I used to run them on gainsborough (without mpi) >>>> and that worked fine. >>>> >>>> Home directories are not nfs shared, they are individual and >>>> separate, only the name is repeated. >>>> >>>> I had tried password free ssh in all directions and permutations. >>>> >>>> I had iptables down on sargeant and up on the other two. >>>> >>>> Yes, I installed the gcc-gfortran.x86_64 AFTER I took those >>>> screnshots, and the post install output was identical to the top one >>>> (sargeant). >>>> >>>> I am unsure about cpi and fortran... >>>> >>>> Stuff remaining to get sorted out: >>>> 1. Get that hyperthreading set up - what is your suggestions? Disable >>>> and let mpi manage the cores? >>> It depends on your applications, whether you can make use of it. Test >>> it with HT switched off, switched on and increase the number of >>> processes, until you see that any process is no longer running at 100% >>> and you judge that you don't tolerate this slow down. >>> >>> It's not uncommon to see an improvement up to 150% with HT turned on, >>> but not 200% (depending on the workload). >>> >>> >>>> 2. run mpiexec with iptables up, need to figure out what traffic to >>>> allow. >>> https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_How_do… >>> >>> >>> >>> -- Reuti >>> >>> >>>> Great many thanks to all, and Gus, Reuti and Rajeev in particular. >>>> >>>> Cheers, >>>> Ron >>>> >>>> On 3/03/2014 10:09, Gustavo Correa wrote: >>>>> Hi Ron >>>>> >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- *Ron Palmer*MSc MBA. Principal Geophysicist ron.palmer(a)pgcgroup.com.au <mailto:[email protected]> 0413 579 099 07 3103 4963

1 0

Re: [mpich-discuss] Problems with running make
by Reuti 03 Mar '14

03 Mar '14

Am 03.03.2014 um 23:54 schrieb Gus Correa: > > On 03/03/2014 04:36 PM, Ron Palmer wrote: >> Thanks Reuti for your comments. I will peruse that FAQ detail. >> >> I just thought of the fact that each of these rack computers have 4 >> ethernet sockets, eth0 - eth3... I could connect the cluster on a >> separate ethernet sockets via an extra switch not connected to the >> internet or any other computers, and accept all communication among >> them, and keep iptables up on the ethx connected to the outside world. I >> guess I would have to set up routing tables or something. Ah, more >> reading :-) >> >> Thanks for your help. >> Ron >> > Hi Ron > > If those extra interfaces are not in use, > and if you have a spare switch, > you can setup a separate private subnet exclusively for MPI. > You need to configure the interfaces consistently (IP, subnet mask, > perhaps a gateway). Configuring them statically is easy: > > https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… > > Use a subnet that doesn't intersect the existent/original IP range. > > http://en.wikipedia.org/wiki/Private_network > > You could also create host names associated to those IPs (say > node01, node02, node02), resolve them via /etc/hosts on each computer, > set passwordless ssh across these newly named "hosts". > This may simpler/safer than messing with the iptables. > > [Actually, the IP addresses you showed 192.168.X.Y, sound as a private > subnet already, not Internet, but that may be the subnet for your > organization/school/department already. So, you may set up a different > one on these three computers for MPI and very-local access.] Yep, maybe in the 10.0.0.0/8 range. BTW: are you setting up a cluster from scratch - will you also add any queuing system later on? > OpenMPI allows you to choose the interface that it will use, > so you can direct it to your very-local subnet: > > http://www.open-mpi.org/faq/?category=tcp#tcp-selection MPICH: http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Hydra… It should work for other Ethernet interfaces too. Nevertheless: in any case it might be easier for the applications to use the primary interface solely for MPI, and any other one for the external access (this is usually my setup, as I don't have to provide the assignment of other interfaces this way). -- Reuti > I hope this helps, > Gus > > >> On 4/03/2014 00:13, Reuti wrote: >>> Am 03.03.2014 um 01:34 schrieb Ron Palmer: >>> >>>> Gus, >>>> I have just replied with the details of the success, but I will >>>> clarify your questions here, if it helps next time. >>>> >>>> Re the actual application to be run, 'inversion', I have only >>>> received binaries. I used to run them on gainsborough (without mpi) >>>> and that worked fine. >>>> >>>> Home directories are not nfs shared, they are individual and >>>> separate, only the name is repeated. >>>> >>>> I had tried password free ssh in all directions and permutations. >>>> >>>> I had iptables down on sargeant and up on the other two. >>>> >>>> Yes, I installed the gcc-gfortran.x86_64 AFTER I took those >>>> screnshots, and the post install output was identical to the top one >>>> (sargeant). >>>> >>>> I am unsure about cpi and fortran... >>>> >>>> Stuff remaining to get sorted out: >>>> 1. Get that hyperthreading set up - what is your suggestions? Disable >>>> and let mpi manage the cores? >>> It depends on your applications, whether you can make use of it. Test >>> it with HT switched off, switched on and increase the number of >>> processes, until you see that any process is no longer running at 100% >>> and you judge that you don't tolerate this slow down. >>> >>> It's not uncommon to see an improvement up to 150% with HT turned on, >>> but not 200% (depending on the workload). >>> >>> >>>> 2. run mpiexec with iptables up, need to figure out what traffic to >>>> allow. >>> https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_How_do… >>> >>> >>> -- Reuti >>> >>> >>>> Great many thanks to all, and Gus, Reuti and Rajeev in particular. >>>> >>>> Cheers, >>>> Ron >>>> >>>> On 3/03/2014 10:09, Gustavo Correa wrote: >>>>> Hi Ron >>>>> >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Problems with running make
by Ron Palmer 03 Mar '14

03 Mar '14

Gus, thanks for your suggestions and links to more info; I was contemplating using any of the three unused ethernet interfaces at the back of each of the three computers, and using a cluster-only subnet for the inversion. You comments about a separate hostname and so forth are great, and I can re-activate iptables on those interfaces connected to the outside world. Thanks, Ron On 4/03/2014 08:54, Gus Correa wrote: > > On 03/03/2014 04:36 PM, Ron Palmer wrote: >> Thanks Reuti for your comments. I will peruse that FAQ detail. >> >> I just thought of the fact that each of these rack computers have 4 >> ethernet sockets, eth0 - eth3... I could connect the cluster on a >> separate ethernet sockets via an extra switch not connected to the >> internet or any other computers, and accept all communication among >> them, and keep iptables up on the ethx connected to the outside world. I >> guess I would have to set up routing tables or something. Ah, more >> reading :-) >> >> Thanks for your help. >> Ron >> > Hi Ron > > If those extra interfaces are not in use, > and if you have a spare switch, > you can setup a separate private subnet exclusively for MPI. > You need to configure the interfaces consistently (IP, subnet mask, > perhaps a gateway). Configuring them statically is easy: > > https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… > > > Use a subnet that doesn't intersect the existent/original IP range. > > http://en.wikipedia.org/wiki/Private_network > > You could also create host names associated to those IPs (say > node01, node02, node02), resolve them via /etc/hosts on each computer, > set passwordless ssh across these newly named "hosts". > This may simpler/safer than messing with the iptables. > > [Actually, the IP addresses you showed 192.168.X.Y, sound as a private > subnet already, not Internet, but that may be the subnet for your > organization/school/department already. So, you may set up a different > one on these three computers for MPI and very-local access.] > > OpenMPI allows you to choose the interface that it will use, > so you can direct it to your very-local subnet: > > http://www.open-mpi.org/faq/?category=tcp#tcp-selection > > I hope this helps, > Gus > > >> On 4/03/2014 00:13, Reuti wrote: >>> Am 03.03.2014 um 01:34 schrieb Ron Palmer: >>> >>>> Gus, >>>> I have just replied with the details of the success, but I will >>>> clarify your questions here, if it helps next time. >>>> >>>> Re the actual application to be run, 'inversion', I have only >>>> received binaries. I used to run them on gainsborough (without mpi) >>>> and that worked fine. >>>> >>>> Home directories are not nfs shared, they are individual and >>>> separate, only the name is repeated. >>>> >>>> I had tried password free ssh in all directions and permutations. >>>> >>>> I had iptables down on sargeant and up on the other two. >>>> >>>> Yes, I installed the gcc-gfortran.x86_64 AFTER I took those >>>> screnshots, and the post install output was identical to the top one >>>> (sargeant). >>>> >>>> I am unsure about cpi and fortran... >>>> >>>> Stuff remaining to get sorted out: >>>> 1. Get that hyperthreading set up - what is your suggestions? Disable >>>> and let mpi manage the cores? >>> It depends on your applications, whether you can make use of it. Test >>> it with HT switched off, switched on and increase the number of >>> processes, until you see that any process is no longer running at 100% >>> and you judge that you don't tolerate this slow down. >>> >>> It's not uncommon to see an improvement up to 150% with HT turned on, >>> but not 200% (depending on the workload). >>> >>> >>>> 2. run mpiexec with iptables up, need to figure out what traffic to >>>> allow. >>> https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_How_do… >>> >>> >>> >>> -- Reuti >>> >>> >>>> Great many thanks to all, and Gus, Reuti and Rajeev in particular. >>>> >>>> Cheers, >>>> Ron >>>> >>>> On 3/03/2014 10:09, Gustavo Correa wrote: >>>>> Hi Ron >>>>> >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- *Ron Palmer*MSc MBA. Principal Geophysicist ron.palmer(a)pgcgroup.com.au <mailto:[email protected]> 0413 579 099 07 3103 4963

1 0

Re: [mpich-discuss] Problems with running make
by Gus Correa 03 Mar '14

03 Mar '14

On 03/03/2014 04:36 PM, Ron Palmer wrote: > Thanks Reuti for your comments. I will peruse that FAQ detail. > > I just thought of the fact that each of these rack computers have 4 > ethernet sockets, eth0 - eth3... I could connect the cluster on a > separate ethernet sockets via an extra switch not connected to the > internet or any other computers, and accept all communication among > them, and keep iptables up on the ethx connected to the outside world. I > guess I would have to set up routing tables or something. Ah, more > reading :-) > > Thanks for your help. > Ron > Hi Ron If those extra interfaces are not in use, and if you have a spare switch, you can setup a separate private subnet exclusively for MPI. You need to configure the interfaces consistently (IP, subnet mask, perhaps a gateway). Configuring them statically is easy: https://access.redhat.com/site/documentation//en-US/Red_Hat_Enterprise_Linu… Use a subnet that doesn't intersect the existent/original IP range. http://en.wikipedia.org/wiki/Private_network You could also create host names associated to those IPs (say node01, node02, node02), resolve them via /etc/hosts on each computer, set passwordless ssh across these newly named "hosts". This may simpler/safer than messing with the iptables. [Actually, the IP addresses you showed 192.168.X.Y, sound as a private subnet already, not Internet, but that may be the subnet for your organization/school/department already. So, you may set up a different one on these three computers for MPI and very-local access.] OpenMPI allows you to choose the interface that it will use, so you can direct it to your very-local subnet: http://www.open-mpi.org/faq/?category=tcp#tcp-selection I hope this helps, Gus > On 4/03/2014 00:13, Reuti wrote: >> Am 03.03.2014 um 01:34 schrieb Ron Palmer: >> >>> Gus, >>> I have just replied with the details of the success, but I will >>> clarify your questions here, if it helps next time. >>> >>> Re the actual application to be run, 'inversion', I have only >>> received binaries. I used to run them on gainsborough (without mpi) >>> and that worked fine. >>> >>> Home directories are not nfs shared, they are individual and >>> separate, only the name is repeated. >>> >>> I had tried password free ssh in all directions and permutations. >>> >>> I had iptables down on sargeant and up on the other two. >>> >>> Yes, I installed the gcc-gfortran.x86_64 AFTER I took those >>> screnshots, and the post install output was identical to the top one >>> (sargeant). >>> >>> I am unsure about cpi and fortran... >>> >>> Stuff remaining to get sorted out: >>> 1. Get that hyperthreading set up - what is your suggestions? Disable >>> and let mpi manage the cores? >> It depends on your applications, whether you can make use of it. Test >> it with HT switched off, switched on and increase the number of >> processes, until you see that any process is no longer running at 100% >> and you judge that you don't tolerate this slow down. >> >> It's not uncommon to see an improvement up to 150% with HT turned on, >> but not 200% (depending on the workload). >> >> >>> 2. run mpiexec with iptables up, need to figure out what traffic to >>> allow. >> https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_How_do… >> >> >> -- Reuti >> >> >>> Great many thanks to all, and Gus, Reuti and Rajeev in particular. >>> >>> Cheers, >>> Ron >>> >>> On 3/03/2014 10:09, Gustavo Correa wrote: >>>> Hi Ron >>>> > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_init(377)... gethostbyname failed, mac (errno 1)
by Luis Cebamanos 03 Mar '14

03 Mar '14

Hi Reuti, I have done what you suggested and it WORKS with the wireless off lcebaman@mac~/$ mpiexec -iface lo0 -n 4 ./hello Hello world from process 0 of 4 Hello world from process 3 of 4 Hello world from process 1 of 4 Hello world from process 2 of 4 What is the problem here then? Cheers, Luis On 3 Mar 2014, at 14:54, Reuti <reuti(a)staff.uni-marburg.de> wrote: > Am 03.03.2014 um 11:29 schrieb Luis Cebamanos: > >> I have also noticed that working without any internet connection does not work either. > > Just for curiosity: > > $ mpiexec -iface lo0 -n 4 ./foo > > -- Reuti > > >> On 3 Mar 2014, at 09:53, Luis Cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: >> >>> Hi there, >>> >>> I’ve got more news. The last IP that I showed in my last email it turns out that works OKAY, however now that I am back to the office and working on the wireless, it does not work. And that is the result of nslookup: >>> >>> lcebaman@mac~/Projects/SPRINT$ nslookup localhost >>> Server: 129.215.205.191 >>> Address: 129.215.205.191#53 >>> >>> Name: localhost.ed.ac.uk >>> Address: 127.0.0.1 >>> >>> lcebaman@mac~/Projects/SPRINT$ nslookup mac >>> Server: 129.215.205.191 >>> Address: 129.215.205.191#53 >>> >>> ** server can't find mac: NXDOMAIN >>> >>> Any thoughts? >>> >>> Cheers, >>> Luis >>> >>> On 1 Mar 2014, at 19:40, l.cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: >>> >>>> Well, is not fixed. Its just the one I got here, but it changes. Any ideas of how to fix this? >>>> >>>> -------- Original message -------- >>>> From: Reuti <reuti(a)staff.uni-marburg.de> >>>> Date:01/03/2014 17:38 (GMT+00:00) >>>> To: discuss(a)mpich.org >>>> Subject: Re: [mpich-discuss] MPID_nem_tcp_init(377)... gethostbyname failed, >>>> mac (errno 1) >>>> >>>> Am 01.03.2014 um 17:58 schrieb Luis Cebamanos: >>>> >>>>> Here it is the output of those commands: >>>>> >>>>> lcebaman@mac~/$ nslookup localhost >>>>> Server: 192.168.1.1 >>>>> Address: 192.168.1.1#53 >>>>> >>>>> Name: localhost >>>>> Address: 127.0.0.1 >>>>> >>>>> lcebaman@mac~/$ nslookup mac >>>>> Server: 192.168.1.1 >>>>> Address: 192.168.1.1#53 >>>>> >>>>> Name: mac >>>>> Address: 192.168.1.127 >>>> >>>> Is this a fixed address you get everytime? You could add it to /etc/hosts. >>>> >>>> Interesting that MPICH throws an error about resolving "mac", although DNS is working. >>>> >>>> -- Reuti >>>> >>>> >>>>> lcebaman@mac~/$ hostname >>>>> mac >>>>> >>>>> >>>>> >>>>> >>>>> On 28 Feb 2014, at 23:35, Reuti <reuti(a)staff.uni-marburg.de> wrote: >>>>> >>>>>> Am 28.02.2014 um 14:39 schrieb Luis Cebamanos: >>>>>> >>>>>>> Hi Pavan, >>>>>>> >>>>>>> Yes, that is the output: >>>>>>> >>>>>>> $ cat /etc/hosts >>>>>>> ## >>>>>>> # Host Database >>>>>>> # >>>>>>> # localhost is used to configure the loopback interface >>>>>>> # when the system is booting. Do not change this entry. >>>>>>> ## >>>>>>> 127.0.0.1 localhost >>>>>>> 255.255.255.255 broadcasthost >>>>>>> ::1 localhost >>>>>>> fe80::1%lo0 localhost >>>>>> >>>>>> What is the output of: >>>>>> >>>>>> $ nslookup localhost >>>>>> $ nslookup mac >>>>>> $ hostname >>>>>> >>>>>> AFAICS your machine has no domainname, otherwise it would show up in the MPICH error message after "mac". >>>>>> >>>>>> -- Reuti >>>>>> >>>>>> >>>>>>> On 28/02/14 13:35, Balaji, Pavan wrote: >>>>>>>> $ cat /etc/hosts >>>>>>>>>> ## >>>>>>>>>> # Host Database >>>>>>>>>> # >>>>>>>>>> # localhost is used to configure the loopback interface >>>>>>>>>> # when the system is booting. Do not change this entry. >>>>>>>>>> ## >>>>>>>>>> 127.0.0.1 localhost >>>>>>>>>> 255.255.255.255 broadcasthost >>>>>>>>>> ::1 localhost >>>>>>>>>> fe80::1%lo0 localhost >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> The University of Edinburgh is a charitable body, registered in >>>>>>> Scotland, with registration number SC005336. >>>>>>> >>>>>>> _______________________________________________ >>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>> To manage subscription options or unsubscribe: >>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> The University of Edinburgh is a charitable body, registered in >>>>> Scotland, with registration number SC005336. >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

MPI_Publish_name
by Ramesh Vinayagam 03 Mar '14

03 Mar '14

Is it possible to use MPI_Publish_name for a server/client style approach? Server: MPI_Publish_name Client MPI_Lookup_name Thanks Ramesh

1 0

Re: [mpich-discuss] Problems with running make
by Ron Palmer 03 Mar '14

03 Mar '14

Thanks Reuti for your comments. I will peruse that FAQ detail. I just thought of the fact that each of these rack computers have 4 ethernet sockets, eth0 - eth3... I could connect the cluster on a separate ethernet sockets via an extra switch not connected to the internet or any other computers, and accept all communication among them, and keep iptables up on the ethx connected to the outside world. I guess I would have to set up routing tables or something. Ah, more reading :-) Thanks for your help. Ron On 4/03/2014 00:13, Reuti wrote: > Am 03.03.2014 um 01:34 schrieb Ron Palmer: > >> Gus, >> I have just replied with the details of the success, but I will clarify your questions here, if it helps next time. >> >> Re the actual application to be run, 'inversion', I have only received binaries. I used to run them on gainsborough (without mpi) and that worked fine. >> >> Home directories are not nfs shared, they are individual and separate, only the name is repeated. >> >> I had tried password free ssh in all directions and permutations. >> >> I had iptables down on sargeant and up on the other two. >> >> Yes, I installed the gcc-gfortran.x86_64 AFTER I took those screnshots, and the post install output was identical to the top one (sargeant). >> >> I am unsure about cpi and fortran... >> >> Stuff remaining to get sorted out: >> 1. Get that hyperthreading set up - what is your suggestions? Disable and let mpi manage the cores? > It depends on your applications, whether you can make use of it. Test it with HT switched off, switched on and increase the number of processes, until you see that any process is no longer running at 100% and you judge that you don't tolerate this slow down. > > It's not uncommon to see an improvement up to 150% with HT turned on, but not 200% (depending on the workload). > > >> 2. run mpiexec with iptables up, need to figure out what traffic to allow. > https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_How_do… > > -- Reuti > > >> Great many thanks to all, and Gus, Reuti and Rajeev in particular. >> >> Cheers, >> Ron >> >> On 3/03/2014 10:09, Gustavo Correa wrote: >>> Hi Ron >>>

1 0

Re: [mpich-discuss] very slow file writes independent of file size
by Wei-keng Liao 03 Mar '14

03 Mar '14

Hi, Geoffrey Edison has a Lustre file system and I assume your program writes files there. If this is the case, please check the file striping setting of your output files, using command "lfs getstripe filename" Usually, a high-performance I/O can be obtained if you have configured the lustre striping setting to a higher stripe_count (96 or 144 max on Edison). In addition, MPI_File_write_ordered is for shared file pointers. It can have some impact to the performance. Using independent file pointers often results in a better performance. You might want to consider to change your program to use those functions (eg. MPI_File_write_all). Wei-keng On Mar 3, 2014, at 12:21 AM, Geoffrey Irving wrote: > I'm doing postmortem on a 2048 node (16384 rank) job on Edison, and > trying to understand why my I/O performance might have been slow. > > Here's the data: > > Measured I/O bandwidth: > slice 35 write sparse bandwidth = 6082640 / (3.52519e+06 s / 16384) = > 2.63287e-05 GB/s > slice 34 write sparse bandwidth = 13824080 / (3.66608e+06 s / 16384) = > 5.75379e-05 GB/s > slice 33 write sparse bandwidth = 24754256 / (2.83647e+06 s / 16384) = > 0.000133166 GB/s > slice 32 write sparse bandwidth = 39370832 / (3.47016e+06 s / 16384) = > 0.000173119 GB/s > slice 31 write sparse bandwidth = 55812176 / (2.53623e+06 s / 16384) = > 0.000335785 GB/s > slice 30 write sparse bandwidth = 74741840 / (2.5714e+06 s / 16384) = > 0.00044352 GB/s > slice 29 write sparse bandwidth = 93560912 / (2.67336e+06 s / 16384) = > 0.000534019 GB/s > slice 28 write sparse bandwidth = 112803920 / (2.74639e+06 s / 16384) > = 0.000626733 GB/s > slice 27 write sparse bandwidth = 128194640 / (3.1603e+06 s / 16384) = > 0.000618958 GB/s > slice 26 write sparse bandwidth = 141281360 / (3.12754e+06 s / 16384) > = 0.00068929 GB/s > slice 25 write sparse bandwidth = 148193360 / (2.62376e+06 s / 16384) > = 0.000861835 GB/s > slice 24 write sparse bandwidth = 151861328 / (3.2145e+06 s / 16384) = > 0.000720865 GB/s > slice 23 write sparse bandwidth = 148193360 / (2.44736e+06 s / 16384) > = 0.000923956 GB/s > slice 22 write sparse bandwidth = 142055504 / (3.15962e+06 s / 16384) > = 0.000686031 GB/s > slice 21 write sparse bandwidth = 130388048 / (3.09774e+06 s / 16384) > = 0.000642263 GB/s > slice 20 write sparse bandwidth = 117964880 / (3.02676e+06 s / 16384) > = 0.000594696 GB/s > slice 19 write sparse bandwidth = 101560400 / (2.97198e+06 s / 16384) > = 0.000521434 GB/s > slice 18 write sparse bandwidth = 86372432 / (2.96247e+06 s / 16384) = > 0.000444878 GB/s > slice 18 write sections bandwidth = 1954957518434 / (1.83937e+07 s / > 16384) = 1.62177 GB/s > slice 17 write sparse bandwidth = 70170704 / (2.88973e+06 s / 16384) = > 0.000370526 GB/s > slice 17 write sections bandwidth = 1475380615039 / (1.36018e+07 s / > 16384) = 1.65511 GB/s > slice 17 read bandwidth (192 nodes) = 1475380615039 / 3383.36 s = 0.406122 GB/s > per node: measured = 2.16598 MB/s, theoretical peak = 33.1341 MB/s > > Focusing on the "sparse" lines, the main point is that the time seems > to be roughly independent of file size (plot attached). Each timing > sample consists of (1) setup which I believe is negligible, (2) > MPI_File_open, (3) MPI_File_write_ordered, (4) MPI_File_close. > > What might have caused these file writes to take so long? > > Geoffrey > <2014-03-02-221950_1220x500.png>_______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_init(377)... gethostbyname failed, mac (errno 1)
by Reuti 03 Mar '14

03 Mar '14

Am 03.03.2014 um 11:29 schrieb Luis Cebamanos: > I have also noticed that working without any internet connection does not work either. Just for curiosity: $ mpiexec -iface lo0 -n 4 ./foo -- Reuti > On 3 Mar 2014, at 09:53, Luis Cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: > >> Hi there, >> >> I’ve got more news. The last IP that I showed in my last email it turns out that works OKAY, however now that I am back to the office and working on the wireless, it does not work. And that is the result of nslookup: >> >> lcebaman@mac~/Projects/SPRINT$ nslookup localhost >> Server: 129.215.205.191 >> Address: 129.215.205.191#53 >> >> Name: localhost.ed.ac.uk >> Address: 127.0.0.1 >> >> lcebaman@mac~/Projects/SPRINT$ nslookup mac >> Server: 129.215.205.191 >> Address: 129.215.205.191#53 >> >> ** server can't find mac: NXDOMAIN >> >> Any thoughts? >> >> Cheers, >> Luis >> >> On 1 Mar 2014, at 19:40, l.cebamanos <l.cebamanos(a)epcc.ed.ac.uk> wrote: >> >>> Well, is not fixed. Its just the one I got here, but it changes. Any ideas of how to fix this? >>> >>> -------- Original message -------- >>> From: Reuti <reuti(a)staff.uni-marburg.de> >>> Date:01/03/2014 17:38 (GMT+00:00) >>> To: discuss(a)mpich.org >>> Subject: Re: [mpich-discuss] MPID_nem_tcp_init(377)... gethostbyname failed, >>> mac (errno 1) >>> >>> Am 01.03.2014 um 17:58 schrieb Luis Cebamanos: >>> >>>> Here it is the output of those commands: >>>> >>>> lcebaman@mac~/$ nslookup localhost >>>> Server: 192.168.1.1 >>>> Address: 192.168.1.1#53 >>>> >>>> Name: localhost >>>> Address: 127.0.0.1 >>>> >>>> lcebaman@mac~/$ nslookup mac >>>> Server: 192.168.1.1 >>>> Address: 192.168.1.1#53 >>>> >>>> Name: mac >>>> Address: 192.168.1.127 >>> >>> Is this a fixed address you get everytime? You could add it to /etc/hosts. >>> >>> Interesting that MPICH throws an error about resolving "mac", although DNS is working. >>> >>> -- Reuti >>> >>> >>>> lcebaman@mac~/$ hostname >>>> mac >>>> >>>> >>>> >>>> >>>> On 28 Feb 2014, at 23:35, Reuti <reuti(a)staff.uni-marburg.de> wrote: >>>> >>>>> Am 28.02.2014 um 14:39 schrieb Luis Cebamanos: >>>>> >>>>>> Hi Pavan, >>>>>> >>>>>> Yes, that is the output: >>>>>> >>>>>> $ cat /etc/hosts >>>>>> ## >>>>>> # Host Database >>>>>> # >>>>>> # localhost is used to configure the loopback interface >>>>>> # when the system is booting. Do not change this entry. >>>>>> ## >>>>>> 127.0.0.1 localhost >>>>>> 255.255.255.255 broadcasthost >>>>>> ::1 localhost >>>>>> fe80::1%lo0 localhost >>>>> >>>>> What is the output of: >>>>> >>>>> $ nslookup localhost >>>>> $ nslookup mac >>>>> $ hostname >>>>> >>>>> AFAICS your machine has no domainname, otherwise it would show up in the MPICH error message after "mac". >>>>> >>>>> -- Reuti >>>>> >>>>> >>>>>> On 28/02/14 13:35, Balaji, Pavan wrote: >>>>>>> $ cat /etc/hosts >>>>>>>>> ## >>>>>>>>> # Host Database >>>>>>>>> # >>>>>>>>> # localhost is used to configure the loopback interface >>>>>>>>> # when the system is booting. Do not change this entry. >>>>>>>>> ## >>>>>>>>> 127.0.0.1 localhost >>>>>>>>> 255.255.255.255 broadcasthost >>>>>>>>> ::1 localhost >>>>>>>>> fe80::1%lo0 localhost >>>>>> >>>>>> >>>>>> -- >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0