March 2016 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] an error in install mpich( HYDU_sock_is_local (/home/mpi/mpich-3.2/src/pm/hydra/utils/sock/sock.c:532): gethostbyname error on mpi1: Host name lookup failure)
by ll 20 Mar '16

20 Mar '16

Thanks. But my problems is still not sloved. I always use ssh to manage my server. But the problem is still unsloved whether I executed the command "mpiexec -launcher ssh -f machinefile -n 3 hostname" by ssh or by terminal. ps:My system is centos 6.5. At 2016-03-20 13:30:18, "sanjesh pant" <spant3474(a)gmail.com> wrote: Yes i had got that problem during execution What you are using rsh or ssh? As i was using rsh and using it i execute as mpiexec -launcher rsh -f machinefile -n 3 ./a.out a.out is executable file or object file after compilation On 20 Mar 2016 09:36, "ll" <baibai1983(a)126.com> wrote: Hi everyone： I am a beginner in using mpich. There was an error in installing mpich 3.2. when I ran command “ mpiexec -f machinefile -n 3 hostname", there was a error which said that: [mpiexec@mpi1] HYDU_sock_is_local (/home/mpi/mpich-3.2/src/pm/hydra/utils/sock/sock.c:532): gethostbyname error on mpi1: Host name lookup failure [mpiexec@mpi1] main (/home/mpi/mpich-3.2/src/pm/hydra/ui/mpich/mpiexec.c:291): unable to check if www.plaust.edu.cn is local There was only one line in my machinefile: mpi2 And when I tried to ping mpi2, there was an respond: [mpi@mpi1 ~]$ ping mpi2 PING mpi2 (172.18.0.217) 56(84) bytes of data. 64 bytes from mpi2 (172.18.0.217): icmp_seq=1 ttl=64 time=1.38 ms 64 bytes from mpi2 (172.18.0.217): icmp_seq=2 ttl=64 time=0.263 ms Is there anyone who have encountered this problem? _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] an error in install mpich( HYDU_sock_is_local (/home/mpi/mpich-3.2/src/pm/hydra/utils/sock/sock.c:532): gethostbyname error on mpi1: Host name lookup failure)
by sanjesh pant 20 Mar '16

20 Mar '16

Yes i had got that problem during execution What you are using rsh or ssh? As i was using rsh and using it i execute as mpiexec -launcher rsh -f machinefile -n 3 ./a.out a.out is executable file or object file after compilation On 20 Mar 2016 09:36, "ll" <baibai1983(a)126.com> wrote: > Hi everyone： > I am a beginner in using mpich. There was an error in installing mpich > 3.2. when I ran command “ mpiexec -f machinefile -n 3 hostname", there was > a error which said that: > [mpiexec@mpi1] HYDU_sock_is_local > (/home/mpi/mpich-3.2/src/pm/hydra/utils/sock/sock.c:532): gethostbyname > error on mpi1: Host name lookup failure > [mpiexec@mpi1] main > (/home/mpi/mpich-3.2/src/pm/hydra/ui/mpich/mpiexec.c:291): unable to check > if www.plaust.edu.cn is local > > There was only one line in my machinefile: > mpi2 > > > And when I tried to ping mpi2, there was an respond: > [mpi@mpi1 ~]$ ping mpi2 > PING mpi2 (172.18.0.217) 56(84) bytes of data. > 64 bytes from mpi2 (172.18.0.217): icmp_seq=1 ttl=64 time=1.38 ms > 64 bytes from mpi2 (172.18.0.217): icmp_seq=2 ttl=64 time=0.263 ms > > Is there anyone who have encountered this problem? > > > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

an error in install mpich( HYDU_sock_is_local (/home/mpi/mpich-3.2/src/pm/hydra/utils/sock/sock.c:532): gethostbyname error on mpi1: Host name lookup failure)
by ll 19 Mar '16

19 Mar '16

Hi everyone： I am a beginner in using mpich. There was an error in installing mpich 3.2. when I ran command “ mpiexec -f machinefile -n 3 hostname", there was a error which said that: [mpiexec@mpi1] HYDU_sock_is_local (/home/mpi/mpich-3.2/src/pm/hydra/utils/sock/sock.c:532): gethostbyname error on mpi1: Host name lookup failure [mpiexec@mpi1] main (/home/mpi/mpich-3.2/src/pm/hydra/ui/mpich/mpiexec.c:291): unable to check if www.plaust.edu.cn is local There was only one line in my machinefile: mpi2 And when I tried to ping mpi2, there was an respond: [mpi@mpi1 ~]$ ping mpi2 PING mpi2 (172.18.0.217) 56(84) bytes of data. 64 bytes from mpi2 (172.18.0.217): icmp_seq=1 ttl=64 time=1.38 ms 64 bytes from mpi2 (172.18.0.217): icmp_seq=2 ttl=64 time=0.263 ms Is there anyone who have encountered this problem? _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_connpoll(1835): Communication error with rank 1: Connection timed out
by Kenneth Raffenetti 18 Mar '16

18 Mar '16

Thanks for this Amelie. I've created a wiki page from your instructions and linked it from the FAQ. https://wiki.mpich.org/mpich/index.php/Using_MPICH_in_Amazon_EC2 Ken On 03/18/2016 02:09 AM, amelie chi zhou wrote: > Hi, Pavan, > > Attached is a little summary on "How to run MPICH on Amazon EC2". I'm > not sure whether it's clear enough. Please check. > > Regards, > Amelie > > On Thu, Mar 17, 2016 at 9:22 AM, amelie chi zhou <amelie.czhou(a)gmail.com > <mailto:[email protected]>> wrote: > > Sure. I'm glad to. > > Amelie > > > On 17 Mar 2016, at 1:58 AM, Balaji, Pavan <balaji(a)anl.gov > <mailto:[email protected]>> wrote: > > > > Hi Amelie, > > > > Would you be willing to write up some documentation on "How to > use MPICH on Amazon EC2" including details on using servers in a > single region vs. multiple regions? We'd like to put this up on our > FAQ page. > > > > Thanks, > > > > -- Pavan > > > >> On Mar 16, 2016, at 2:53 AM, amelie chi zhou > <amelie.czhou(a)gmail.com <mailto:[email protected]>> wrote: > >> > >> Hi, Pavan, > >> > >> Thanks a lot. It does work now! > >> > >> Best Regards, > >> Amelie > >> > >> On Wed, Mar 16, 2016 at 3:26 PM, Balaji, Pavan <balaji(a)anl.gov > <mailto:[email protected]>> wrote: > >> Amelie, > >> > >> OK, I just tried it across two different subnets. Here's the > problem -- > >> > >> The amazon compute nodes hide their public IP addresses from the > list of IP addresses visible locally. So when each node queries for > its local IP address, it only gets its private IP address (which is > obviously useless for other processes to connect). > >> > >> You can workaround this by making two changes to your environment: > >> > >> 1. Explicitly use the public IP addresses directly instead of > the hostnames in your hostfile. That is, instead of > "ec2-52-36-15-57.us-west-2.compute.amazonaws.com > <http://ec2-52-36-15-57.us-west-2.compute.amazonaws.com>", use > "52.36.15.57". > >> > >> 2. Pass the -localhost option to mpiexec to give the public IP > address of the host from which you are running mpiexec. > >> > >> I created two VM instances, one on the west subnet and the other > on the east subnet: > >> > >> ec2-52-35-56-228.us-west-2.compute.amazonaws.com > <http://ec2-52-35-56-228.us-west-2.compute.amazonaws.com> > >> ec2-54-172-35-159.compute-1.amazonaws.com > <http://ec2-54-172-35-159.compute-1.amazonaws.com> > >> > >> To run my application, I do this: > >> > >> % ./install/bin/mpiexec -localhost 52.35.56.228 -hosts > 52.35.56.228,54.172.35.159 -n 2 ./examples/cpi > >> > >> Let us know if that works. > >> > >> -- Pavan > >> > >>> On Mar 16, 2016, at 12:24 AM, amelie chi zhou > <amelie.czhou(a)gmail.com <mailto:[email protected]>> wrote: > >>> > >>> Hi, Pavan, > >>> > >>> Thanks a lot for trying that. > >>> > >>> I have enabled inbound traffic for all types of protocols > including tcp, udp and icmp for all ports (0 - 65535) and for all ip > addresses. I noticed that the two instances you created are from the > same region (us west I suppose). The thing is, for instances in the > same region, mpiexec can run successfully with no problem in my > setup. But when I run mpi programs across regions, in my case, > between an instance in us east and an instance in us west, the error > in MPI_Send appears. > >>> It seems that there might be some problems with the firewall or > network interfaces, but I have checked and ruled out those > possibilities (instances in different regions can ssh and scp to > each other and there's no dropping rule in my firewall setting). So > that's where I'm confused. > >>> > >>> Regards, > >>> Amelie > >>> > >>>> On Wed, Mar 16, 2016 at 1:01 PM, Balaji, Pavan <balaji(a)anl.gov > <mailto:[email protected]>> wrote: > >>>> > >>>> On Mar 15, 2016, at 10:26 PM, amelie chi zhou > <amelie.czhou(a)gmail.com <mailto:[email protected]>> wrote: > >>>> Here is the full output info. Thanks! > >>> > >>> The IP addresses and ports seem to be correctly setup, so > that's not the problem. > >>> > >>> I created my own amazon instances to see what the problem is. > It looks like the instances are not able to communicate even though > there's no explicit firewall enabled that is shown inside the Linux > instance. I did some digging and found the "Security group" > settings and found that the inbound rules only allowed ssh. I > changed it to "All traffic" and now I can run my jobs fine. > >>> > >>> % ./install/bin/mpiexec -hosts > ec2-52-36-15-57.us-west-2.compute.amazonaws.com > <http://ec2-52-36-15-57.us-west-2.compute.amazonaws.com>,ec2-5 > >>> 2-37-222-189.us-west-2.compute.amazonaws.com > <http://2-37-222-189.us-west-2.compute.amazonaws.com> -n 4 > ./examples/cpi > >>> Process 3 of 4 is on ip-172-31-28-127 > >>> Process 2 of 4 is on ip-172-31-21-12 > >>> Process 1 of 4 is on ip-172-31-28-127 > >>> Process 0 of 4 is on ip-172-31-21-12 > >>> pi is approximately 3.1415926544231243, Error is 0.0000000008333312 > >>> wall clock time = 0.010181 > >>> > >>> Can you try that? > >>> > >>> -- Pavan > >>> > >>> _______________________________________________ > >>> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > >>> To manage subscription options or unsubscribe: > >>> https://lists.mpich.org/mailman/listinfo/discuss > >>> > >>> _______________________________________________ > >>> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > >>> To manage subscription options or unsubscribe: > >>> https://lists.mpich.org/mailman/listinfo/discuss > >> > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > >> > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_connpoll(1835): Communication error with rank 1: Connection timed out
by amelie chi zhou 18 Mar '16

18 Mar '16

Hi, Pavan, Attached is a little summary on "How to run MPICH on Amazon EC2". I'm not sure whether it's clear enough. Please check. Regards, Amelie On Thu, Mar 17, 2016 at 9:22 AM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: > Sure. I'm glad to. > > Amelie > > > On 17 Mar 2016, at 1:58 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > > > Hi Amelie, > > > > Would you be willing to write up some documentation on "How to use MPICH > on Amazon EC2" including details on using servers in a single region vs. > multiple regions? We'd like to put this up on our FAQ page. > > > > Thanks, > > > > -- Pavan > > > >> On Mar 16, 2016, at 2:53 AM, amelie chi zhou <amelie.czhou(a)gmail.com> > wrote: > >> > >> Hi, Pavan, > >> > >> Thanks a lot. It does work now! > >> > >> Best Regards, > >> Amelie > >> > >> On Wed, Mar 16, 2016 at 3:26 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > >> Amelie, > >> > >> OK, I just tried it across two different subnets. Here's the problem -- > >> > >> The amazon compute nodes hide their public IP addresses from the list > of IP addresses visible locally. So when each node queries for its local > IP address, it only gets its private IP address (which is obviously useless > for other processes to connect). > >> > >> You can workaround this by making two changes to your environment: > >> > >> 1. Explicitly use the public IP addresses directly instead of the > hostnames in your hostfile. That is, instead of " > ec2-52-36-15-57.us-west-2.compute.amazonaws.com", use "52.36.15.57". > >> > >> 2. Pass the -localhost option to mpiexec to give the public IP address > of the host from which you are running mpiexec. > >> > >> I created two VM instances, one on the west subnet and the other on the > east subnet: > >> > >> ec2-52-35-56-228.us-west-2.compute.amazonaws.com > >> ec2-54-172-35-159.compute-1.amazonaws.com > >> > >> To run my application, I do this: > >> > >> % ./install/bin/mpiexec -localhost 52.35.56.228 -hosts > 52.35.56.228,54.172.35.159 -n 2 ./examples/cpi > >> > >> Let us know if that works. > >> > >> -- Pavan > >> > >>> On Mar 16, 2016, at 12:24 AM, amelie chi zhou <amelie.czhou(a)gmail.com> > wrote: > >>> > >>> Hi, Pavan, > >>> > >>> Thanks a lot for trying that. > >>> > >>> I have enabled inbound traffic for all types of protocols including > tcp, udp and icmp for all ports (0 - 65535) and for all ip addresses. I > noticed that the two instances you created are from the same region (us > west I suppose). The thing is, for instances in the same region, mpiexec > can run successfully with no problem in my setup. But when I run mpi > programs across regions, in my case, between an instance in us east and an > instance in us west, the error in MPI_Send appears. > >>> It seems that there might be some problems with the firewall or > network interfaces, but I have checked and ruled out those possibilities > (instances in different regions can ssh and scp to each other and there's > no dropping rule in my firewall setting). So that's where I'm confused. > >>> > >>> Regards, > >>> Amelie > >>> > >>>> On Wed, Mar 16, 2016 at 1:01 PM, Balaji, Pavan <balaji(a)anl.gov> > wrote: > >>>> > >>>> On Mar 15, 2016, at 10:26 PM, amelie chi zhou <amelie.czhou(a)gmail.com> > wrote: > >>>> Here is the full output info. Thanks! > >>> > >>> The IP addresses and ports seem to be correctly setup, so that's not > the problem. > >>> > >>> I created my own amazon instances to see what the problem is. It > looks like the instances are not able to communicate even though there's no > explicit firewall enabled that is shown inside the Linux instance. I did > some digging and found the "Security group" settings and found that the > inbound rules only allowed ssh. I changed it to "All traffic" and now I > can run my jobs fine. > >>> > >>> % ./install/bin/mpiexec -hosts > ec2-52-36-15-57.us-west-2.compute.amazonaws.com,ec2-5 > >>> 2-37-222-189.us-west-2.compute.amazonaws.com -n 4 ./examples/cpi > >>> Process 3 of 4 is on ip-172-31-28-127 > >>> Process 2 of 4 is on ip-172-31-21-12 > >>> Process 1 of 4 is on ip-172-31-28-127 > >>> Process 0 of 4 is on ip-172-31-21-12 > >>> pi is approximately 3.1415926544231243, Error is 0.0000000008333312 > >>> wall clock time = 0.010181 > >>> > >>> Can you try that? > >>> > >>> -- Pavan > >>> > >>> _______________________________________________ > >>> discuss mailing list discuss(a)mpich.org > >>> To manage subscription options or unsubscribe: > >>> https://lists.mpich.org/mailman/listinfo/discuss > >>> > >>> _______________________________________________ > >>> discuss mailing list discuss(a)mpich.org > >>> To manage subscription options or unsubscribe: > >>> https://lists.mpich.org/mailman/listinfo/discuss > >> > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > >> > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_connpoll(1835): Communication error with rank 1: Connection timed out
by amelie chi zhou 16 Mar '16

16 Mar '16

Sure. I'm glad to. Amelie > On 17 Mar 2016, at 1:58 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > Hi Amelie, > > Would you be willing to write up some documentation on "How to use MPICH on Amazon EC2" including details on using servers in a single region vs. multiple regions? We'd like to put this up on our FAQ page. > > Thanks, > > -- Pavan > >> On Mar 16, 2016, at 2:53 AM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: >> >> Hi, Pavan, >> >> Thanks a lot. It does work now! >> >> Best Regards, >> Amelie >> >> On Wed, Mar 16, 2016 at 3:26 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >> Amelie, >> >> OK, I just tried it across two different subnets. Here's the problem -- >> >> The amazon compute nodes hide their public IP addresses from the list of IP addresses visible locally. So when each node queries for its local IP address, it only gets its private IP address (which is obviously useless for other processes to connect). >> >> You can workaround this by making two changes to your environment: >> >> 1. Explicitly use the public IP addresses directly instead of the hostnames in your hostfile. That is, instead of "ec2-52-36-15-57.us-west-2.compute.amazonaws.com", use "52.36.15.57". >> >> 2. Pass the -localhost option to mpiexec to give the public IP address of the host from which you are running mpiexec. >> >> I created two VM instances, one on the west subnet and the other on the east subnet: >> >> ec2-52-35-56-228.us-west-2.compute.amazonaws.com >> ec2-54-172-35-159.compute-1.amazonaws.com >> >> To run my application, I do this: >> >> % ./install/bin/mpiexec -localhost 52.35.56.228 -hosts 52.35.56.228,54.172.35.159 -n 2 ./examples/cpi >> >> Let us know if that works. >> >> -- Pavan >> >>> On Mar 16, 2016, at 12:24 AM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: >>> >>> Hi, Pavan, >>> >>> Thanks a lot for trying that. >>> >>> I have enabled inbound traffic for all types of protocols including tcp, udp and icmp for all ports (0 - 65535) and for all ip addresses. I noticed that the two instances you created are from the same region (us west I suppose). The thing is, for instances in the same region, mpiexec can run successfully with no problem in my setup. But when I run mpi programs across regions, in my case, between an instance in us east and an instance in us west, the error in MPI_Send appears. >>> It seems that there might be some problems with the firewall or network interfaces, but I have checked and ruled out those possibilities (instances in different regions can ssh and scp to each other and there's no dropping rule in my firewall setting). So that's where I'm confused. >>> >>> Regards, >>> Amelie >>> >>>> On Wed, Mar 16, 2016 at 1:01 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>> >>>> On Mar 15, 2016, at 10:26 PM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: >>>> Here is the full output info. Thanks! >>> >>> The IP addresses and ports seem to be correctly setup, so that's not the problem. >>> >>> I created my own amazon instances to see what the problem is. It looks like the instances are not able to communicate even though there's no explicit firewall enabled that is shown inside the Linux instance. I did some digging and found the "Security group" settings and found that the inbound rules only allowed ssh. I changed it to "All traffic" and now I can run my jobs fine. >>> >>> % ./install/bin/mpiexec -hosts ec2-52-36-15-57.us-west-2.compute.amazonaws.com,ec2-5 >>> 2-37-222-189.us-west-2.compute.amazonaws.com -n 4 ./examples/cpi >>> Process 3 of 4 is on ip-172-31-28-127 >>> Process 2 of 4 is on ip-172-31-21-12 >>> Process 1 of 4 is on ip-172-31-28-127 >>> Process 0 of 4 is on ip-172-31-21-12 >>> pi is approximately 3.1415926544231243, Error is 0.0000000008333312 >>> wall clock time = 0.010181 >>> >>> Can you try that? >>> >>> -- Pavan >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_connpoll(1835): Communication error with rank 1: Connection timed out
by Balaji, Pavan 16 Mar '16

16 Mar '16

Hi Amelie, Would you be willing to write up some documentation on "How to use MPICH on Amazon EC2" including details on using servers in a single region vs. multiple regions? We'd like to put this up on our FAQ page. Thanks, -- Pavan > On Mar 16, 2016, at 2:53 AM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: > > Hi, Pavan, > > Thanks a lot. It does work now! > > Best Regards, > Amelie > > On Wed, Mar 16, 2016 at 3:26 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > Amelie, > > OK, I just tried it across two different subnets. Here's the problem -- > > The amazon compute nodes hide their public IP addresses from the list of IP addresses visible locally. So when each node queries for its local IP address, it only gets its private IP address (which is obviously useless for other processes to connect). > > You can workaround this by making two changes to your environment: > > 1. Explicitly use the public IP addresses directly instead of the hostnames in your hostfile. That is, instead of "ec2-52-36-15-57.us-west-2.compute.amazonaws.com", use "52.36.15.57". > > 2. Pass the -localhost option to mpiexec to give the public IP address of the host from which you are running mpiexec. > > I created two VM instances, one on the west subnet and the other on the east subnet: > > ec2-52-35-56-228.us-west-2.compute.amazonaws.com > ec2-54-172-35-159.compute-1.amazonaws.com > > To run my application, I do this: > > % ./install/bin/mpiexec -localhost 52.35.56.228 -hosts 52.35.56.228,54.172.35.159 -n 2 ./examples/cpi > > Let us know if that works. > > -- Pavan > > > On Mar 16, 2016, at 12:24 AM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: > > > > Hi, Pavan, > > > > Thanks a lot for trying that. > > > > I have enabled inbound traffic for all types of protocols including tcp, udp and icmp for all ports (0 - 65535) and for all ip addresses. I noticed that the two instances you created are from the same region (us west I suppose). The thing is, for instances in the same region, mpiexec can run successfully with no problem in my setup. But when I run mpi programs across regions, in my case, between an instance in us east and an instance in us west, the error in MPI_Send appears. > > It seems that there might be some problems with the firewall or network interfaces, but I have checked and ruled out those possibilities (instances in different regions can ssh and scp to each other and there's no dropping rule in my firewall setting). So that's where I'm confused. > > > > Regards, > > Amelie > > > > On Wed, Mar 16, 2016 at 1:01 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > > > > On Mar 15, 2016, at 10:26 PM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: > > > Here is the full output info. Thanks! > > > > The IP addresses and ports seem to be correctly setup, so that's not the problem. > > > > I created my own amazon instances to see what the problem is. It looks like the instances are not able to communicate even though there's no explicit firewall enabled that is shown inside the Linux instance. I did some digging and found the "Security group" settings and found that the inbound rules only allowed ssh. I changed it to "All traffic" and now I can run my jobs fine. > > > > % ./install/bin/mpiexec -hosts ec2-52-36-15-57.us-west-2.compute.amazonaws.com,ec2-5 > > 2-37-222-189.us-west-2.compute.amazonaws.com -n 4 ./examples/cpi > > Process 3 of 4 is on ip-172-31-28-127 > > Process 2 of 4 is on ip-172-31-21-12 > > Process 1 of 4 is on ip-172-31-28-127 > > Process 0 of 4 is on ip-172-31-21-12 > > pi is approximately 3.1415926544231243, Error is 0.0000000008333312 > > wall clock time = 0.010181 > > > > Can you try that? > > > > -- Pavan > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Error with MPI_Register_datarep
by Éric Chamberland 16 Mar '16

16 Mar '16

Le 16-03-13 18:57, Thakur, Rajeev a écrit : > To get file portability, most users use some higher-level library such as PnetCDF or HDF-5. Maybe you could write a small utility for the 32-bit users to do the data conversion before using the data. I know but please see my answer here: https://www.open-mpi.org/community/lists/users/2016/03/28711.php >> Maybe I can contribute? :) > Code contributions are always welcome :-). I will note that MPICH didn’t support the external32 file format for many years until Intel contributed code for it. Great, that is the way to go! I will have a look at the external32 implementation into ROMIO, that will help me understand how to add user datarep which will be very usefull for me. I hope to have your help to guide me! Thanks, Eric > Rajeev > > >> On Mar 13, 2016, at 4:49 PM, Eric Chamberland <Eric.Chamberland(a)giref.ulaval.ca> wrote: >> >> Le 2016-03-13 16:22, Thakur, Rajeev a écrit : >>> In the 19 years since the MPI-2 standard was released, you may be the first one requesting this functionality :-). I am curious what are you using it for? >> Here is the reason.... >> >> In 2012, when I read your book about MPI-2, I wrote all our I/O functions to use collective MPI I/O calls with "native" as datarep. I knew it would not be portable "as is" but the main usage is 64bits native format... then I told myself "It will be ok when I will want to read/write from 32 bits architectures or different endian style, since there is that user defined datarep possibility, as illustrated in chapet 9.1 of the book...", but unfortunately for me, I didn't _tested_ the functionality in 2012... :/ >> >> So we moved all our file formats (meshes, results, etc) to be aligned with the output of "native" datarep into a 64bits standard linux x86 architecture... That means we have produced, for example, 1200 validations tests with input/output files into that format... >> >> That is, of course, excluding all user data since 4 years.. >> >> No I am in the phase of porting the code to fully support 32bits architecture (we have an 32bit ubuntu running here, and working on windows port). >> >> So, I did test for the first time the datarep functionality to be able to convert 64 bits "long int" from/to 32 bits "long int" for writing/reading in respect to existing file format... >> >> It was all an easy task in my mind with the user datarep functionality just as it is in the standard.. >> >> But it appears I am the only one in the world (wow...) to request a way to have his files portable with MPI... without performance losses for the "main" usage which is 64bits... >> >> Maybe there is another simple way to do this? >> >> Anyway, I am now expecting that you will tell me it will never be implemented _just for me_... :/ >> >> Maybe I can contribute? :) >> >> Thanks, >> >> Eric >> >>> Rajeev >>> >>> >>>> On Mar 13, 2016, at 1:51 PM, Eric Chamberland <Eric.Chamberland(a)giref.ulaval.ca> wrote: >>>> >>>> Le 2016-03-10 18:04, Thakur, Rajeev a écrit : >>>>> Support for user-defined data representations has not been added to MPICH yet. There has not been much demand for it. >>>> Thanks for your answer... >>>> >>>> Is it a "big" task? >>>> >>>> Should it be scheduled so it may appear in a forthcoming version? >>>> >>>> Thanks, >>>> >>>> Eric >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_connpoll(1835): Communication error with rank 1: Connection timed out
by amelie chi zhou 16 Mar '16

16 Mar '16

Hi, Pavan, Thanks a lot. It does work now! Best Regards, Amelie On Wed, Mar 16, 2016 at 3:26 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > Amelie, > > OK, I just tried it across two different subnets. Here's the problem -- > > The amazon compute nodes hide their public IP addresses from the list of > IP addresses visible locally. So when each node queries for its local IP > address, it only gets its private IP address (which is obviously useless > for other processes to connect). > > You can workaround this by making two changes to your environment: > > 1. Explicitly use the public IP addresses directly instead of the > hostnames in your hostfile. That is, instead of " > ec2-52-36-15-57.us-west-2.compute.amazonaws.com", use "52.36.15.57". > > 2. Pass the -localhost option to mpiexec to give the public IP address of > the host from which you are running mpiexec. > > I created two VM instances, one on the west subnet and the other on the > east subnet: > > ec2-52-35-56-228.us-west-2.compute.amazonaws.com > ec2-54-172-35-159.compute-1.amazonaws.com > > To run my application, I do this: > > % ./install/bin/mpiexec -localhost 52.35.56.228 -hosts > 52.35.56.228,54.172.35.159 -n 2 ./examples/cpi > > Let us know if that works. > > -- Pavan > > > On Mar 16, 2016, at 12:24 AM, amelie chi zhou <amelie.czhou(a)gmail.com> > wrote: > > > > Hi, Pavan, > > > > Thanks a lot for trying that. > > > > I have enabled inbound traffic for all types of protocols including tcp, > udp and icmp for all ports (0 - 65535) and for all ip addresses. I noticed > that the two instances you created are from the same region (us west I > suppose). The thing is, for instances in the same region, mpiexec can run > successfully with no problem in my setup. But when I run mpi programs > across regions, in my case, between an instance in us east and an instance > in us west, the error in MPI_Send appears. > > It seems that there might be some problems with the firewall or network > interfaces, but I have checked and ruled out those possibilities (instances > in different regions can ssh and scp to each other and there's no dropping > rule in my firewall setting). So that's where I'm confused. > > > > Regards, > > Amelie > > > > On Wed, Mar 16, 2016 at 1:01 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > > > > On Mar 15, 2016, at 10:26 PM, amelie chi zhou <amelie.czhou(a)gmail.com> > wrote: > > > Here is the full output info. Thanks! > > > > The IP addresses and ports seem to be correctly setup, so that's not the > problem. > > > > I created my own amazon instances to see what the problem is. It looks > like the instances are not able to communicate even though there's no > explicit firewall enabled that is shown inside the Linux instance. I did > some digging and found the "Security group" settings and found that the > inbound rules only allowed ssh. I changed it to "All traffic" and now I > can run my jobs fine. > > > > % ./install/bin/mpiexec -hosts > ec2-52-36-15-57.us-west-2.compute.amazonaws.com,ec2-5 > > 2-37-222-189.us-west-2.compute.amazonaws.com -n 4 ./examples/cpi > > Process 3 of 4 is on ip-172-31-28-127 > > Process 2 of 4 is on ip-172-31-21-12 > > Process 1 of 4 is on ip-172-31-28-127 > > Process 0 of 4 is on ip-172-31-21-12 > > pi is approximately 3.1415926544231243, Error is 0.0000000008333312 > > wall clock time = 0.010181 > > > > Can you try that? > > > > -- Pavan > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPID_nem_tcp_connpoll(1835): Communication error with rank 1: Connection timed out
by Balaji, Pavan 16 Mar '16

16 Mar '16

Amelie, OK, I just tried it across two different subnets. Here's the problem -- The amazon compute nodes hide their public IP addresses from the list of IP addresses visible locally. So when each node queries for its local IP address, it only gets its private IP address (which is obviously useless for other processes to connect). You can workaround this by making two changes to your environment: 1. Explicitly use the public IP addresses directly instead of the hostnames in your hostfile. That is, instead of "ec2-52-36-15-57.us-west-2.compute.amazonaws.com", use "52.36.15.57". 2. Pass the -localhost option to mpiexec to give the public IP address of the host from which you are running mpiexec. I created two VM instances, one on the west subnet and the other on the east subnet: ec2-52-35-56-228.us-west-2.compute.amazonaws.com ec2-54-172-35-159.compute-1.amazonaws.com To run my application, I do this: % ./install/bin/mpiexec -localhost 52.35.56.228 -hosts 52.35.56.228,54.172.35.159 -n 2 ./examples/cpi Let us know if that works. -- Pavan > On Mar 16, 2016, at 12:24 AM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: > > Hi, Pavan, > > Thanks a lot for trying that. > > I have enabled inbound traffic for all types of protocols including tcp, udp and icmp for all ports (0 - 65535) and for all ip addresses. I noticed that the two instances you created are from the same region (us west I suppose). The thing is, for instances in the same region, mpiexec can run successfully with no problem in my setup. But when I run mpi programs across regions, in my case, between an instance in us east and an instance in us west, the error in MPI_Send appears. > It seems that there might be some problems with the firewall or network interfaces, but I have checked and ruled out those possibilities (instances in different regions can ssh and scp to each other and there's no dropping rule in my firewall setting). So that's where I'm confused. > > Regards, > Amelie > > On Wed, Mar 16, 2016 at 1:01 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > > On Mar 15, 2016, at 10:26 PM, amelie chi zhou <amelie.czhou(a)gmail.com> wrote: > > Here is the full output info. Thanks! > > The IP addresses and ports seem to be correctly setup, so that's not the problem. > > I created my own amazon instances to see what the problem is. It looks like the instances are not able to communicate even though there's no explicit firewall enabled that is shown inside the Linux instance. I did some digging and found the "Security group" settings and found that the inbound rules only allowed ssh. I changed it to "All traffic" and now I can run my jobs fine. > > % ./install/bin/mpiexec -hosts ec2-52-36-15-57.us-west-2.compute.amazonaws.com,ec2-5 > 2-37-222-189.us-west-2.compute.amazonaws.com -n 4 ./examples/cpi > Process 3 of 4 is on ip-172-31-28-127 > Process 2 of 4 is on ip-172-31-21-12 > Process 1 of 4 is on ip-172-31-28-127 > Process 0 of 4 is on ip-172-31-21-12 > pi is approximately 3.1415926544231243, Error is 0.0000000008333312 > wall clock time = 0.010181 > > Can you try that? > > -- Pavan > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0