- discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] mpich hangs
by Pavan Balaji 27 Jun '13

27 Jun '13

IIRC, Ubuntu has a firewall setup by default. Try this: % sudo ufw status -- Pavan On 06/27/2013 09:48 PM, Syed. Jahanzeb Maqbool Hashmi wrote: > > As I already read some discussion on the mpich thread and I came to know > these questions before: > > * *Can you ssh from "node01" to "node02"?* > Yes I have passwordless ssh from each node to other (and vice versa) > * *Make sure the firewalls are turned off on all machines.* > I am using Ubuntu Linaro, 12.04, and I have no firewall installed on > all the nodes. > > > Any other workaround :( > > On Fri, Jun 28, 2013 at 11:43 AM, Pavan Balaji <balaji(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > > On 06/27/2013 09:41 PM, Syed. Jahanzeb Maqbool Hashmi wrote: > > MPID_nem_tcp_connpoll(1826): Communication error with rank 0: > Connection > refused > > > Sounds like a firewall problem. > > http://wiki.mpich.org/mpich/__index.php/Frequently_Asked___Questions#Q:_My_… > <http://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_My_MPI_…> > > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] mpich hangs
by Syed. Jahanzeb Maqbool Hashmi 27 Jun '13

27 Jun '13

As I already read some discussion on the mpich thread and I came to know these questions before: - *Can you ssh from "node01" to "node02"?* Yes I have passwordless ssh from each node to other (and vice versa) - *Make sure the firewalls are turned off on all machines.* I am using Ubuntu Linaro, 12.04, and I have no firewall installed on all the nodes. Any other workaround :( On Fri, Jun 28, 2013 at 11:43 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > On 06/27/2013 09:41 PM, Syed. Jahanzeb Maqbool Hashmi wrote: > >> MPID_nem_tcp_connpoll(1826): Communication error with rank 0: Connection >> refused >> > > Sounds like a firewall problem. > > http://wiki.mpich.org/mpich/**index.php/Frequently_Asked_** > Questions#Q:_My_MPI_program_**aborts_with_an_error_saying_** > it_cannot_communicate_with_**other_processes<http://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_My_MPI_…> > > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji >

1 0

Re: [mpich-discuss] mpich hangs
by Pavan Balaji 27 Jun '13

27 Jun '13

On 06/27/2013 09:41 PM, Syed. Jahanzeb Maqbool Hashmi wrote: > MPID_nem_tcp_connpoll(1826): Communication error with rank 0: Connection > refused Sounds like a firewall problem. http://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_My_MPI_… -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] mpich hangs
by Syed. Jahanzeb Maqbool Hashmi 27 Jun '13

27 Jun '13

sorry for giving such little information. ok here is the output after a long hang (which sometimes comes out) ================START OF OUTPUT===================== linaro@weiser1:/mnt/nfs/jahanzeb/bench/hpl/hpl-2.1/bin/armv7-a$ mpirun -np 8 -machinefile machines ./xhp l Fatal error in MPI_Send: A process has failed, error stack: MPI_Send(171)..............: MPI_Send(buf=0xbe84fc50, count=1, MPI_INT, dest=0, tag=9001, MPI_COMM_WORLD ) failed MPID_nem_tcp_connpoll(1826): Communication error with rank 0: Connection refused =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 1 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== [proxy:0:0@weiser1] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed [proxy:0:0@weiser1] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status [proxy:0:0@weiser1] main (./pm/pmiserv/pmip.c:206): demux engine error waiting for event [mpiexec@weiser1] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting [mpiexec@weiser1] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion [mpiexec@weiser1] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:217): launcher returned error waiting for completion [mpiexec@weiser1] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion ================ENDOF OUTPUT===================== On Fri, Jun 28, 2013 at 11:39 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > On 06/27/2013 09:36 PM, Syed. Jahanzeb Maqbool Hashmi wrote: > >> I am trying to run HPL on a cluster of nodes. The problem I am facing is >> with mpich, as I have successfully configured mpich. The program runs on >> single node without passing -machinefile argument. But as long as I >> execute of multiple nodes (-machinefile nodes) then the program hangs on >> indefinitely right after issuing the command. >> > > Given how little information you have provided, here's the only response I > can give: > > You are doing something wrong. > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji >

1 0

Re: [mpich-discuss] mpich hangs
by Pavan Balaji 27 Jun '13

27 Jun '13

On 06/27/2013 09:36 PM, Syed. Jahanzeb Maqbool Hashmi wrote: > I am trying to run HPL on a cluster of nodes. The problem I am facing is > with mpich, as I have successfully configured mpich. The program runs on > single node without passing -machinefile argument. But as long as I > execute of multiple nodes (-machinefile nodes) then the program hangs on > indefinitely right after issuing the command. Given how little information you have provided, here's the only response I can give: You are doing something wrong. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

mpich hangs
by Syed. Jahanzeb Maqbool Hashmi 27 Jun '13

27 Jun '13

Hello I am trying to run HPL on a cluster of nodes. The problem I am facing is with mpich, as I have successfully configured mpich. The program runs on single node without passing -machinefile argument. But as long as I execute of multiple nodes (-machinefile nodes) then the program hangs on indefinitely right after issuing the command. Any help?

1 0

Process-to-core binding in MPI_Comm_spawn
by Gonzalo Martín Cruz 27 Jun '13

27 Jun '13

Hi all, I am working with MPI_Comm_spawn to launch dynamic processes in my MPI application and I would like to use the process-to-core binding allocation strategy in processes spawned dynamically. I already know that it is possible to set the "host" key in the MPI_Info parameter of the MPI_Comm_spawn call to spawn the process in a specific host. However, I would like to know how to bound a dynamic process to a specific processor core. I tried passing to MPI_Comm_spawn the "host" info key as "compute-node-X binding:user=4", but it does not work and the process goes to any core randomly. Thank you very much, Regards, Gonzalo.

1 0

Re: [mpich-discuss] Non-blocking Collectives
by Jiri Simsa 27 Jun '13

27 Jun '13

Hi Pavan, To rephrase, I am interested in understanding when would MPI_Wait() block indefinitely, waiting for other process to make progress. I believe that your response answers my question. Thanks again. --Jiri On Wed, Jun 26, 2013 at 4:48 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > Hi Jiri, > > > On 06/26/2013 03:08 PM, Jiri Simsa wrote: > >> Thank you for your quick answer. I am trying to understand the blocking >> behavior of MPI_Wait in the case of non-blocking collectives. Is it safe >> to assume that, for a non-blocking collective, MPI_Wait is guaranteed to >> return once all other processes call the corresponding completion >> operation (e.g. MPI_Wait or MPI_Test)? >> > > I'm not sure I understand your question. Are you asking if MPI_WAIT in a > process is guaranteed to return after some finite amount of time after > every other process has called MPI_WAIT? Then, yes. > > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji >

1 0

Re: [mpich-discuss] mpich2 and windows server 2012
by Costas Yamin 27 Jun '13

27 Jun '13

1 0

Re: [mpich-discuss] mpich2 and windows server 2012
by Antonio J. Peña 27 Jun '13

27 Jun '13

Hi Lars, Unfortunately the MPICH team doesn't provide support for Windows anymore, as the development for Windows was discontinued some time ago, and we don't have any Windows experts anymore within our team. Maybe someone else in this mailing list can help you. Antonio On Thursday, June 27, 2013 09:59:37 AM Lars Lindbom wrote: Hi, I have a problem getting mpich2 to run on Windows Server 2012. We have a small set of servers successfully configured and running Windows Server 2008 R2 and MPICH2 without any problem. The error I get seems to indicate a problem in the authentication process. >"c:\Program Files (x86)\MPICH2\bin\mpiexec.exe" –validate SUCCESS >"c:\Program Files (x86)\MPICH2\bin\mpiexec.exe" -host localhost nonmem.exe Credentials for Lars rejected connecting to localhost Aborting: Unable to connect to localhost The credentials are correct and I have tried multiple user accounts with the same result. I don't think it's related but for what it's worth I have made sure that I have the same firewall settings for the mpich executables as on the 2008 R2 servers. I would appreciate any help in getting this solved. Thanks, Lars

1 0