My bad, I just found out that there was a duplicate entry like: weiser1 127.0.1.1 weiser1 192.168.0.101 so i removed teh 127.x.x.x. entry and kept the hostfile contents similar on both nodes. Now previous error is reduced to this one: ------ START OF OUTPUT ------- ....some HPL startup string (no final result) ...skip..... =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 9 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== [proxy:0:0@weiser1] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed [proxy:0:0@weiser1] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status [proxy:0:0@weiser1] main (./pm/pmiserv/pmip.c:206): demux engine error waiting for event [mpiexec@weiser1] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting [mpiexec@weiser1] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion [mpiexec@weiser1] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:217): launcher returned error waiting for completion [mpiexec@weiser1] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion ------ END OF OUTPUT ------- On Fri, Jun 28, 2013 at 12:12 PM, Pavan Balaji <[email protected]> wrote:

On 06/27/2013 10:08 PM, Syed. Jahanzeb Maqbool Hashmi wrote:

...

P4-businesscard=description#**weiser2$port#57651$ifname#192.**168.0.102$ P5-businesscard=description#**weiser2$port#52622$ifname#192.**168.0.102$ P6-businesscard=description#**weiser2$port#55935$ifname#192.**168.0.102$ P7-businesscard=description#**weiser2$port#54952$ifname#192.**168.0.102$ P0-businesscard=description#**weiser1$port#41958$ifname#127.**0.1.1$ P2-businesscard=description#**weiser1$port#35049$ifname#127.**0.1.1$ P1-businesscard=description#**weiser1$port#39634$ifname#127.**0.1.1$ P3-businesscard=description#**weiser1$port#51802$ifname#127.**0.1.1$

I have two concerns with your output. Let's start with the first.

Did you look at this question on the FAQ page?

"Is your /etc/hosts file consistent across all nodes? Unless you are using an external DNS server, the /etc/hosts file on every machine should contain the correct IP information about all hosts in the system."

-- Pavan

-- Pavan Balaji http://www.mcs.anl.gov/~balaji