That would be the one that comes with PGI 14.6 (MPICH 3.0.4) Bill On 7/22/14, 11:52 MDT, "Kenneth Raffenetti" <[email protected]> wrote:
What version of MPICH/Hydra is this?
On 07/22/2014 12:48 PM, Capehart, William J wrote:
Hi All
We¹re running MPICH on a couple machines with a brand new UNIX distro (SL 6.5) and that are on a vulnerable network and rather than leave the firewalls dropped we would like to run it through the firewall.
We have included the MPIEXEC_PORT_RANGE and MPIR_CVAR_CH3_PORT_RANGE fields and have adjusted our iptables accordingly and in line with the ³FAQ² guidance.
Our passwordless SSH works fine between the machines.
But all of this gives us momentary success with the cpi and fpi MPICH test programs. But they crash with the firewall up. (but of course run happily with the firewall down).
An example of the basic output is below (node short sends one process to ³this.machine² and one to remote ³that.machine²
[this.machine]% mpiexec -n 2 -f nodesshort cpi.exe
Process 0 of 2 is on this.machine
Process 1 of 2 is on that.machine
Fatal error in PMPI_Reduce: A process has failed, error stack:
PMPI_Reduce(1217)...............: MPI_Reduce(sbuf=0x7fff466a94d0, rbuf=0x7fff466a94d8, count=1, MPI_DOUBLE, MPI_SUM, root=0, MPI_COMM_WORLD) failed
MPIR_Reduce_impl(1029)..........:
MPIR_Reduce_intra(835)..........:
MPIR_Reduce_binomial(144).......:
MPIDI_CH3U_Recvq_FDU_or_AEP(667): Communication error with rank 1
========================================================================= ==========
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= EXIT CODE: 1
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
========================================================================= ==========
[proxy:0:[email protected]] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed
[proxy:0:[email protected]] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
[proxy:0:[email protected]] main (./pm/pmiserv/pmip.c:206): demux engine error waiting for event
[[email protected]] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting
[[email protected]] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion
[[email protected]] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:217): launcher returned error waiting for completion
[[email protected]] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion
In debug mode it affirms that it is at least *starting with the first available port as listed in MPIEXEC_PORT_RANGE
But later we get output like this:
[[email protected]] PMI response to fd 6 pid 4: cmd=keyval_cache
P0-businesscard=description#{this.machine¹s.ip.address}$port#54105$ifname #{this.machine¹s.ip.address}$
P1-businesscard=description#{that.machine¹s.ip.address}$port#47302$ifname #{that.machine¹s.ip.address}$
Does this mean that we have missed a firewall setting either in the environment variables or in the ip tables themselves?
Ideas?
Thanks Much
Bill
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss