Hi, yesterday I have built mpich-master-v3.2b4-211-gf91baf0296ce on my machines (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 x86_64) with gcc-5.1.0 and Sun C 5.13. I get the following errors if I run small programs that spawn processes on two Sparc machines. "mpiexec" is aliased to 'mpiexec -genvnone'. It doesn't matter if I use my cc- or gcc-version of MPICH. tyr spawn 120 mpiexec -np 1 --host tyr,rs0 spawn_master Parent process 0 running on tyr.informatik.hs-fulda.de I create 4 slave processes Fatal error in MPI_Init: Unknown error class, error stack: MPIR_Init_thread(472).................: MPID_Init(302)........................: spawned process group was unable to connect back to the parent on port <tag#0$description#tyr$port#40568$ifname#193.174.24.39$> MPID_Comm_connect(191)................: MPIDI_Comm_connect(488)...............: SetupNewIntercomm(1187)...............: MPIR_Barrier_intra(150)...............: barrier_smp_intra(96).................: MPIR_Barrier_impl(332)................: Failure during collective MPIR_Barrier_impl(327)................: MPIR_Barrier(292).....................: MPIR_Barrier_intra(169)...............: MPIDU_Complete_posted_with_error(1137): Process failed barrier_smp_intra(111)................: MPIR_Bcast_impl(1452).................: MPIR_Bcast(1476)......................: MPIR_Bcast_intra(1287)................: MPIR_Bcast_binomial(310)..............: Failure during collective Fatal error in MPI_Init: Unknown error class, error stack: MPIR_Init_thread(472)...: MPID_Init(302)..........: spawned process group was unable to connect back to the parent on port <tag#0$description#tyr$port#40568$ifname#193.174.24.39$> MPID_Comm_connect(191)..: MPIDI_Comm_connect(488).: SetupNewIntercomm(1187).: MPIR_Barrier_intra(150).: barrier_smp_intra(111)..: MPIR_Bcast_impl(1452)...: MPIR_Bcast(1476)........: MPIR_Bcast_intra(1287)..: MPIR_Bcast_binomial(310): Failure during collective tyr spawn 121 I get the following error or something similar to the above error message with "mpiexec -np 1 --host tyr,rs0 spawn_multiple_master" and "mpiexec -np 1 --host tyr,rs0 spawn_intra_comm". tyr spawn 127 mpiexec -np 1 --host tyr,rs0 spawn_multiple_master Parent process 0 running on tyr.informatik.hs-fulda.de I create 3 slave processes. Fatal error in MPI_Comm_spawn_multiple: Unknown error class, error stack: MPI_Comm_spawn_multiple(162)..: MPI_Comm_spawn_multiple(count=2, cmds=ffffffff7fffdf08, argvs=ffffffff7fffdef8, maxprocs=ffffffff7fffdef0, infos=ffffffff7fffdee8, root=0, MPI_COMM_WORLD, intercomm=ffffffff7fffdee4, errors=0) failed MPIDI_Comm_spawn_multiple(274): MPID_Comm_accept(153).........: MPIDI_Comm_accept(1057).......: MPIR_Bcast_intra(1287)........: MPIR_Bcast_binomial(310)......: Failure during collective =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 14925 RUNNING AT rs0 = EXIT CODE: 10 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== tyr spawn 128 Sometimes I also get this error message. tyr spawn 129 mpiexec -np 1 --host tyr,rs0 spawn_multiple_master Parent process 0 running on tyr.informatik.hs-fulda.de I create 3 slave processes. =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 11444 RUNNING AT tyr = EXIT CODE: 10 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== [proxy:0:[email protected]] HYD_pmcd_pmip_control_cmd_cb (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/pm/pmiserv/pmip_cb.c:885): assert (!closed) failed [proxy:0:[email protected]] HYDT_dmxu_poll_wait_for_event (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/tools/demux/demux_poll.c:76): callback returned error status [proxy:0:[email protected]] main (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/pm/pmiserv/pmip.c:206): demux engine error waiting for event [proxy:1:[email protected]] HYD_pmcd_pmip_control_cmd_cb (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/pm/pmiserv/pmip_cb.c:885): assert (!closed) failed [proxy:1:[email protected]] HYDT_dmxu_poll_wait_for_event (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/tools/demux/demux_poll.c:76): callback returned error status [proxy:1:[email protected]] main (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/pm/pmiserv/pmip.c:206): demux engine error waiting for event [[email protected]] HYDT_bscu_wait_for_completion (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/tools/bootstrap/utils/bscu_wait.c:75): one of the processes terminated badly; aborting [[email protected]] HYDT_bsci_wait_for_completion (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion [[email protected]] HYD_pmci_wait_for_completion (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/pm/pmiserv/pmiserv_pmci.c:218): launcher returned error waiting for completion [[email protected]] main (../../../../mpich-master-v3.2b4-211-gf91baf0296ce/src/pm/hydra/ui/mpich/mpiexec.c:344): process manager error waiting for completion tyr spawn 130 Sometimes it even works. tyr spawn 131 mpiexec -np 1 --host tyr,rs0 spawn_multiple_master Parent process 0 running on tyr.informatik.hs-fulda.de I create 3 slave processes. Parent process 0: tasks in MPI_COMM_WORLD: 1 tasks in COMM_CHILD_PROCESSES local group: 1 tasks in COMM_CHILD_PROCESSES remote group: 3 Slave process 2 of 3 running on rs0.informatik.hs-fulda.de Slave process 0 of 3 running on rs0.informatik.hs-fulda.de Slave process 1 of 3 running on tyr.informatik.hs-fulda.de spawn_slave 0: argv[0]: spawn_slave spawn_slave 0: argv[1]: program type 1 spawn_slave 2: argv[0]: spawn_slave spawn_slave 2: argv[1]: program type 2 spawn_slave 2: argv[2]: another parameter spawn_slave 1: argv[0]: spawn_slave spawn_slave 1: argv[1]: program type 2 spawn_slave 1: argv[2]: another parameter tyr spawn 132 It seems that the programs work fine on my x86_64 machines. At least I wasn't able to produce an error. tyr spawn 121 ssh linpc1 linpc1 fd1026 107 mpiexec -np 1 --host sunpc0,linpc1 spawn_master Parent process 0 running on sunpc0 I create 4 slave processes Parent process 0: tasks in MPI_COMM_WORLD: 1 tasks in COMM_CHILD_PROCESSES local group: 1 tasks in COMM_CHILD_PROCESSES remote group: 4 Slave process 0 of 4 running on linpc0 Slave process 2 of 4 running on linpc0 Slave process 1 of 4 running on sunpc0 Slave process 3 of 4 running on sunpc0 spawn_slave 1: argv[0]: spawn_slave spawn_slave 3: argv[0]: spawn_slave spawn_slave 0: argv[0]: spawn_slave spawn_slave 2: argv[0]: spawn_slave linpc1 fd1026 102 mpiexec -np 1 --host sunpc0,linpc0 spawn_multiple_master Parent process 0 running on sunpc0 I create 3 slave processes. Parent process 0: tasks in MPI_COMM_WORLD: 1 tasks in COMM_CHILD_PROCESSES local group: 1 tasks in COMM_CHILD_PROCESSES remote group: 3 Slave process 0 of 3 running on linpc0 Slave process 2 of 3 running on linpc0 Slave process 1 of 3 running on sunpc0 spawn_slave 0: argv[0]: spawn_slave spawn_slave 0: argv[1]: program type 1 spawn_slave 1: argv[0]: spawn_slave spawn_slave 1: argv[1]: program type 2 spawn_slave 1: argv[2]: another parameter spawn_slave 2: argv[0]: spawn_slave spawn_slave 2: argv[1]: program type 2 spawn_slave 2: argv[2]: another parameter linpc1 fd1026 103 linpc1 fd1026 103 mpiexec -np 1 --host sunpc0,linpc0 spawn_intra_comm Parent process 0: I create 2 slave processes Parent process 0 running on sunpc0 MPI_COMM_WORLD ntasks: 1 COMM_CHILD_PROCESSES ntasks_local: 1 COMM_CHILD_PROCESSES ntasks_remote: 2 COMM_ALL_PROCESSES ntasks: 3 mytid in COMM_ALL_PROCESSES: 0 Child process 1 running on sunpc0 MPI_COMM_WORLD ntasks: 2 COMM_ALL_PROCESSES ntasks: 3 mytid in COMM_ALL_PROCESSES: 2 Child process 0 running on linpc0 MPI_COMM_WORLD ntasks: 2 COMM_ALL_PROCESSES ntasks: 3 mytid in COMM_ALL_PROCESSES: 1 linpc1 fd1026 104 Kind regards Siegmar _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss