Hello Tony, Thank you very much for your reply. I am posting you the output of 'ps x', when 'mpirun -n 5 mpich-3.0.4/examples/hellow' freezes: " marto@dragonfly% ps x PID TT STAT TIME COMMAND 1025 ?? I6s 0:00.14 /usr/local/bin/dbus-daemon --syslog-only --fork --print-pid 5 --print-address 7 --session 1028 ?? I4s 0:00.08 kdeinit5: Running... (kdeinit5) 1029 ?? I2 0:00.26 /usr/local/lib/libexec/kf5/klauncher --fd=8 1031 ?? I7 0:00.65 kded5 1040 ?? I3 0:00.21 /usr/local/bin/kaccess 1047 ?? I4 0:00.02 /usr/local/libexec/dconf-service 1058 ?? I1 0:00.26 /usr/local/bin/ksmserver 1061 ?? I6 0:00.38 /usr/local/bin/kglobalaccel5 1067 ?? I7 0:00.10 /usr/local/lib/libexec/kf5/kscreen_backend_launcher 1069 ?? I0 0:18.15 /usr/local/bin/kwin_x11 -session 100000000158893236700000019470008_1589011398_40052 1071 ?? I3 0:07.36 /usr/local/bin/plasmashell 1073 ?? I6 0:00.06 /usr/local/bin/xembedsniproxy 1075 ?? I3 0:00.17 /usr/local/lib/libexec/polkit-kde-authentication-agent-1 1080 ?? I6 0:00.05 /usr/local/bin/kwrited 1084 ?? I5 0:00.07 /usr/local/bin/gmenudbusmenuproxy 1093 ?? I2 0:00.24 /usr/local/lib/libexec/DiscoverNotifier -session 1014ce0c7d3000158899951500000013340005_1589011398_16101 1121 ?? I2 0:00.45 /usr/local/bin/korgac -session 100000000158893236700000019470009_1589011398_16368 1128 ?? I2 0:00.00 /usr/local/libexec/at-spi-bus-launcher 1130 ?? I3 0:00.24 /usr/local/lib/libexec/kactivitymanagerd 1131 ?? S5 0:00.11 /usr/local/bin/dbus-daemon --config-file=/usr/local/share/defaults/at-spi2/accessibility.conf --nofork --print-address 3 1134 ?? I2 0:00.07 /usr/local/libexec/at-spi2-registryd --use-gnome-session 1136 ?? I1 0:07.00 /usr/local/lib/thunderbird/thunderbird --sm-client-id 100000000158893248800000019470014 1140 ?? S5 0:01.73 /usr/local/bin/gkrellm --sm-client-id 100000000158893266700000019470017 1142 ?? I6 0:00.54 /usr/local/bin/dolphin -session 100000000158893337700000019470020_1589011398_16282 1144 ?? I7 0:00.35 /usr/local/bin/kmix -session 1014ce0c7d3000158899952000000013340007_1589011398_17041 1146 ?? I0 0:01.01 /usr/local/bin/konsole -session 1014ce0c7d3000158899955500000013340009_1589011398_16527 1152 ?? I1 0:00.24 /usr/local/lib/libexec/org_kde_powerdevil 1214 ?? I4 0:00.00 kdeinit5: file.so file local:/var/run/user/1001/klauncherPKUbmP.1.slave-socket local:/var/run/user/1001/kio_desktopMgyNoK.1.slave-socket (kdeinit5) 1216 ?? Z 0:00.00 (sh) 1217 ?? Z 0:00.00 (sh) 1218 ?? Z 0:00.00 (sh) 1219 ?? Z 0:00.01 (sh) 1220 ?? Z 0:00.00 (sh) 1221 ?? Z 0:00.01 (sh) 1222 ?? Z 0:00.00 (sh) 1223 ?? Z 0:00.00 (sh) 1224 ?? Z 0:00.01 (sh) 1225 ?? Z 0:00.00 (sh) 1226 ?? Z 0:00.00 (sh) 1227 ?? Z 0:00.00 (sh) 1228 ?? Z 0:00.00 (sh) 1229 ?? Z 0:00.00 (sh) 1230 ?? Z 0:00.00 (sh) 1231 ?? Z 0:00.00 (sh) 1232 ?? Z 0:00.01 (sh) 1233 ?? Z 0:00.00 (sh) 1234 ?? Z 0:00.01 (sh) 1235 ?? Z 0:00.00 (sh) 1277 ?? S4 0:00.00 kdeinit5: file.so file local:/var/run/user/1001/klauncherPKUbmP.1.slave-socket local:/var/run/user/1001/kded5wwlydC.1.slave-socket (kdeinit5) 1278 ?? I7 0:05.48 chrome: (chrome) 1281 ?? S1 0:01.06 chrome: --type=utility --field-trial-handle=9865741493051962933,5833454360386249874,131072 --lang=en-US --service-sandbox-type=network --disable-webrtc-apm-in-audio-service --shared 1282 ?? I0 0:02.61 chrome: --type=gpu-process --field-trial-handle=9865741493051962933,5833454360386249874,131072 --gpu-preferences=MAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAABgAAAAAAAQAAAAAAAAAAAAAAAAAAAACAAA 1302 ?? I0 0:00.67 chrome: --type=renderer --disable-webrtc-apm-in-audio-service --field-trial-handle=9865741493051962933,5833454360386249874,131072 --disable-gpu-compositing --lang=en-US --enable-aut 1315 ?? I6 0:14.93 chrome: --type=renderer --disable-webrtc-apm-in-audio-service --field-trial-handle=9865741493051962933,5833454360386249874,131072 --disable-gpu-compositing --lang=en-US --enable-aut 1320 ?? I1 0:00.26 chrome: --type=renderer --disable-webrtc-apm-in-audio-service --field-trial-handle=9865741493051962933,5833454360386249874,131072 --disable-gpu-compositing --lang=en-US --enable-aut 1321 ?? I4 0:00.20 chrome: --type=renderer --disable-webrtc-apm-in-audio-service --field-trial-handle=9865741493051962933,5833454360386249874,131072 --disable-gpu-compositing --lang=en-US --enable-aut 1322 ?? I5 0:00.25 chrome: --type=utility --field-trial-handle=9865741493051962933,5833454360386249874,131072 --lang=en-US --service-sandbox-type=audio --disable-webrtc-apm-in-audio-service --shared-f 1422 ?? I2s 0:00.00 /home/marto/WRF/Build_WRF/LIBRARIES/mpich/bin/hydra_pmi_proxy --control-port dragonfly:2416 --rmk user --launcher ssh --demux poll --pgid 0 --retries 10 --usize -2 --proxy-id 0 1423 ?? R4s 1:11.89 mpich-3.0.4/examples/hellow 1424 ?? I2s 0:00.00 mpich-3.0.4/examples/hellow 1425 ?? R3s 1:11.04 mpich-3.0.4/examples/hellow 1426 ?? R2s 1:11.39 mpich-3.0.4/examples/hellow 1427 ?? R6s 1:11.96 mpich-3.0.4/examples/hellow 972 v0 I0 0:00.00 -tcsh (tcsh) 977 v0 I0+ 0:00.00 /bin/sh /usr/local/bin/startx 1001 v0 I0+ 0:00.00 xinit /home/marto/.xinitrc -- /usr/local/bin/X :0 -auth /home/marto/.serverauth.977 1004 v0 I0 0:00.00 ck-launch-session startplasma-x11 1013 v0 I1 0:00.03 startplasma-x11 1024 v0 I6 0:00.00 dbus-launch --autolaunch 7a25735143fe35ec86d2d9be5eac81cc --binary-syntax --close-stderr 1043 v0 I2 0:00.13 /usr/local/bin/plasma_session 1196 1 I2s 0:00.07 /bin/tcsh 1421 1 I2+ 0:00.00 mpirun -n 5 mpich-3.0.4/examples/hellow (mpiexec.hydra) 1199 2 I5s+ 0:00.04 /bin/tcsh 1194 3 S6s 0:00.04 /bin/tcsh 1429 3 R6+ 0:00.00 ps x " After that freeze, I killed mpirun with Ctrl + C: " marto@dragonfly% mpirun -n 5 mpich-3.0.4/examples/hellow ^C[mpiexec@dragonfly] Sending Ctrl-C to processes as requested [mpiexec@dragonfly] Press Ctrl-C again to force abort [proxy:0:0@dragonfly] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:71): assert (!(pollfds[i].revents & ~POLLIN & ~POLLOUT & ~POLLHUP & ~POLLERR)) failed [proxy:0:0@dragonfly] main (./pm/pmiserv/pmip.c:206): demux engine error waiting for event [mpiexec@dragonfly] control_cb (./pm/pmiserv/pmiserv_cb.c:202): assert (!closed) failed [mpiexec@dragonfly] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status [mpiexec@dragonfly] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:197): error waiting for event [mpiexec@dragonfly] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion " The next relaunch of mpirun with 5 cores was successful. For completeness, I am attaching the output of 'ps x' after mpirun with 2 cores freezes, which it actually with 2 cores always does. I hope this was helpful. I am looking forward to your reply. Best regards, Martin On Sat, May 9, 2020 at 2:15 PM Tony Curtis <[email protected]> wrote:

On May 9, 2020, at 2:24 AM, Martin Ivanov <[email protected]> wrote:

Hello Tony, Thank you very much for your reply. I followed your advice and gave the hostname 'dragonfly' to my machine. Then, in /etc/hosts I provided gave the alias 'dragonfly' to localhost as you suggested:

" marto@dragonfly% cat /etc/hosts ::1 localhost dragonfly 127.0.0.1 localhost dragonfly "

I compiled both the icpi and hellow examples. Now mpiexec seems to work, although not as reliably as I might wish. Running any of the examples with one core like this:

mpiexec -n 1 mpich-3.0.4/examples/hellow

is always successful. I could never get the command to finish with 2, 3, or 4 cores: it simply hangs. E.g. with 2 cores after running the above command I get:

" marto@dragonfly% ps x | grep hellow 166493 ?? I6s 0:00.00 mpich-3.0.4/examples/hellow 166494 ?? I6s 0:00.00 mpich-3.0.4/examples/hellow "

Ok, good, that seemed like it was the obvious problem. Can you show all the processes involved in the launch, not just the application (i.e. also the mpirun and anything that is spawned)? This might provide a further clue about what is happening underneath.

Tony