August 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Problem with build MPICH2.
by Anatoly G 29 Aug '13

29 Aug '13

It works. Thank you very much. Anatoly. On Wed, Aug 28, 2013 at 1:43 PM, Reuti <reuti(a)staff.uni-marburg.de> wrote: > Am 28.08.2013 um 09:24 schrieb Anatoly G: > > > Dear MPICH2. > > I want to build MPICH2 package. > > I have downloaded mpich2-1.5.tar.gz from > > > http://www.mcs.anl.gov/research/projects/mpich2staging/goodell/downloads/in… > > > > My computer OS: Red Hat Enterprise Linux Server release 6.2 (Santiago) > > I created folder: > > /tmp/MPICH2/src > > there I executed: tar xfz mpich2-1.5.tar.gz > > and got new folder (/tmp/MPICH2/src/mpich2-1.5) with mpich2 files and > sub-folders. > > Now I created folder: /tmp/MPICH2/build > > from directory /tmp/MPICH2/build, I executed: > > > > ../src/mpich2-1.5/configure --prefix=/tmp/MPICH2/install > --enable-error-checking=runtime --enable-g=dbg CFLAGS=-fPIC CXXFLAGS=-fPIC > FFLAGS=-fpic --enable-threads=runtime --enable-totalview --enable-static > --disable-f77 --disable-fc --no-create --no-recursion | & tee c.txt > > Does it work if you leave out "--no-create"? > > -- Reuti > > > > Then I tried to execute (still from folder: /tmp/MPICH2/build): make > > but got an error: > > make: *** No targets specified and no makefile found. Stop. > > > > Then I tried to execute "ls" (still from folder: /tmp/MPICH2/build) and > got following ouput: > > config.log config.lt config.status config.system c.txt lib > libtool src > > > > Then I tried to execute "ls" (from folder: /tmp/MPICH2/src/mpich2-1.5) > and got following ouput: > > -rw-r--r-- 1 anatol-g ws 43319 Oct 8 2012 aclocal.m4 > > -rwxr-xr-x 1 anatol-g ws 26809 Sep 27 2012 autogen.sh > > -rw-r--r-- 1 anatol-g ws 55773 Oct 8 2012 CHANGES > > drwxr-xr-x 2 anatol-g ws 4096 Oct 8 2012 confdb > > -rwxr-xr-x 1 anatol-g ws 1802576 Oct 8 2012 configure > > -rw-r--r-- 1 anatol-g ws 228771 Oct 4 2012 configure.ac > > drwxr-xr-x 3 anatol-g ws 4096 Oct 8 2012 contrib > > -rw-r--r-- 1 anatol-g ws 1439 Jul 17 2012 COPYRIGHT > > -rw-r--r-- 1 anatol-g ws 8076 Feb 27 2009 COPYRIGHT.rtf > > drwxr-xr-x 13 anatol-g ws 4096 Oct 8 2012 doc > > drwxr-xr-x 8 anatol-g ws 4096 Oct 8 2012 examples > > drwxr-xr-x 5 anatol-g ws 4096 Oct 8 2012 maint > > -rw-r--r-- 1 anatol-g ws 17216 Oct 4 2012 Makefile.am > > -rw-r--r-- 1 anatol-g ws 2579709 Oct 8 2012 Makefile.in > > -rwxr-xr-x 1 anatol-g ws 9886 Nov 12 2010 makewindist.bat > > drwxr-xr-x 4 anatol-g ws 4096 Oct 8 2012 man > > -rw-r--r-- 1 anatol-g ws 21620 Oct 5 2010 mpich2.def > > -rw-r--r-- 1 anatol-g ws 51983 May 2 2012 mpich2-doxygen.in > > -rw-r--r-- 1 anatol-g ws 556769 Nov 12 2010 mpich2.sln > > -rw-r--r-- 1 anatol-g ws 115749 Jan 24 2011 mpich2s.vcproj > > -rw-r--r-- 1 anatol-g ws 234598 Jan 24 2011 mpich2.vcproj > > -rw-r--r-- 1 anatol-g ws 15828 Oct 5 2010 mpi.def > > -rw-r--r-- 1 anatol-g ws 10010 May 29 2010 mpi.vcproj > > -rwxr-xr-x 1 anatol-g ws 4759 May 29 2010 mpl.vcproj > > -rw-r--r-- 1 anatol-g ws 38842 Oct 8 2012 README > > -rw-r--r-- 1 anatol-g ws 12689 Oct 8 2012 README.envvar > > -rw-r--r-- 1 anatol-g ws 20773 Aug 24 2012 README.winbin.rtf > > -rw-r--r-- 1 anatol-g ws 3758 Oct 3 2012 RELEASE_NOTES > > drwxr-xr-x 19 anatol-g ws 4096 Oct 8 2012 src > > -rw-r--r-- 1 anatol-g ws 2622 Oct 8 2012 subsys_include.m4 > > drwxr-xr-x 9 anatol-g ws 4096 Oct 8 2012 test > > -rwxr-xr-x 1 anatol-g ws 335084 Jul 10 2012 winconfigure.wsf > > drwxr-xr-x 4 anatol-g ws 4096 Oct 8 2012 www > > > > > > > > Can you please advice: > > - What I did wrong? (Why I can't perform make, according to manual). > > - Is my configure options are correct to build MPICH2 w/o any .so files? > > > > c.txt attached to mail. > > > > Regards, > > Anatoly. > > > > <c.txt>_______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] Problem with build MPICH2.
by Antonio J. Peña 28 Aug '13

28 Aug '13

Please try with our most recent stable version (3.0.4). We can be more supportive on recent versions. Here you have the link to the downloads page: http://www.mpich.org/downloads/ Antonio On Wednesday, August 28, 2013 10:24:28 AM Anatoly G wrote: Dear MPICH2. I want to build MPICH2 package. I have downloaded mpich2-1.5.tar.gz from http://www.mcs.anl.gov/research/projects/mpich2staging/goodell/download s/index.php?s=downloads[1] My computer OS: Red Hat Enterprise Linux Server release 6.2 (Santiago) I created folder: /tmp/MPICH2/src there I executed: tar xfz mpich2-1.5.tar.gz and got new folder (/tmp/MPICH2/src/mpich2-1.5) with mpich2 files and sub- folders. Now I created folder: /tmp/MPICH2/build from directory /tmp/MPICH2/build, I executed: ../src/mpich2-1.5/configure --prefix=/tmp/MPICH2/install --enable-error- checking=runtime --enable-g=dbg CFLAGS=-fPIC CXXFLAGS=-fPIC FFLAGS=- fpic --enable-threads=runtime --enable-totalview --enable-static --disable- f77 --disable-fc --no-create --no-recursion | & tee c.txt Then I tried to execute (still from folder: /tmp/MPICH2/build): make but got an error: make: *** No targets specified and no makefile found. Stop. Then I tried to execute "ls" (still from folder: /tmp/MPICH2/build) and got following ouput: config.log config.lt[2] config.status config.system c.txt lib libtool src Then I tried to execute "ls" (from folder: /tmp/MPICH2/src/mpich2-1.5) and got following ouput: -rw-r--r-- 1 anatol-g ws 43319 Oct 8 2012 aclocal.m4 -rwxr-xr-x 1 anatol-g ws 26809 Sep 27 2012 autogen.sh -rw-r--r-- 1 anatol-g ws 55773 Oct 8 2012 CHANGES drwxr-xr-x 2 anatol-g ws 4096 Oct 8 2012 confdb -rwxr-xr-x 1 anatol-g ws 1802576 Oct 8 2012 configure -rw-r--r-- 1 anatol-g ws 228771 Oct 4 2012 configure.ac[3] drwxr-xr-x 3 anatol-g ws 4096 Oct 8 2012 contrib -rw-r--r-- 1 anatol-g ws 1439 Jul 17 2012 COPYRIGHT -rw-r--r-- 1 anatol-g ws 8076 Feb 27 2009 COPYRIGHT.rtf drwxr-xr-x 13 anatol-g ws 4096 Oct 8 2012 doc drwxr-xr-x 8 anatol-g ws 4096 Oct 8 2012 examples drwxr-xr-x 5 anatol-g ws 4096 Oct 8 2012 maint -rw-r--r-- 1 anatol-g ws 17216 Oct 4 2012 Makefile.am -rw-r--r-- 1 anatol-g ws 2579709 Oct 8 2012 Makefile.in -rwxr-xr-x 1 anatol-g ws 9886 Nov 12 2010 makewindist.bat drwxr-xr-x 4 anatol-g ws 4096 Oct 8 2012 man -rw-r--r-- 1 anatol-g ws 21620 Oct 5 2010 mpich2.def -rw-r--r-- 1 anatol-g ws 51983 May 2 2012 mpich2-doxygen.in[4] -rw-r--r-- 1 anatol-g ws 556769 Nov 12 2010 mpich2.sln -rw-r--r-- 1 anatol-g ws 115749 Jan 24 2011 mpich2s.vcproj -rw-r--r-- 1 anatol-g ws 234598 Jan 24 2011 mpich2.vcproj -rw-r--r-- 1 anatol-g ws 15828 Oct 5 2010 mpi.def -rw-r--r-- 1 anatol-g ws 10010 May 29 2010 mpi.vcproj -rwxr-xr-x 1 anatol-g ws 4759 May 29 2010 mpl.vcproj -rw-r--r-- 1 anatol-g ws 38842 Oct 8 2012 README -rw-r--r-- 1 anatol-g ws 12689 Oct 8 2012 README.envvar -rw-r--r-- 1 anatol-g ws 20773 Aug 24 2012 README.winbin.rtf -rw-r--r-- 1 anatol-g ws 3758 Oct 3 2012 RELEASE_NOTES drwxr-xr-x 19 anatol-g ws 4096 Oct 8 2012 src -rw-r--r-- 1 anatol-g ws 2622 Oct 8 2012 subsys_include.m4 drwxr-xr-x 9 anatol-g ws 4096 Oct 8 2012 test -rwxr-xr-x 1 anatol-g ws 335084 Jul 10 2012 winconfigure.wsf drwxr-xr-x 4 anatol-g ws 4096 Oct 8 2012 www Can you please advice: - What I did wrong? (Why I can't perform make, according to manual). - Is my configure options are correct to build MPICH2 w/o any .so files? c.txt attached to mail. Regards, Anatoly. -------- [1] http://www.mcs.anl.gov/research/projects/mpich2staging/goodell/download s/index.php?s=downloads [2] http://config.lt [3] http://configure.ac [4] http://mpich2-doxygen.in

1 0

Re: [mpich-discuss] Problem with build MPICH2.
by Reuti 28 Aug '13

28 Aug '13

Am 28.08.2013 um 09:24 schrieb Anatoly G: > Dear MPICH2. > I want to build MPICH2 package. > I have downloaded mpich2-1.5.tar.gz from > http://www.mcs.anl.gov/research/projects/mpich2staging/goodell/downloads/in… > > My computer OS: Red Hat Enterprise Linux Server release 6.2 (Santiago) > I created folder: > /tmp/MPICH2/src > there I executed: tar xfz mpich2-1.5.tar.gz > and got new folder (/tmp/MPICH2/src/mpich2-1.5) with mpich2 files and sub-folders. > Now I created folder: /tmp/MPICH2/build > from directory /tmp/MPICH2/build, I executed: > > ../src/mpich2-1.5/configure --prefix=/tmp/MPICH2/install --enable-error-checking=runtime --enable-g=dbg CFLAGS=-fPIC CXXFLAGS=-fPIC FFLAGS=-fpic --enable-threads=runtime --enable-totalview --enable-static --disable-f77 --disable-fc --no-create --no-recursion | & tee c.txt Does it work if you leave out "--no-create"? -- Reuti > Then I tried to execute (still from folder: /tmp/MPICH2/build): make > but got an error: > make: *** No targets specified and no makefile found. Stop. > > Then I tried to execute "ls" (still from folder: /tmp/MPICH2/build) and got following ouput: > config.log config.lt config.status config.system c.txt lib libtool src > > Then I tried to execute "ls" (from folder: /tmp/MPICH2/src/mpich2-1.5) and got following ouput: > -rw-r--r-- 1 anatol-g ws 43319 Oct 8 2012 aclocal.m4 > -rwxr-xr-x 1 anatol-g ws 26809 Sep 27 2012 autogen.sh > -rw-r--r-- 1 anatol-g ws 55773 Oct 8 2012 CHANGES > drwxr-xr-x 2 anatol-g ws 4096 Oct 8 2012 confdb > -rwxr-xr-x 1 anatol-g ws 1802576 Oct 8 2012 configure > -rw-r--r-- 1 anatol-g ws 228771 Oct 4 2012 configure.ac > drwxr-xr-x 3 anatol-g ws 4096 Oct 8 2012 contrib > -rw-r--r-- 1 anatol-g ws 1439 Jul 17 2012 COPYRIGHT > -rw-r--r-- 1 anatol-g ws 8076 Feb 27 2009 COPYRIGHT.rtf > drwxr-xr-x 13 anatol-g ws 4096 Oct 8 2012 doc > drwxr-xr-x 8 anatol-g ws 4096 Oct 8 2012 examples > drwxr-xr-x 5 anatol-g ws 4096 Oct 8 2012 maint > -rw-r--r-- 1 anatol-g ws 17216 Oct 4 2012 Makefile.am > -rw-r--r-- 1 anatol-g ws 2579709 Oct 8 2012 Makefile.in > -rwxr-xr-x 1 anatol-g ws 9886 Nov 12 2010 makewindist.bat > drwxr-xr-x 4 anatol-g ws 4096 Oct 8 2012 man > -rw-r--r-- 1 anatol-g ws 21620 Oct 5 2010 mpich2.def > -rw-r--r-- 1 anatol-g ws 51983 May 2 2012 mpich2-doxygen.in > -rw-r--r-- 1 anatol-g ws 556769 Nov 12 2010 mpich2.sln > -rw-r--r-- 1 anatol-g ws 115749 Jan 24 2011 mpich2s.vcproj > -rw-r--r-- 1 anatol-g ws 234598 Jan 24 2011 mpich2.vcproj > -rw-r--r-- 1 anatol-g ws 15828 Oct 5 2010 mpi.def > -rw-r--r-- 1 anatol-g ws 10010 May 29 2010 mpi.vcproj > -rwxr-xr-x 1 anatol-g ws 4759 May 29 2010 mpl.vcproj > -rw-r--r-- 1 anatol-g ws 38842 Oct 8 2012 README > -rw-r--r-- 1 anatol-g ws 12689 Oct 8 2012 README.envvar > -rw-r--r-- 1 anatol-g ws 20773 Aug 24 2012 README.winbin.rtf > -rw-r--r-- 1 anatol-g ws 3758 Oct 3 2012 RELEASE_NOTES > drwxr-xr-x 19 anatol-g ws 4096 Oct 8 2012 src > -rw-r--r-- 1 anatol-g ws 2622 Oct 8 2012 subsys_include.m4 > drwxr-xr-x 9 anatol-g ws 4096 Oct 8 2012 test > -rwxr-xr-x 1 anatol-g ws 335084 Jul 10 2012 winconfigure.wsf > drwxr-xr-x 4 anatol-g ws 4096 Oct 8 2012 www > > > > Can you please advice: > - What I did wrong? (Why I can't perform make, according to manual). > - Is my configure options are correct to build MPICH2 w/o any .so files? > > c.txt attached to mail. > > Regards, > Anatoly. > > <c.txt>_______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] ./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed
by Joni-Pekka Kurronen 28 Aug '13

28 Aug '13

hi, I can get time to time 6 processs running and sometime it crash when 5 started,... so takeing third machine, what so ever start's the problem,... it's question communication between two slave's,... and crash when two salves are started to work together mpirin -np 2 -hosts ugh,kaak ls File rights should be ok, ports are open, useing rsh,.... Communication, demux engine ? Could PAM controll affect ? Still wondering NFS4 ? joni hosts ====== mpi1:2 ugh:2 kaak:2 Below error's debug after crash ====================== mpiexec -np 5 hostname .... [mpiexec@mpi1] Launch arguments: /mpi3/C3/mpich-3.0.4/bin/hydra_pmi_proxy --control-port 192.168.0.41:7000 --debug --rmk user --launcher rsh --launcher-exec /usr/bin/rsh-redone-rsh --demux poll --pgid 0 --retries 2 --usize -2 --proxy-id 0 [mpiexec@mpi1] Launch arguments: /usr/bin/rsh-redone-rsh 192.168.0.42 "/mpi3/C3/mpich-3.0.4/bin/hydra_pmi_proxy" --control-port 192.168.0.41:7000--debug --rmk user --launcher rsh --launcher-exec /usr/bin/rsh-redone-rsh --demux poll --pgid 0 --retries 2 --usize -2 --proxy-id 1 [mpiexec@mpi1] Launch arguments: /usr/bin/rsh-redone-rsh 192.168.0.43 "/mpi3/C3/mpich-3.0.4/bin/hydra_pmi_proxy" --control-port 192.168.0.41:7000--debug --rmk user --launcher rsh --launcher-exec /usr/bin/rsh-redone-rsh --demux poll --pgid 0 --retries 2 --usize -2 --proxy-id 2 mpi1 mpi1 ^X^C[mpiexec@mpi1] Sending Ctrl-C to processes as requested [mpiexec@mpi1] Press Ctrl-C again to force abort [mpiexec@mpi1] HYDU_sock_write (./utils/sock/sock.c:291): write error (Bad file descriptor) [mpiexec@mpi1] HYD_pmcd_pmiserv_send_signal (./pm/pmiserv/pmiserv_cb.c:170): unable to write data to proxy [mpiexec@mpi1] ui_cmd_cb (./pm/pmiserv/pmiserv_pmci.c:79): unable to send signal downstream [mpiexec@mpi1] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status [mpiexec@mpi1] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:197): error waiting for event [mpiexec@mpi1] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion joni@mpi1:~$ echo $HYDRA_DEMUX select joni@mpi1:~$ sudo killall rsh-redone-rsh [sudo] password for joni: rsh-redone-rsh: ei prosesseja joni@mpi1:~$ Below blcr test after re-compile ====================== root@mpi1:/mpi3/S3/blcr-0.8.5# master kaak errors: 34 child 21967 completed #ST_ALARM:60 035 child 22072 is READY (context=SIGNAL stopped=YES) 036 child 22072 is STOPped !!! Alarm clock expired !!! Missing final DONE !!! Test killed unexpectedly by signal 9 FAIL: stopped.st root@ug:/mpi3/S3/blcr-0.8.5# slave kaak errors: root@kaak:/mpi3/S3/blcr-0.8.5# slave kaak errors: make check 34 child 7472 completed #ST_ALARM:60 035 child 7577 is READY (context=SIGNAL stopped=YES) 036 child 7577 is STOPped !!! Alarm clock expired !!! Missing final DONE !!! Test killed unexpectedly by signal 9 FAIL: stopped.st PASS: edeadlk.st /mpi3/S3/blcr-0.8.5/tests/.libs/lt-filedescriptors[8051]: file "filedescriptors.c", line 270, in check_stat_simple: File attributes changed. 1 mismatches /mpi3/S3/blcr-0.8.5/tests/.libs/lt-filedescriptors[8051]: file "crut.c", line 615, in crut_main: test_restart() unexpectedly returned -1 restart/nonzeroexit (255) FAIL: filedescriptors.ct 2013/8/27 Joni-Pekka Kurronen <joni.kurronen(a)gmail.com> > > > Another: > > joni@mpi1:/mpi3/S3/mpich-3.0.4$ mpiexec -np 6 hostname > host: 192.168.0.41 > host: 192.168.0.42 > host: 192.168.0.43 > > > ================================================================================================== > mpiexec options: > ---------------- > Base path: /mpi3/C3/mpich-3.0.4/bin/ > Launcher: (null) > Debug level: 1 > Enable X: -1 > > Global environment: > ------------------- > > MUMPS=/mpi3/S3/MUMPS_4.10.0 > LC_PAPER=fi_FI.UTF-8 > LC_ADDRESS=fi_FI.UTF-8 > SSH_AGENT_PID=12144 > LC_MONETARY=fi_FI.UTF-8 > MUMPS_I=/mpi3/C3/MUMPS_4.10.0 > HYDRA_DEMUX=select > GPG_AGENT_INFO=/tmp/keyring-kJwpJQ/gpg:0:1 > JPK_LMETISDIR_S5=/mpi3/S3/parmetis-4.0.2 > TERM=xterm > SHELL=/bin/bash > > XDG_SESSION_COOKIE=6d6390cb56a32b6678c10da600000412-1377606907.629665-1922379047 > FFT=/mpi3/C3/fftw2 > HYDRA_ENV=all > JPK_NETGEN=/mpi3/C3/netgen_668 > JPK_VER_S=S3 > HYDRA_CKPOINTLIB=blcr > HYDRA_CKPOINT_INTERVAL=10800 > WINDOWID=54602522 > LC_NUMERIC=fi_FI.UTF-8 > HYDRA_CKPOINT_PREFIX=/mpi3/chekpoint/default.chk > GNOME_KEYRING_CONTROL=/tmp/keyring-kJwpJQ > JPK_ELMER=/mpi3/C3/elmer_6283 > PARDISO_LIC_PATH=/mpi3/C3/pardiso > METIS_INCLUDE_DIR=/mpi3/C3/ParMetis-3.2.0 > JPK_NETGEN_S=/mpi3/S3/netgen_668 > USER=joni > > LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36: > JPK_TOGL=/mpi3/C3/Togl-1.7 > > LD_LIBRARY_PATH=/mpi3/C3/mpich-3.0.4/lib:/mpi3/C3/mpich-3.0.4/bin:/mpi3/C3/blcr-0.8.5/lib:/mpi3/C3/ParMetis-3.2.0:/mpi3/C3/acml5.3.1/gfortran64_mp/lib:/lib:/mpi3/C3/hypre-2.8.0b/lib:/mpi3/C3/scotch_6.0.0/lib:/mpi3/S3/MUMPS_4.10.0/lib:/mpi3/C3/acml5.3.1/gfortran64_mp/lib:/mpi3/C3/scalapack-2.0.2/lib:/mpi3/C3/hypre-2.8.0b/lib:/mpi3/C3/pardiso:/mpi3/C3/ParMetis-3.2.0:/mpi3/C3/ARPACK:/mpi3/C3/hdf5-1.8.10-patch1/lib:/mpi3/C3/VTK-5.8.0/lib/vtk-5.8:/mpi3/C3/elmer_6283/lib:/mpi3/C3/Togl-1.7:/mpi3/C3/netgen_668/lib:/usr/lib/:/usr/local/lib://mpi3/C3/vrpn/lib://mpi3/C3/hidapi/lib:/usr/include/libusb-1.0 > LC_TELEPHONE=fi_FI.UTF-8 > XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0 > JPK_OCC=/usr/include/oce > XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0 > HYDRA_HOST_FILE=/mpi4/hosts > SSH_AUTH_SOCK=/tmp/ssh-NnhxNTH12143/agent.12143 > SCOTCHDIR=/mpi3/C3/scotch_6.0.0 > HYDRA_LAUNCHER=rsh > JPK_VER_B=B3 > SESSION_MANAGER=local/mpi1:@ > /tmp/.ICE-unix/4284,unix/mpi1:/tmp/.ICE-unix/4284 > DEFAULTS_PATH=/usr/share/gconf/ubuntu.default.path > ELMER_HOME=/mpi3/C3/elmer_6283 > BLACS=/mpi3/C3/scalapack-2.0.2 > BLAS32=/mpi3/C3/acml5.3.1/gfortran64_mp > METIS_DIR= > MPI_LIBS=-L/mpi3/C3/mpich-3.0.4/lib -lmpich -lmpichf90 -lmpl -lopa > -lmpichcxx > XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/etc/xdg > JPK_MPI_DIR=/mpi3 > JPK_HDF5_S=/mpi3/S3/hdf5-1.8.10-patch1 > MPIEXEC_PORT_RANGE=7000:7500 > > PATH=/mpi3/C3/cmake-2.8.10.2/bin:/mpi3/C3/blcr-0.8.5/bin:/mpi3/C3/mpich-3.0.4/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/mpi3/C3/elmer_6283/bin:/mpi3/C3/elmer_6283/lib:/mpi3/C3/ParaView3 > DESKTOP_SESSION=ubuntu > BLAS=/mpi3/C3/acml5.3.1/gfortran64_mp > METIS_LIBDIR=/mpi3/C3/ParMetis-3.2.0 > CMAKE_COMMAND=/mpi3/C3/cmake-2.8.10.2/bin > QT_QMAKE_EXECUTABLE=/usr/bin/qmake-qt4 > LC_IDENTIFICATION=fi_FI.UTF-8 > JPK_SCOTCHDIR_S=/mpi3/S3/scotch_6.0.0_esmumps > JPK_LMETISDIR_S=/mpi3/S3/ParMetis-3.2.0 > PWD=/mpi3/S3/mpich-3.0.4 > NETGENDIR=/mpi3/C3/netgen_668/bin > EDITOR=nano > JPK_LMETISDIR=/mpi3/C3/ParMetis-3.2.0 > GNOME_KEYRING_PID=4273 > LANG=fi_FI.UTF-8 > MANDATORY_PATH=/usr/share/gconf/ubuntu.mandatory.path > OMPI_77=//mpi3/C3/mpich-3.0.4/bin/mpif77 > LC_MEASUREMENT=fi_FI.UTF-8 > JPK_HDF5=/mpi3/C3/hdf5-1.8.10-patch1 > UBUNTU_MENUPROXY=libappmenu.so > COMPIZ_CONFIG_PROFILE=ubuntu > ELMER_POST_HOME=/mpi3/C3/elmer_6283/bin > JPK_INS=/mpi3/C3 > ELMER_LIB=/mpi3/C3/elmer_6283/share/elmersolver/lib > HYDRA_PROXY_RETRY_COUNT=3 > GDMSESSION=ubuntu > JPK_ELMER_S=/mpi3/S3/elmer_6283 > JPK_LMETISDIR5=/mpi3/C3/parmetis-4.0.2 > JPK_LMETISDIR32=/mpi3/C3/ParMetis-3.2.0 > HYDRA_DEBUG=1 > JPK_BUI=/mpi3/S3 > VTK_INCLUDEPATH=/mpi3/C3/VTK-5.8.0/include > SHLVL=1 > HOME=/home/joni > OMPI_CC=//mpi3/C3/mpich-3.0.4/bin/mpicc > LANGUAGE=fi:en > OMPI_90=//mpi3/C3/mpich-3.0.4/bin/mpif90 > ELMERGUI_HOME=/mpi3/C3/elmer_6283/bin > GNOME_DESKTOP_SESSION_ID=this-is-deprecated > MPI_IMPLEMENTATION=mpich > MKL_SERIAL=YES > LOGNAME=joni > HYPRE=/mpi3/C3/hypre-2.8.0b > JPK_ARPACK_S=/mpi3/S3/ARPACK > JPK_JOBS=7 > JPK_VTK_DIR=/mpi3/C3/VTK-5.8.0 > SCALAPACK=/mpi3/C3/scalapack-2.0.2 > > XDG_DATA_DIRS=/usr/share/ubuntu:/usr/share/gnome:/usr/local/share/:/usr/share/ > > DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-gAYmjGaklf,guid=26c3f15a7a5ee55e8782415700000034 > JPK_ARPACK=/mpi3/C3/ARPACK > MPI_HOME=/mpi3/C3/mpich-3.0.4 > LESSOPEN=| /usr/bin/lesspipe %s > LACPACK=/mpi3/C3/acml5.3.1/gfortran64_mp > OMPI_CXX=//mpi3/C3/mpich-3.0.4/bin/mpicxx > OMP_NUM_THREADS=6 > JPK_TOGL_S=/mpi3/S3/Togl-1.7 > HYDRA_LAUNCHER_EXEC=/usr/bin/rsh-redone-rsh > JPK_MPICH2=/mpi3/C3/mpich-3.0.4 > PARDISO=/mpi3/C3/pardiso > PARDISOLICMESSAGE=1 > JPK_VER=C3 > XDG_CURRENT_DESKTOP=Unity > LESSCLOSE=/usr/bin/lesspipe %s %s > LC_TIME=fi_FI.UTF-8 > JPK_HYPRE_S=/mpi3/S3/hypre-2.8.0b > JPK_MPICH2_S=/mpi3/S3/mpich-3.0.4 > COLORTERM=gnome-terminal > XAUTHORITY=/home/joni/.Xauthority > LC_NAME=fi_FI.UTF-8 > _=/mpi3/C3/mpich-3.0.4/bin/mpiexec > OLDPWD=/home/joni > > Hydra internal environment: > --------------------------- > MPICH_ENABLE_CKPOINT=1 > GFORTRAN_UNBUFFERED_PRECONNECTED=y > > > Proxy information: > ********************* > [1] proxy: 192.168.0.41 (2 cores) > Exec list: hostname (2 processes); > > [2] proxy: 192.168.0.42 (2 cores) > Exec list: hostname (2 processes); > > [3] proxy: 192.168.0.43 (2 cores) > Exec list: hostname (2 processes); > > > > ================================================================================================== > > [mpiexec@mpi1] Timeout set to -1 (-1 means infinite) > [mpiexec@mpi1] Got a control port string of 192.168.0.41:7001 > > Proxy launch args: /mpi3/C3/mpich-3.0.4/bin/hydra_pmi_proxy --control-port > 192.168.0.41:7001 --debug --rmk user --launcher rsh --launcher-exec > /usr/bin/rsh-redone-rsh --demux poll --pgid 0 --retries 3 --usize -2 > --proxy-id > > Arguments being passed to proxy 0: > --version 3.0.4 --iface-ip-env-name MPICH_INTERFACE_HOSTNAME --hostname > 192.168.0.41 --global-core-map 0,2,6 --pmi-id-map 0,0 > --global-process-count 6 --auto-cleanup 1 --pmi-kvsname kvs_12243_0 > --pmi-process-mapping (vector,(0,3,2)) --ckpoint-prefix > /mpi3/chekpoint/default.chk --ckpoint-num -1 --global-inherited-env 121 > 'MUMPS=/mpi3/S3/MUMPS_4.10.0' 'LC_PAPER=fi_FI.UTF-8' > 'LC_ADDRESS=fi_FI.UTF-8' 'SSH_AGENT_PID=12144' 'LC_MONETARY=fi_FI.UTF-8' > 'MUMPS_I=/mpi3/C3/MUMPS_4.10.0' 'HYDRA_DEMUX=select' > 'GPG_AGENT_INFO=/tmp/keyring-kJwpJQ/gpg:0:1' > 'JPK_LMETISDIR_S5=/mpi3/S3/parmetis-4.0.2' 'TERM=xterm' 'SHELL=/bin/bash' > 'XDG_SESSION_COOKIE=6d6390cb56a32b6678c10da600000412-1377606907.629665-1922379047' > 'FFT=/mpi3/C3/fftw2' 'HYDRA_ENV=all' 'JPK_NETGEN=/mpi3/C3/netgen_668' > 'JPK_VER_S=S3' 'HYDRA_CKPOINTLIB=blcr' 'HYDRA_CKPOINT_INTERVAL=10800' > 'WINDOWID=54602522' 'LC_NUMERIC=fi_FI.UTF-8' > 'HYDRA_CKPOINT_PREFIX=/mpi3/chekpoint/default.chk' > 'GNOME_KEYRING_CONTROL=/tmp/keyring-kJwpJQ' 'JPK_ELMER=/mpi3/C3/elmer_6283' > 'PARDISO_LIC_PATH=/mpi3/C3/pardiso' > 'METIS_INCLUDE_DIR=/mpi3/C3/ParMetis-3.2.0' > 'JPK_NETGEN_S=/mpi3/S3/netgen_668' 'USER=joni' > 'LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:' > 'JPK_TOGL=/mpi3/C3/Togl-1.7' > 'LD_LIBRARY_PATH=/mpi3/C3/mpich-3.0.4/lib:/mpi3/C3/mpich-3.0.4/bin:/mpi3/C3/blcr-0.8.5/lib:/mpi3/C3/ParMetis-3.2.0:/mpi3/C3/acml5.3.1/gfortran64_mp/lib:/lib:/mpi3/C3/hypre-2.8.0b/lib:/mpi3/C3/scotch_6.0.0/lib:/mpi3/S3/MUMPS_4.10.0/lib:/mpi3/C3/acml5.3.1/gfortran64_mp/lib:/mpi3/C3/scalapack-2.0.2/lib:/mpi3/C3/hypre-2.8.0b/lib:/mpi3/C3/pardiso:/mpi3/C3/ParMetis-3.2.0:/mpi3/C3/ARPACK:/mpi3/C3/hdf5-1.8.10-patch1/lib:/mpi3/C3/VTK-5.8.0/lib/vtk-5.8:/mpi3/C3/elmer_6283/lib:/mpi3/C3/Togl-1.7:/mpi3/C3/netgen_668/lib:/usr/lib/:/usr/local/lib://mpi3/C3/vrpn/lib://mpi3/C3/hidapi/lib:/usr/include/libusb-1.0' > 'LC_TELEPHONE=fi_FI.UTF-8' > 'XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0' > 'JPK_OCC=/usr/include/oce' > 'XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0' > 'HYDRA_HOST_FILE=/mpi4/hosts' > 'SSH_AUTH_SOCK=/tmp/ssh-NnhxNTH12143/agent.12143' > 'SCOTCHDIR=/mpi3/C3/scotch_6.0.0' 'HYDRA_LAUNCHER=rsh' 'JPK_VER_B=B3' > 'SESSION_MANAGER=local/mpi1:@/tmp/.ICE-unix/4284,unix/mpi1:/tmp/.ICE-unix/4284' > 'DEFAULTS_PATH=/usr/share/gconf/ubuntu.default.path' > 'ELMER_HOME=/mpi3/C3/elmer_6283' 'BLACS=/mpi3/C3/scalapack-2.0.2' > 'BLAS32=/mpi3/C3/acml5.3.1/gfortran64_mp' 'METIS_DIR=' > 'MPI_LIBS=-L/mpi3/C3/mpich-3.0.4/lib -lmpich -lmpichf90 -lmpl -lopa > -lmpichcxx' 'XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/etc/xdg' > 'JPK_MPI_DIR=/mpi3' 'JPK_HDF5_S=/mpi3/S3/hdf5-1.8.10-patch1' > 'MPIEXEC_PORT_RANGE=7000:7500' > 'PATH=/mpi3/C3/cmake-2.8.10.2/bin:/mpi3/C3/blcr-0.8.5/bin:/mpi3/C3/mpich-3.0.4/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/mpi3/C3/elmer_6283/bin:/mpi3/C3/elmer_6283/lib:/mpi3/C3/ParaView3' > 'DESKTOP_SESSION=ubuntu' 'BLAS=/mpi3/C3/acml5.3.1/gfortran64_mp' > 'METIS_LIBDIR=/mpi3/C3/ParMetis-3.2.0' > 'CMAKE_COMMAND=/mpi3/C3/cmake-2.8.10.2/bin' > 'QT_QMAKE_EXECUTABLE=/usr/bin/qmake-qt4' 'LC_IDENTIFICATION=fi_FI.UTF-8' > 'JPK_SCOTCHDIR_S=/mpi3/S3/scotch_6.0.0_esmumps' > 'JPK_LMETISDIR_S=/mpi3/S3/ParMetis-3.2.0' 'PWD=/mpi3/S3/mpich-3.0.4' > 'NETGENDIR=/mpi3/C3/netgen_668/bin' 'EDITOR=nano' > 'JPK_LMETISDIR=/mpi3/C3/ParMetis-3.2.0' 'GNOME_KEYRING_PID=4273' > 'LANG=fi_FI.UTF-8' 'MANDATORY_PATH=/usr/share/gconf/ubuntu.mandatory.path' > 'OMPI_77=//mpi3/C3/mpich-3.0.4/bin/mpif77' 'LC_MEASUREMENT=fi_FI.UTF-8' > 'JPK_HDF5=/mpi3/C3/hdf5-1.8.10-patch1' 'UBUNTU_MENUPROXY=libappmenu.so' > 'COMPIZ_CONFIG_PROFILE=ubuntu' 'ELMER_POST_HOME=/mpi3/C3/elmer_6283/bin' > 'JPK_INS=/mpi3/C3' 'ELMER_LIB=/mpi3/C3/elmer_6283/share/elmersolver/lib' > 'HYDRA_PROXY_RETRY_COUNT=3' 'GDMSESSION=ubuntu' > 'JPK_ELMER_S=/mpi3/S3/elmer_6283' 'JPK_LMETISDIR5=/mpi3/C3/parmetis-4.0.2' > 'JPK_LMETISDIR32=/mpi3/C3/ParMetis-3.2.0' 'HYDRA_DEBUG=1' > 'JPK_BUI=/mpi3/S3' 'VTK_INCLUDEPATH=/mpi3/C3/VTK-5.8.0/include' 'SHLVL=1' > 'HOME=/home/joni' 'OMPI_CC=//mpi3/C3/mpich-3.0.4/bin/mpicc' > 'LANGUAGE=fi:en' 'OMPI_90=//mpi3/C3/mpich-3.0.4/bin/mpif90' > 'ELMERGUI_HOME=/mpi3/C3/elmer_6283/bin' > 'GNOME_DESKTOP_SESSION_ID=this-is-deprecated' 'MPI_IMPLEMENTATION=mpich' > 'MKL_SERIAL=YES' 'LOGNAME=joni' 'HYPRE=/mpi3/C3/hypre-2.8.0b' > 'JPK_ARPACK_S=/mpi3/S3/ARPACK' 'JPK_JOBS=7' > 'JPK_VTK_DIR=/mpi3/C3/VTK-5.8.0' 'SCALAPACK=/mpi3/C3/scalapack-2.0.2' > 'XDG_DATA_DIRS=/usr/share/ubuntu:/usr/share/gnome:/usr/local/share/:/usr/share/' > 'DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-gAYmjGaklf,guid=26c3f15a7a5ee55e8782415700000034' > 'JPK_ARPACK=/mpi3/C3/ARPACK' 'MPI_HOME=/mpi3/C3/mpich-3.0.4' 'LESSOPEN=| > /usr/bin/lesspipe %s' 'LACPACK=/mpi3/C3/acml5.3.1/gfortran64_mp' > 'OMPI_CXX=//mpi3/C3/mpich-3.0.4/bin/mpicxx' 'OMP_NUM_THREADS=6' > 'JPK_TOGL_S=/mpi3/S3/Togl-1.7' > 'HYDRA_LAUNCHER_EXEC=/usr/bin/rsh-redone-rsh' > 'JPK_MPICH2=/mpi3/C3/mpich-3.0.4' 'PARDISO=/mpi3/C3/pardiso' > 'PARDISOLICMESSAGE=1' 'JPK_VER=C3' 'XDG_CURRENT_DESKTOP=Unity' > 'LESSCLOSE=/usr/bin/lesspipe %s %s' 'LC_TIME=fi_FI.UTF-8' > 'JPK_HYPRE_S=/mpi3/S3/hypre-2.8.0b' 'JPK_MPICH2_S=/mpi3/S3/mpich-3.0.4' > 'COLORTERM=gnome-terminal' 'XAUTHORITY=/home/joni/.Xauthority' > 'LC_NAME=fi_FI.UTF-8' '_=/mpi3/C3/mpich-3.0.4/bin/mpiexec' > 'OLDPWD=/home/joni' --global-user-env 0 --global-system-env 2 > 'MPICH_ENABLE_CKPOINT=1' 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --genv-prop > all --proxy-core-count 2 --exec --exec-appnum 0 --exec-proc-count 2 > --exec-local-env 0 --exec-wdir /mpi3/S3/mpich-3.0.4 --exec-args 1 hostname > > Arguments being passed to proxy 1: > --version 3.0.4 --iface-ip-env-name MPICH_INTERFACE_HOSTNAME --hostname > 192.168.0.42 --global-core-map 0,2,6 --pmi-id-map 0,2 > --global-process-count 6 --auto-cleanup 1 --pmi-kvsname kvs_12243_0 > --pmi-process-mapping (vector,(0,3,2)) --ckpoint-prefix > /mpi3/chekpoint/default.chk --ckpoint-num -1 --global-inherited-env 121 > 'MUMPS=/mpi3/S3/MUMPS_4.10.0' 'LC_PAPER=fi_FI.UTF-8' > 'LC_ADDRESS=fi_FI.UTF-8' 'SSH_AGENT_PID=12144' 'LC_MONETARY=fi_FI.UTF-8' > 'MUMPS_I=/mpi3/C3/MUMPS_4.10.0' 'HYDRA_DEMUX=select' > 'GPG_AGENT_INFO=/tmp/keyring-kJwpJQ/gpg:0:1' > 'JPK_LMETISDIR_S5=/mpi3/S3/parmetis-4.0.2' 'TERM=xterm' 'SHELL=/bin/bash' > 'XDG_SESSION_COOKIE=6d6390cb56a32b6678c10da600000412-1377606907.629665-1922379047' > 'FFT=/mpi3/C3/fftw2' 'HYDRA_ENV=all' 'JPK_NETGEN=/mpi3/C3/netgen_668' > 'JPK_VER_S=S3' 'HYDRA_CKPOINTLIB=blcr' 'HYDRA_CKPOINT_INTERVAL=10800' > 'WINDOWID=54602522' 'LC_NUMERIC=fi_FI.UTF-8' > 'HYDRA_CKPOINT_PREFIX=/mpi3/chekpoint/default.chk' > 'GNOME_KEYRING_CONTROL=/tmp/keyring-kJwpJQ' 'JPK_ELMER=/mpi3/C3/elmer_6283' > 'PARDISO_LIC_PATH=/mpi3/C3/pardiso' > 'METIS_INCLUDE_DIR=/mpi3/C3/ParMetis-3.2.0' > 'JPK_NETGEN_S=/mpi3/S3/netgen_668' 'USER=joni' > 'LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:' > 'JPK_TOGL=/mpi3/C3/Togl-1.7' > 'LD_LIBRARY_PATH=/mpi3/C3/mpich-3.0.4/lib:/mpi3/C3/mpich-3.0.4/bin:/mpi3/C3/blcr-0.8.5/lib:/mpi3/C3/ParMetis-3.2.0:/mpi3/C3/acml5.3.1/gfortran64_mp/lib:/lib:/mpi3/C3/hypre-2.8.0b/lib:/mpi3/C3/scotch_6.0.0/lib:/mpi3/S3/MUMPS_4.10.0/lib:/mpi3/C3/acml5.3.1/gfortran64_mp/lib:/mpi3/C3/scalapack-2.0.2/lib:/mpi3/C3/hypre-2.8.0b/lib:/mpi3/C3/pardiso:/mpi3/C3/ParMetis-3.2.0:/mpi3/C3/ARPACK:/mpi3/C3/hdf5-1.8.10-patch1/lib:/mpi3/C3/VTK-5.8.0/lib/vtk-5.8:/mpi3/C3/elmer_6283/lib:/mpi3/C3/Togl-1.7:/mpi3/C3/netgen_668/lib:/usr/lib/:/usr/local/lib://mpi3/C3/vrpn/lib://mpi3/C3/hidapi/lib:/usr/include/libusb-1.0' > 'LC_TELEPHONE=fi_FI.UTF-8' > 'XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0' > 'JPK_OCC=/usr/include/oce' > 'XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0' > 'HYDRA_HOST_FILE=/mpi4/hosts' > 'SSH_AUTH_SOCK=/tmp/ssh-NnhxNTH12143/agent.12143' > 'SCOTCHDIR=/mpi3/C3/scotch_6.0.0' 'HYDRA_LAUNCHER=rsh' 'JPK_VER_B=B3' > 'SESSION_MANAGER=local/mpi1:@/tmp/.ICE-unix/4284,unix/mpi1:/tmp/.ICE-unix/4284' > 'DEFAULTS_PATH=/usr/share/gconf/ubuntu.default.path' > 'ELMER_HOME=/mpi3/C3/elmer_6283' 'BLACS=/mpi3/C3/scalapack-2.0.2' > 'BLAS32=/mpi3/C3/acml5.3.1/gfortran64_mp' 'METIS_DIR=' > 'MPI_LIBS=-L/mpi3/C3/mpich-3.0.4/lib -lmpich -lmpichf90 -lmpl -lopa > -lmpichcxx' 'XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/etc/xdg' > 'JPK_MPI_DIR=/mpi3' 'JPK_HDF5_S=/mpi3/S3/hdf5-1.8.10-patch1' > 'MPIEXEC_PORT_RANGE=7000:7500' > 'PATH=/mpi3/C3/cmake-2.8.10.2/bin:/mpi3/C3/blcr-0.8.5/bin:/mpi3/C3/mpich-3.0.4/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/mpi3/C3/elmer_6283/bin:/mpi3/C3/elmer_6283/lib:/mpi3/C3/ParaView3' > 'DESKTOP_SESSION=ubuntu' 'BLAS=/mpi3/C3/acml5.3.1/gfortran64_mp' > 'METIS_LIBDIR=/mpi3/C3/ParMetis-3.2.0' > 'CMAKE_COMMAND=/mpi3/C3/cmake-2.8.10.2/bin' > 'QT_QMAKE_EXECUTABLE=/usr/bin/qmake-qt4' 'LC_IDENTIFICATION=fi_FI.UTF-8' > 'JPK_SCOTCHDIR_S=/mpi3/S3/scotch_6.0.0_esmumps' > 'JPK_LMETISDIR_S=/mpi3/S3/ParMetis-3.2.0' 'PWD=/mpi3/S3/mpich-3.0.4' > 'NETGENDIR=/mpi3/C3/netgen_668/bin' 'EDITOR=nano' > 'JPK_LMETISDIR=/mpi3/C3/ParMetis-3.2.0' 'GNOME_KEYRING_PID=4273' > 'LANG=fi_FI.UTF-8' 'MANDATORY_PATH=/usr/share/gconf/ubuntu.mandatory.path' > 'OMPI_77=//mpi3/C3/mpich-3.0.4/bin/mpif77' 'LC_MEASUREMENT=fi_FI.UTF-8' > 'JPK_HDF5=/mpi3/C3/hdf5-1.8.10-patch1' 'UBUNTU_MENUPROXY=libappmenu.so' > 'COMPIZ_CONFIG_PROFILE=ubuntu' 'ELMER_POST_HOME=/mpi3/C3/elmer_6283/bin' > 'JPK_INS=/mpi3/C3' 'ELMER_LIB=/mpi3/C3/elmer_6283/share/elmersolver/lib' > 'HYDRA_PROXY_RETRY_COUNT=3' 'GDMSESSION=ubuntu' > 'JPK_ELMER_S=/mpi3/S3/elmer_6283' 'JPK_LMETISDIR5=/mpi3/C3/parmetis-4.0.2' > 'JPK_LMETISDIR32=/mpi3/C3/ParMetis-3.2.0' 'HYDRA_DEBUG=1' > 'JPK_BUI=/mpi3/S3' 'VTK_INCLUDEPATH=/mpi3/C3/VTK-5.8.0/include' 'SHLVL=1' > 'HOME=/home/joni' 'OMPI_CC=//mpi3/C3/mpich-3.0.4/bin/mpicc' > 'LANGUAGE=fi:en' 'OMPI_90=//mpi3/C3/mpich-3.0.4/bin/mpif90' > 'ELMERGUI_HOME=/mpi3/C3/elmer_6283/bin' > 'GNOME_DESKTOP_SESSION_ID=this-is-deprecated' 'MPI_IMPLEMENTATION=mpich' > 'MKL_SERIAL=YES' 'LOGNAME=joni' 'HYPRE=/mpi3/C3/hypre-2.8.0b' > 'JPK_ARPACK_S=/mpi3/S3/ARPACK' 'JPK_JOBS=7' > 'JPK_VTK_DIR=/mpi3/C3/VTK-5.8.0' 'SCALAPACK=/mpi3/C3/scalapack-2.0.2' > 'XDG_DATA_DIRS=/usr/share/ubuntu:/usr/share/gnome:/usr/local/share/:/usr/share/' > 'DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-gAYmjGaklf,guid=26c3f15a7a5ee55e8782415700000034' > 'JPK_ARPACK=/mpi3/C3/ARPACK' 'MPI_HOME=/mpi3/C3/mpich-3.0.4' 'LESSOPEN=| > /usr/bin/lesspipe %s' 'LACPACK=/mpi3/C3/acml5.3.1/gfortran64_mp' > 'OMPI_CXX=//mpi3/C3/mpich-3.0.4/bin/mpicxx' 'OMP_NUM_THREADS=6' > 'JPK_TOGL_S=/mpi3/S3/Togl-1.7' > 'HYDRA_LAUNCHER_EXEC=/usr/bin/rsh-redone-rsh' > 'JPK_MPICH2=/mpi3/C3/mpich-3.0.4' 'PARDISO=/mpi3/C3/pardiso' > 'PARDISOLICMESSAGE=1' 'JPK_VER=C3' 'XDG_CURRENT_DESKTOP=Unity' > 'LESSCLOSE=/usr/bin/lesspipe %s %s' 'LC_TIME=fi_FI.UTF-8' > 'JPK_HYPRE_S=/mpi3/S3/hypre-2.8.0b' 'JPK_MPICH2_S=/mpi3/S3/mpich-3.0.4' > 'COLORTERM=gnome-terminal' 'XAUTHORITY=/home/joni/.Xauthority' > 'LC_NAME=fi_FI.UTF-8' '_=/mpi3/C3/mpich-3.0.4/bin/mpiexec' > 'OLDPWD=/home/joni' --global-user-env 0 --global-system-env 2 > 'MPICH_ENABLE_CKPOINT=1' 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --genv-prop > all --proxy-core-count 2 --exec --exec-appnum 0 --exec-proc-count 2 > --exec-local-env 0 --exec-wdir /mpi3/S3/mpich-3.0.4 --exec-args 1 hostname > > Arguments being passed to proxy 2: > --version 3.0.4 --iface-ip-env-name MPICH_INTERFACE_HOSTNAME --hostname > 192.168.0.43 --global-core-map 0,2,6 --pmi-id-map 0,4 > --global-process-count 6 --auto-cleanup 1 --pmi-kvsname kvs_12243_0 > --pmi-process-mapping (vector,(0,3,2)) --ckpoint-prefix > /mpi3/chekpoint/default.chk --ckpoint-num -1 --global-inherited-env 121 > 'MUMPS=/mpi3/S3/MUMPS_4.10.0' 'LC_PAPER=fi_FI.UTF-8' > 'LC_ADDRESS=fi_FI.UTF-8' 'SSH_AGENT_PID=12144' 'LC_MONETARY=fi_FI.UTF-8' > 'MUMPS_I=/mpi3/C3/MUMPS_4.10.0' 'HYDRA_DEMUX=select' > 'GPG_AGENT_INFO=/tmp/keyring-kJwpJQ/gpg:0:1' > 'JPK_LMETISDIR_S5=/mpi3/S3/parmetis-4.0.2' 'TERM=xterm' 'SHELL=/bin/bash' > 'XDG_SESSION_COOKIE=6d6390cb56a32b6678c10da600000412-1377606907.629665-1922379047' > 'FFT=/mpi3/C3/fftw2' 'HYDRA_ENV=all' 'JPK_NETGEN=/mpi3/C3/netgen_668' > 'JPK_VER_S=S3' 'HYDRA_CKPOINTLIB=blcr' 'HYDRA_CKPOINT_INTERVAL=10800' > 'WINDOWID=54602522' 'LC_NUMERIC=fi_FI.UTF-8' > 'HYDRA_CKPOINT_PREFIX=/mpi3/chekpoint/default.chk' > 'GNOME_KEYRING_CONTROL=/tmp/keyring-kJwpJQ' 'JPK_ELMER=/mpi3/C3/elmer_6283' > 'PARDISO_LIC_PATH=/mpi3/C3/pardiso' > 'METIS_INCLUDE_DIR=/mpi3/C3/ParMetis-3.2.0' > 'JPK_NETGEN_S=/mpi3/S3/netgen_668' 'USER=joni' > 'LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:' > 'JPK_TOGL=/mpi3/C3/Togl-1.7' > 'LD_LIBRARY_PATH=/mpi3/C3/mpich-3.0.4/lib:/mpi3/C3/mpich-3.0.4/bin:/mpi3/C3/blcr-0.8.5/lib:/mpi3/C3/ParMetis-3.2.0:/mpi3/C3/acml5.3.1/gfortran64_mp/lib:/lib:/mpi3/C3/hypre-2.8.0b/lib:/mpi3/C3/scotch_6.0.0/lib:/mpi3/S3/MUMPS_4.10.0/lib:/mpi3/C3/acml5.3.1/gfortran64_mp/lib:/mpi3/C3/scalapack-2.0.2/lib:/mpi3/C3/hypre-2.8.0b/lib:/mpi3/C3/pardiso:/mpi3/C3/ParMetis-3.2.0:/mpi3/C3/ARPACK:/mpi3/C3/hdf5-1.8.10-patch1/lib:/mpi3/C3/VTK-5.8.0/lib/vtk-5.8:/mpi3/C3/elmer_6283/lib:/mpi3/C3/Togl-1.7:/mpi3/C3/netgen_668/lib:/usr/lib/:/usr/local/lib://mpi3/C3/vrpn/lib://mpi3/C3/hidapi/lib:/usr/include/libusb-1.0' > 'LC_TELEPHONE=fi_FI.UTF-8' > 'XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0' > 'JPK_OCC=/usr/include/oce' > 'XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0' > 'HYDRA_HOST_FILE=/mpi4/hosts' > 'SSH_AUTH_SOCK=/tmp/ssh-NnhxNTH12143/agent.12143' > 'SCOTCHDIR=/mpi3/C3/scotch_6.0.0' 'HYDRA_LAUNCHER=rsh' 'JPK_VER_B=B3' > 'SESSION_MANAGER=local/mpi1:@/tmp/.ICE-unix/4284,unix/mpi1:/tmp/.ICE-unix/4284' > 'DEFAULTS_PATH=/usr/share/gconf/ubuntu.default.path' > 'ELMER_HOME=/mpi3/C3/elmer_6283' 'BLACS=/mpi3/C3/scalapack-2.0.2' > 'BLAS32=/mpi3/C3/acml5.3.1/gfortran64_mp' 'METIS_DIR=' > 'MPI_LIBS=-L/mpi3/C3/mpich-3.0.4/lib -lmpich -lmpichf90 -lmpl -lopa > -lmpichcxx' 'XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/etc/xdg' > 'JPK_MPI_DIR=/mpi3' 'JPK_HDF5_S=/mpi3/S3/hdf5-1.8.10-patch1' > 'MPIEXEC_PORT_RANGE=7000:7500' > 'PATH=/mpi3/C3/cmake-2.8.10.2/bin:/mpi3/C3/blcr-0.8.5/bin:/mpi3/C3/mpich-3.0.4/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/mpi3/C3/elmer_6283/bin:/mpi3/C3/elmer_6283/lib:/mpi3/C3/ParaView3' > 'DESKTOP_SESSION=ubuntu' 'BLAS=/mpi3/C3/acml5.3.1/gfortran64_mp' > 'METIS_LIBDIR=/mpi3/C3/ParMetis-3.2.0' > 'CMAKE_COMMAND=/mpi3/C3/cmake-2.8.10.2/bin' > 'QT_QMAKE_EXECUTABLE=/usr/bin/qmake-qt4' 'LC_IDENTIFICATION=fi_FI.UTF-8' > 'JPK_SCOTCHDIR_S=/mpi3/S3/scotch_6.0.0_esmumps' > 'JPK_LMETISDIR_S=/mpi3/S3/ParMetis-3.2.0' 'PWD=/mpi3/S3/mpich-3.0.4' > 'NETGENDIR=/mpi3/C3/netgen_668/bin' 'EDITOR=nano' > 'JPK_LMETISDIR=/mpi3/C3/ParMetis-3.2.0' 'GNOME_KEYRING_PID=4273' > 'LANG=fi_FI.UTF-8' 'MANDATORY_PATH=/usr/share/gconf/ubuntu.mandatory.path' > 'OMPI_77=//mpi3/C3/mpich-3.0.4/bin/mpif77' 'LC_MEASUREMENT=fi_FI.UTF-8' > 'JPK_HDF5=/mpi3/C3/hdf5-1.8.10-patch1' 'UBUNTU_MENUPROXY=libappmenu.so' > 'COMPIZ_CONFIG_PROFILE=ubuntu' 'ELMER_POST_HOME=/mpi3/C3/elmer_6283/bin' > 'JPK_INS=/mpi3/C3' 'ELMER_LIB=/mpi3/C3/elmer_6283/share/elmersolver/lib' > 'HYDRA_PROXY_RETRY_COUNT=3' 'GDMSESSION=ubuntu' > 'JPK_ELMER_S=/mpi3/S3/elmer_6283' 'JPK_LMETISDIR5=/mpi3/C3/parmetis-4.0.2' > 'JPK_LMETISDIR32=/mpi3/C3/ParMetis-3.2.0' 'HYDRA_DEBUG=1' > 'JPK_BUI=/mpi3/S3' 'VTK_INCLUDEPATH=/mpi3/C3/VTK-5.8.0/include' 'SHLVL=1' > 'HOME=/home/joni' 'OMPI_CC=//mpi3/C3/mpich-3.0.4/bin/mpicc' > 'LANGUAGE=fi:en' 'OMPI_90=//mpi3/C3/mpich-3.0.4/bin/mpif90' > 'ELMERGUI_HOME=/mpi3/C3/elmer_6283/bin' > 'GNOME_DESKTOP_SESSION_ID=this-is-deprecated' 'MPI_IMPLEMENTATION=mpich' > 'MKL_SERIAL=YES' 'LOGNAME=joni' 'HYPRE=/mpi3/C3/hypre-2.8.0b' > 'JPK_ARPACK_S=/mpi3/S3/ARPACK' 'JPK_JOBS=7' > 'JPK_VTK_DIR=/mpi3/C3/VTK-5.8.0' 'SCALAPACK=/mpi3/C3/scalapack-2.0.2' > 'XDG_DATA_DIRS=/usr/share/ubuntu:/usr/share/gnome:/usr/local/share/:/usr/share/' > 'DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-gAYmjGaklf,guid=26c3f15a7a5ee55e8782415700000034' > 'JPK_ARPACK=/mpi3/C3/ARPACK' 'MPI_HOME=/mpi3/C3/mpich-3.0.4' 'LESSOPEN=| > /usr/bin/lesspipe %s' 'LACPACK=/mpi3/C3/acml5.3.1/gfortran64_mp' > 'OMPI_CXX=//mpi3/C3/mpich-3.0.4/bin/mpicxx' 'OMP_NUM_THREADS=6' > 'JPK_TOGL_S=/mpi3/S3/Togl-1.7' > 'HYDRA_LAUNCHER_EXEC=/usr/bin/rsh-redone-rsh' > 'JPK_MPICH2=/mpi3/C3/mpich-3.0.4' 'PARDISO=/mpi3/C3/pardiso' > 'PARDISOLICMESSAGE=1' 'JPK_VER=C3' 'XDG_CURRENT_DESKTOP=Unity' > 'LESSCLOSE=/usr/bin/lesspipe %s %s' 'LC_TIME=fi_FI.UTF-8' > 'JPK_HYPRE_S=/mpi3/S3/hypre-2.8.0b' 'JPK_MPICH2_S=/mpi3/S3/mpich-3.0.4' > 'COLORTERM=gnome-terminal' 'XAUTHORITY=/home/joni/.Xauthority' > 'LC_NAME=fi_FI.UTF-8' '_=/mpi3/C3/mpich-3.0.4/bin/mpiexec' > 'OLDPWD=/home/joni' --global-user-env 0 --global-system-env 2 > 'MPICH_ENABLE_CKPOINT=1' 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --genv-prop > all --proxy-core-count 2 --exec --exec-appnum 0 --exec-proc-count 2 > --exec-local-env 0 --exec-wdir /mpi3/S3/mpich-3.0.4 --exec-args 1 hostname > > [mpiexec@mpi1] Launch arguments: /mpi3/C3/mpich-3.0.4/bin/hydra_pmi_proxy > --control-port 192.168.0.41:7001 --debug --rmk user --launcher rsh > --launcher-exec /usr/bin/rsh-redone-rsh --demux poll --pgid 0 --retries 3 > --usize -2 --proxy-id 0 > [mpiexec@mpi1] Launch arguments: /usr/bin/rsh-redone-rsh 192.168.0.42 > "/mpi3/C3/mpich-3.0.4/bin/hydra_pmi_proxy" --control-port > 192.168.0.41:7001 --debug --rmk user --launcher rsh --launcher-exec > /usr/bin/rsh-redone-rsh --demux poll --pgid 0 --retries 3 --usize -2 > --proxy-id 1 > [mpiexec@mpi1] Launch arguments: /usr/bin/rsh-redone-rsh 192.168.0.43 > "/mpi3/C3/mpich-3.0.4/bin/hydra_pmi_proxy" --control-port > 192.168.0.41:7001 --debug --rmk user --launcher rsh --launcher-exec > /usr/bin/rsh-redone-rsh --demux poll --pgid 0 --retries 3 --usize -2 > --proxy-id 2 > mpi1 > m > > > 2013/8/27 Pavan Balaji <balaji(a)mcs.anl.gov> > >> >> Please don't drop discuss(a)mpich.org from the cc list. >> >> I doubt demux, --assert-level and blcr are relevant here. Also, the >> output of "make testing" is not helpful for us because those tests can fail >> even if your machines are too slow. >> >> Did you try my suggestion from the previous email? Could you try them >> and report back (just with that information)? >> >> -- Pavan >> >> >> On 08/27/2013 09:35 AM, Joni-Pekka Kurronen wrote: >> >>> >>> hi, >>> >>> I have allraedy cheked up. >>> This is not new install, just jumped up from mpich2 to 3 and >>> gcc 4.6 to gcc 4.7. >>> >>> I have rsh-redone-rsh as main but tested ssh as well,... >>> due at crash clinet's keep running at slave's and useing harddisk,... >>> >>> Following work's: >>> any machine alone >>> mpi1 and kaak or mpi1 and ugh >>> but not all to gether except hostname,... >>> >>> This could be realated: >>> - demux ( have tried select and poll, whit poll have to restart slave >>> machine's) >>> - nfs4 ( some reason nfs4 must be manually mounted at moment after >>> restart at slave's) >>> - have changed --assert-level to 0 (default 2) >>> - blcr >>> >>> >>> ch3:socket setting: >>> >>> ============= >>> hosts file,.. >>> 192.168.0.41:2 <http://192.168.0.41:2> >>> 192.168.0.42:2 <http://192.168.0.42:2> >>> #192.168.0.43:2 <http://192.168.0.43:2> >>> >>> ============= >>> summary.xml errors >>> <MPITEST> >>> <NAME>spawninfo1</NAME> >>> <NP>1</NP> >>> <WORKDIR>./spawn</WORKDIR> >>> <STATUS>fail</STATUS> >>> <TESTDIFF> >>> [mpiexec@mpi1] APPLICATION TIMED OUT >>> [proxy:0:0@mpi1] HYD_pmcd_pmip_control_cmd_cb >>> (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed >>> [proxy:0:0@mpi1] HYDT_dmxu_poll_wait_for_event >>> (./tools/demux/demux_poll.c:**77): callback returned error status >>> [proxy:0:0@mpi1] main (./pm/pmiserv/pmip.c:206): demux engine error >>> waiting for event >>> [proxy:1:0@mpi1] HYD_pmcd_pmip_control_cmd_cb >>> (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed >>> [proxy:1:0@mpi1] HYDT_dmxu_poll_wait_for_event >>> (./tools/demux/demux_poll.c:**77): callback returned error status >>> [proxy:1:0@mpi1] main (./pm/pmiserv/pmip.c:206): demux engine error >>> waiting for event >>> [mpiexec@mpi1] HYDT_bscu_wait_for_completion >>> (./tools/bootstrap/utils/bscu_**wait.c:76): one of the processes >>> terminated badly; aborting >>> [mpiexec@mpi1] HYDT_bsci_wait_for_completion >>> (./tools/bootstrap/src/bsci_**wait.c:23): launcher returned error >>> waiting >>> for completion >>> [mpiexec@mpi1] HYD_pmci_wait_for_completion >>> (./pm/pmiserv/pmiserv_pmci.c:**188): launcher returned error waiting for >>> completion >>> [mpiexec@mpi1] main (./ui/mpich/mpiexec.c:331): process manager error >>> waiting for completion >>> </TESTDIFF> >>> </MPITEST> >>> <MPITEST> >>> <NAME>rdwrord</NAME> >>> <NP>4</NP> >>> <WORKDIR>./io</WORKDIR> >>> <STATUS>fail</STATUS> >>> <TESTDIFF> >>> Fatal error in PMPI_Bcast: Other MPI error >>> [cli_2]: aborting job: >>> Fatal error in PMPI_Bcast: Other MPI error >>> Fatal error in PMPI_Bcast: Other MPI error >>> [cli_0]: aborting job: >>> Fatal error in PMPI_Bcast: Other MPI error >>> >>> ==============================**==============================** >>> ======================= >>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >>> = EXIT CODE: 1 >>> = CLEANING UP REMAINING PROCESSES >>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >>> ==============================**==============================** >>> ======================= >>> </TESTDIFF> >>> </MPITEST> >>> <MPITEST> >>> <NAME>rdwrzero</NAME> >>> <NP>4</NP> >>> <WORKDIR>./io</WORKDIR> >>> <STATUS>fail</STATUS> >>> <TESTDIFF> >>> Fatal error in PMPI_Bcast: Other MPI error >>> [cli_2]: aborting job: >>> Fatal error in PMPI_Bcast: Other MPI error >>> Fatal error in PMPI_Bcast: Other MPI error >>> [cli_0]: aborting job: >>> Fatal error in PMPI_Bcast: Other MPI error >>> >>> ==============================**==============================** >>> ======================= >>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >>> = EXIT CODE: 1 >>> = CLEANING UP REMAINING PROCESSES >>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >>> ==============================**==============================** >>> ======================= >>> </TESTDIFF> >>> </MPITEST> >>> <MPITEST> >>> <NAME>getextent</NAME> >>> <NP>2</NP> >>> <WORKDIR>./io</WORKDIR> >>> <STATUS>pass</STATUS> >>> </MPITEST> >>> <MPITEST> >>> <NAME>setinfo</NAME> >>> <NP>4</NP> >>> <WORKDIR>./io</WORKDIR> >>> <STATUS>fail</STATUS> >>> <TESTDIFF> >>> Fatal error in PMPI_Bcast: Other MPI error >>> [cli_2]: aborting job: >>> Fatal error in PMPI_Bcast: Other MPI error >>> Fatal error in PMPI_Bcast: Other MPI error >>> [cli_0]: aborting job: >>> Fatal error in PMPI_Bcast: Other MPI error >>> >>> ==============================**==============================** >>> ======================= >>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >>> = EXIT CODE: 1 >>> = CLEANING UP REMAINING PROCESSES >>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >>> ==============================**==============================** >>> ======================= >>> </TESTDIFF> >>> </MPITEST> >>> <MPITEST> >>> <NAME>setviewcur</NAME> >>> <NP>4</NP> >>> <WORKDIR>./io</WORKDIR> >>> <STATUS>fail</STATUS> >>> <TESTDIFF> >>> Fatal error in PMPI_Bcast: Other MPI error >>> [cli_2]: aborting job: >>> Fatal error in PMPI_Bcast: Other MPI error >>> Fatal error in PMPI_Bcast: Other MPI error >>> [cli_0]: aborting job: >>> Fatal error in PMPI_Bcast: Other MPI error >>> >>> ==============================**==============================** >>> ======================= >>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >>> = EXIT CODE: 1 >>> = CLEANING UP REMAINING PROCESSES >>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >>> ==============================**==============================** >>> ======================= >>> </TESTDIFF> >>> </MPITEST> >>> >>> .... >>> .... >>> .... >>> >>> >>> ============ >>> ============ >>> hosts file,.. >>> 192.168.0.41:2 <http://192.168.0.41:2> >>> 192.168.0.42:2 <http://192.168.0.42:2> >>> 192.168.0.43:2 <http://192.168.0.43:2> >>> >>> ============ >>> ============ >>> >>> When needed more than 4 process will hang,.. >>> >>> Unexpected output in allred3: [mpiexec@mpi1] APPLICATION TIMED OUT >>> Unexpected output in allred3: [proxy:0:0@mpi1] >>> HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886): assert >>> (!closed) failed >>> Unexpected output in allred3: [proxy:0:0@mpi1] >>> HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:**77): >>> callback >>> returned error status >>> Unexpected output in allred3: [proxy:0:0@mpi1] main >>> (./pm/pmiserv/pmip.c:206): demux engine error waiting for event >>> >>> ====== >>> >>> if useing ch3:nemesis hardisk is not running all the time as whit >>> sockets's,... >>> >>> >>> >>> ==============================**================== >>> This is read by rsh-redone-rsh for every process,... >>> ==============================**================== >>> #!/bin/bash >>> >>> # JPK-Integration for Ubuntu 12.4 LTS >>> # >>> https://sites.google.com/site/**jpsdatareviewstheboy007/** >>> ubuntu-lts-12-4-companion-**whit-ltsp-mpich2-elmer-**openfoam<https://sites.google.com/site/jpsdatareviewstheboy007/ubuntu-lts-12-4-compa…> >>> # >>> # CMAKE goes loop and can not build, cmake build under development says >>> documentation >>> >>> # gcc 4.7 >>> # bdver1 optimization >>> >>> shopt -s expand_aliases >>> export JPK_MPI_DIR=/mpi3 # MAIN DIRECTORY, SUBDIRCTORYES: >>> export JPK_VER=C3 # BINARY CODE >>> export JPK_VER_S=S3 # SOURCE CODE >>> export JPK_VER_B=B3 # BASH FILES TO COMPILE AND CONFIGURE >>> export JPK_INS=$JPK_MPI_DIR/$JPK_VER >>> export JPK_BUI=$JPK_MPI_DIR/$JPK_VER_**S >>> export JPK_ELMER=$JPK_INS/elmer_6283 #035 >>> export JPK_ELMER_S=$JPK_BUI/elmer_**6283 >>> export JPK_NETGEN_S=$JPK_BUI/netgen_**668 >>> export JPK_NETGEN=$JPK_INS/netgen_668 >>> >>> #GCC >>> #export JPK_FLAGS="-Wl,--no-as-needed -fPIC -DAdd_ -m64 -pthread -O3 >>> -fopenmp -lgomp -march=bdver1 -ftree-vectorize -funroll-loops" >>> #export CFLAGS="-Wl,--no-as-needed -fPIC -DAdd_ -m64 -pthread -fopenmp >>> -lgomp" >>> >>> # M A K E >>> >>> export JPK_JOBS=7 >>> >>> # O P E N MP >>> export OMP_NUM_THREADS=6 >>> >>> >>> # M P I C 3 >>> # http://wiki.mcs.anl.gov/**mpich2/index.php/Using_the_** >>> Hydra_Process_Manager<http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager> >>> export JPK_MPICH2_S=$JPK_BUI/mpich-3.**0.4 >>> export JPK_MPICH2=$JPK_INS/mpich-3.0.**4 >>> export PATH=$JPK_MPICH2/bin:$PATH >>> export MPI_HOME=$JPK_MPICH2 >>> export MPI_LIBS="-L$JPK_MPICH2/lib -lmpich -lmpichf90 -lmpl -lopa >>> -lmpichcxx" >>> export LD_LIBRARY_PATH=$JPK_MPICH2/**lib:$JPK_MPICH2/bin # FIRST >>> >>> # M P I >>> >>> export MPI_IMPLEMENTATION=mpich >>> >>> export OMPI_CC=/$JPK_MPI_DIR/$JPK_**VER/mpich-3.0.4/bin/mpicc >>> export OMPI_CXX=/$JPK_MPI_DIR/$JPK_**VER/mpich-3.0.4/bin/mpicxx >>> export OMPI_77=/$JPK_MPI_DIR/$JPK_**VER/mpich-3.0.4/bin/mpif77 >>> export OMPI_90=/$JPK_MPI_DIR/$JPK_**VER/mpich-3.0.4/bin/mpif90 >>> >>> # http://wiki.mcs.anl.gov/**mpich2/index.php/Using_the_** >>> Hydra_Process_Manager<http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager> >>> export HYDRA_DEBUG=0 >>> export HYDRA_HOST_FILE=/mpi4/hosts >>> export HYDRA_LAUNCHER=rsh >>> #export HYDRA_LAUNCHER=ssh >>> #export HYDRA_LAUNCHER_EXEC=/usr/bin/**netkit-rsh >>> export HYDRA_LAUNCHER_EXEC=/usr/bin/**rsh-redone-rsh >>> #export HYDRA_LAUNCHER_EXEC=/usr/bin/**ssh >>> export HYDRA_DEMUX=select >>> #export HYDRA_DEMUX=select #more porseses than core's >>> export HYDRA_PROXY_RETRY_COUNT=3 >>> #export HYDRA_RMK=pbs >>> #export HYDRA_DEFAULT_RMK=pbs >>> export HYDRA_ENV=all >>> export MPIEXEC_PORT_RANGE=7000:7500 >>> #mpirun -launcher rsh -launcher-exec /usr/bin/netkit-rsh -demux select >>> -n 21 ddd ./cpi >>> >>> # b l c r >>> >>> export HYDRA_CKPOINTLIB=blcr >>> export HYDRA_CKPOINT_PREFIX=/mpi3/**chekpoint/default.chk >>> export HYDRA_CKPOINT_INTERVAL=10800 >>> export PATH=$JPK_INS/blcr-0.8.5/bin:$**PATH >>> export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$JPK_INS/blcr-0.8.5/lib >>> #-ckpoint-num 5 >>> >>> # C M A K E - BUILD >>> export PATH=$JPK_INS/cmake-2.8.10.2/**bin:$PATH >>> export CMAKE_COMMAND=$JPK_INS/cmake-**2.8.10.2/bin >>> >>> # T O G L - UI netgen >>> >>> export JPK_TOGL="$JPK_INS/Togl-1.7" >>> export JPK_TOGL_S="$JPK_BUI/Togl-1.7" >>> >>> # OCC >>> export JPK_OCC=/usr/include/oce >>> >>> # M A T H >>> >>> export JPK_ARPACK_S=$JPK_BUI/ARPACK >>> export JPK_ARPACK=$JPK_INS/ARPACK >>> >>> export BLAS=$JPK_INS/acml5.3.1/**gfortran64_mp >>> export BLAS32=$JPK_INS/acml5.3.1/**gfortran64_mp >>> #export BLAS=$JPK_INS/clAmdBlas-1.10.**321/lib64 >>> #export BLAS32=$JPK_INS/clAmdBlas-1.**10.321/include >>> export FFT=$JPK_INS/fftw2 >>> export LACPACK=$BLAS >>> export SCALAPACK=$JPK_INS/scalapack-**2.0.2 >>> export BLACS=$SCALAPACK >>> >>> export JPK_LMETISDIR_S=$JPK_BUI/**ParMetis-3.2.0 >>> export JPK_LMETISDIR=$JPK_INS/**ParMetis-3.2.0 >>> export JPK_LMETISDIR32=$JPK_LMETISDIR >>> export JPK_LMETISDIR_S5=$JPK_BUI/**parmetis-4.0.2 >>> export JPK_LMETISDIR5=$JPK_INS/**parmetis-4.0.2 >>> >>> export METIS_DIR="" #$JPK_LMETISDIR MUST BE EMPTY >>> export METIS_INCLUDE_DIR=$JPK_**LMETISDIR >>> export METIS_LIBDIR=$JPK_LMETISDIR >>> export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$JPK_LMETISDIR:$BLAS/lib:** >>> $FTT/lib >>> #/mpi4/S/metis-5.0.2/GKlib >>> >>> export SCOTCHDIR=$JPK_INS/scotch_6.0.**0 >>> export JPK_SCOTCHDIR_S=$JPK_BUI/**scotch_6.0.0_esmumps >>> >>> export MUMPS_I=$JPK_INS/MUMPS_4.10.0 >>> export MUMPS=$JPK_BUI/MUMPS_4.10.0 >>> >>> export HYPRE=$JPK_INS/hypre-2.8.0b >>> export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$HYPRE/lib >>> export JPK_HYPRE_S=$JPK_BUI/hypre-2.**8.0b >>> >>> export PARDISOLICMESSAGE=1 >>> export PARDISO=$JPK_INS/pardiso >>> export PARDISO_LIC_PATH=$PARDISO >>> export MKL_SERIAL=YES >>> >>> export >>> LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$SCOTCHDIR/lib:$MUMPS/** >>> lib:$BLAS/lib:$SCALAPACK/lib:$**HYPRE/lib:$PARDISO:$METIS_** >>> LIBDIR:$JPK_ARPACK >>> >>> #HDF5 >>> #export JPK_HDF5_S=$JPK_BUI/hdf5-1.8.**10-patch1 for vtk testing >>> #export JPK_HDF5=$JPK_INS/hdf5-1.8.10-**patch1 >>> export JPK_HDF5_S=$JPK_BUI/hdf5-1.8.**10-patch1 >>> export JPK_HDF5=$JPK_INS/hdf5-1.8.10-**patch1 >>> export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$JPK_HDF5/lib >>> >>> # V T K >>> export JPK_VTK_DIR=$JPK_INS/VTK-5.8.0 >>> export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$JPK_VTK_DIR/lib/vtk-5.8 >>> export VTK_INCLUDEPATH=$JPK_VTK_DIR/**include >>> >>> # Q T >>> export QT_QMAKE_EXECUTABLE=/usr/bin/**qmake-qt4 >>> >>> # O P E N F O A M >>> # http://www.openfoam.org/**download/source.php<http://www.openfoam.org/download/source.php> >>> >>> #export WM_SCHEDULER=wmakeScheduler >>> #export WM_HOSTS="192.168.0.41:6 <http://192.168.0.41:6> 192.168.0.42:6 >>> <http://192.168.0.42:6> 192.168.0.43:6 <http://192.168.0.43:6>" >>> >>> #export WM_NCOMPPROCS=$($WM_SCHEDULER -count) >>> #export WM_COLOURS="black blue green cyan red magenta yellow" >>> >>> #export FOAM_INST_DIR=/mpi2/OpenFOAM >>> #foamDotFile=$FOAM_INST_DIR/**OpenFOAM-2.1.x/etc/bashrc >>> #[ -f $foamDotFile ] && . $foamDotFile >>> #source /mpi3/OpenFOAM/OpenFOAM-2.1.x/**etc/bashrc >>> >>> #export FOAM_RUN=/mpi2/om >>> #export OpenBIN=/mpi2/OpenFOAM/**OpenFOAM-2.1.x/bin/tools >>> #export PATH=OpenBIN$:$PATH >>> #export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:/apps/OpenFOAM/tools/**lib64 >>> >>> #export >>> ParaView_DIR=/mpi2/OpenFOAM/**ThirdParty-2.1.x/platforms/** >>> linux64Gcc/paraview-3.12.0 >>> #export PATH=$ParaView_DIR/bin:$PATH >>> #export PV_PLUGIN_PATH=$FOAM_LIBBIN/**paraview-3.12 >>> >>> # E L M E R >>> export ELMER_HOME=$JPK_ELMER >>> export ELMER_LIB=$JPK_ELMER/share/**elmersolver/lib >>> export PATH=$PATH:$ELMER_HOME/bin:$**ELMER_HOME/lib >>> export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$ELMER_HOME/lib >>> export ELMERGUI_HOME=$ELMER_HOME/bin >>> export ELMER_POST_HOME=$ELMER_HOME/**bin >>> >>> # S a l o m é >>> #cd /mpi2/salome-meca/SALOME-MECA-**2012.2-LGPL ; source >>> envSalomeMeca.sh >>> #cd ~/ >>> >>> # Paraview >>> #export PATH=$PATH:$JPK_INS/ParaView-**3.14.1-Linux-64bit >>> export PATH=$PATH:$JPK_INS/ParaView3 >>> >>> # N E T G E N P A R A L L E L $JPK_TCL/lib:$JPK_TK/lib: >>> #export >>> LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$JPK_TOGL:$JPK_NETGEN\** >>> par/lib:/usr/lib/ >>> #export NETGENDIR=$JPK_NETGEN\par/bin >>> # NETGEN >>> export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:$JPK_TOGL:$JPK_NETGEN/** >>> lib:/usr/lib/ >>> export NETGENDIR=$JPK_NETGEN/bin >>> >>> #crontab, ext editor >>> export EDITOR=nano >>> >>> #space ball >>> export LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:/usr/local/lib >>> >>> #vrpn & hidapi >>> export >>> LD_LIBRARY_PATH=$LD_LIBRARY_**PATH:/$JPK_MPI_DIR/$JPK_VER/** >>> vrpn/lib:/$JPK_MPI_DIR/$JPK_**VER/hidapi/lib:/usr/include/**libusb-1.0 >>> >>> >>> >>> >>> >>> 2013/8/27 Pavan Balaji <balaji(a)mcs.anl.gov <mailto:[email protected]>> >>> >>> >>> >>> This is almost certainly a network issue with your third machine >>> (kaak, I presume?). >>> >>> Thanks for making sure "hostname" works fine on all machines. That >>> means that your ssh connections are setup correctly. But a non-MPI >>> program, such as hostname, does not check the connection from kaak >>> back to mpi1. >>> >>> Can you try a simple program like "examples/cpi" in the build >>> directory on all machines? Try it on 2 machines (mpiexec -np 4) and >>> 3 machines (mpiexec -np 6). >>> >>> If the third machine is in fact having problems running the >>> application: >>> >>> 1. Make sure there's no firewall on the third machines. >>> >>> 2. Make sure the /etc/hosts file is consistent on both the machines >>> (mpi1 and kaak). >>> >>> -- Pavan >>> >>> >>> On 08/27/2013 06:46 AM, Joni-Pekka Kurronen wrote: >>> >>> >>> I have: >>> -Ubuntu 12.4 >>> -rsh-redo-rsh >>> -three machines >>> -mpich3 >>> -have tried export HYDRA_DEMUX=select / poll >>> -have tried ssh/rsh >>> -have added to LIBS: event_core event_pthreads >>> >>> I can run test at on to two machines whitout error but >>> when I take third machine to cluster demux engine goes mad,... >>> there is connection hanging,... and nothing happens,... >>> >>> >>> <MPITEST> >>> <NAME>uoplong</NAME> >>> <NP>11</NP> >>> <WORKDIR>./coll</WORKDIR> >>> <STATUS>fail</STATUS> >>> <TESTDIFF> >>> [mpiexec@mpi1] APPLICATION TIMED OUT >>> [proxy:0:0@mpi1] HYD_pmcd_pmip_control_cmd_cb >>> (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed >>> [proxy:0:0@mpi1] HYDT_dmxu_poll_wait_for_event >>> (./tools/demux/demux_poll.c:__**77): callback returned error >>> status >>> >>> [proxy:0:0@mpi1] main (./pm/pmiserv/pmip.c:206): demux engine >>> error >>> waiting for event >>> [mpiexec@mpi1] HYDT_bscu_wait_for_completion >>> (./tools/bootstrap/utils/bscu_**__wait.c:76): one of the >>> processes >>> >>> terminated badly; aborting >>> [mpiexec@mpi1] HYDT_bsci_wait_for_completion >>> (./tools/bootstrap/src/bsci___**wait.c:23): launcher returned >>> >>> error waiting >>> for completion >>> [mpiexec@mpi1] HYD_pmci_wait_for_completion >>> (./pm/pmiserv/pmiserv_pmci.c:_**_188): launcher returned error >>> >>> waiting for >>> completion >>> [mpiexec@mpi1] main (./ui/mpich/mpiexec.c:331): process manager >>> error >>> waiting for completion >>> </TESTDIFF> >>> </MPITEST> >>> >>> Also I can run >>> joni@mpi1:/mpi3/S3/hpcc-1.4.2$ mpiexec -np 6 hostname >>> mpi1 >>> mpi1 >>> ugh >>> ugh >>> kaak >>> kaak >>> >>> but if I run >>> joni@mpi1:/mpi3/S3/hpcc-1.4.2$ mpiexec -np 6 ls >>> I get only one directory as output and >>> system will cease until I have re-started slave machines ! >>> >>> >>> >>> >>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji >>> >>> >>> >>> >>> -- >>> Joni-Pekka Kurronen >>> Joni.Kurronen(a)gmail.com <mailto:Joni.Kurronen@gmail.**com<Joni.Kurronen(a)gmail.com> >>> > >>> >>> gsm. +358 50 521 2279 >>> >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> > > > > -- > Joni-Pekka Kurronen > > Joni.Kurronen(a)gmail.com > gsm. +358 50 521 2279 > -- Joni-Pekka Kurronen Joni.Kurronen(a)gmail.com gsm. +358 50 521 2279

1 0

Problem with build MPICH2.
by Anatoly G 28 Aug '13

28 Aug '13

Dear MPICH2. I want to build MPICH2 package. I have downloaded mpich2-1.5.tar.gz from http://www.mcs.anl.gov/research/projects/mpich2staging/goodell/downloads/in… My computer OS: Red Hat Enterprise Linux Server release 6.2 (Santiago) I created folder: /tmp/MPICH2/src there I executed: tar xfz mpich2-1.5.tar.gz and got new folder (/tmp/MPICH2/src/mpich2-1.5) with mpich2 files and sub-folders. Now I created folder: /tmp/MPICH2/build from directory /tmp/MPICH2/build, I executed: ../src/mpich2-1.5/configure --prefix=/tmp/MPICH2/install --enable-error-checking=runtime --enable-g=dbg CFLAGS=-fPIC CXXFLAGS=-fPIC FFLAGS=-fpic --enable-threads=runtime --enable-totalview --enable-static --disable-f77 --disable-fc --no-create --no-recursion | & tee c.txt Then I tried to execute (still from folder: /tmp/MPICH2/build): make but got an error: make: *** No targets specified and no makefile found. Stop. Then I tried to execute "ls" (still from folder: /tmp/MPICH2/build) and got following ouput: config.log config.lt config.status config.system c.txt lib libtool src Then I tried to execute "ls" (from folder: /tmp/MPICH2/src/mpich2-1.5) and got following ouput: -rw-r--r-- 1 anatol-g ws 43319 Oct 8 2012 aclocal.m4 -rwxr-xr-x 1 anatol-g ws 26809 Sep 27 2012 autogen.sh -rw-r--r-- 1 anatol-g ws 55773 Oct 8 2012 CHANGES drwxr-xr-x 2 anatol-g ws 4096 Oct 8 2012 confdb -rwxr-xr-x 1 anatol-g ws 1802576 Oct 8 2012 configure -rw-r--r-- 1 anatol-g ws 228771 Oct 4 2012 configure.ac drwxr-xr-x 3 anatol-g ws 4096 Oct 8 2012 contrib -rw-r--r-- 1 anatol-g ws 1439 Jul 17 2012 COPYRIGHT -rw-r--r-- 1 anatol-g ws 8076 Feb 27 2009 COPYRIGHT.rtf drwxr-xr-x 13 anatol-g ws 4096 Oct 8 2012 doc drwxr-xr-x 8 anatol-g ws 4096 Oct 8 2012 examples drwxr-xr-x 5 anatol-g ws 4096 Oct 8 2012 maint -rw-r--r-- 1 anatol-g ws 17216 Oct 4 2012 Makefile.am -rw-r--r-- 1 anatol-g ws 2579709 Oct 8 2012 Makefile.in -rwxr-xr-x 1 anatol-g ws 9886 Nov 12 2010 makewindist.bat drwxr-xr-x 4 anatol-g ws 4096 Oct 8 2012 man -rw-r--r-- 1 anatol-g ws 21620 Oct 5 2010 mpich2.def -rw-r--r-- 1 anatol-g ws 51983 May 2 2012 mpich2-doxygen.in -rw-r--r-- 1 anatol-g ws 556769 Nov 12 2010 mpich2.sln -rw-r--r-- 1 anatol-g ws 115749 Jan 24 2011 mpich2s.vcproj -rw-r--r-- 1 anatol-g ws 234598 Jan 24 2011 mpich2.vcproj -rw-r--r-- 1 anatol-g ws 15828 Oct 5 2010 mpi.def -rw-r--r-- 1 anatol-g ws 10010 May 29 2010 mpi.vcproj -rwxr-xr-x 1 anatol-g ws 4759 May 29 2010 mpl.vcproj -rw-r--r-- 1 anatol-g ws 38842 Oct 8 2012 README -rw-r--r-- 1 anatol-g ws 12689 Oct 8 2012 README.envvar -rw-r--r-- 1 anatol-g ws 20773 Aug 24 2012 README.winbin.rtf -rw-r--r-- 1 anatol-g ws 3758 Oct 3 2012 RELEASE_NOTES drwxr-xr-x 19 anatol-g ws 4096 Oct 8 2012 src -rw-r--r-- 1 anatol-g ws 2622 Oct 8 2012 subsys_include.m4 drwxr-xr-x 9 anatol-g ws 4096 Oct 8 2012 test -rwxr-xr-x 1 anatol-g ws 335084 Jul 10 2012 winconfigure.wsf drwxr-xr-x 4 anatol-g ws 4096 Oct 8 2012 www Can you please advice: - What I did wrong? (Why I can't perform make, according to manual). - Is my configure options are correct to build MPICH2 w/o any .so files? c.txt attached to mail. Regards, Anatoly.

1 0

Re: [mpich-discuss] ./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed
by Pavan Balaji 27 Aug '13

27 Aug '13

Please don't drop discuss(a)mpich.org from the cc list. I doubt demux, --assert-level and blcr are relevant here. Also, the output of "make testing" is not helpful for us because those tests can fail even if your machines are too slow. Did you try my suggestion from the previous email? Could you try them and report back (just with that information)? -- Pavan On 08/27/2013 09:35 AM, Joni-Pekka Kurronen wrote: > > hi, > > I have allraedy cheked up. > This is not new install, just jumped up from mpich2 to 3 and > gcc 4.6 to gcc 4.7. > > I have rsh-redone-rsh as main but tested ssh as well,... > due at crash clinet's keep running at slave's and useing harddisk,... > > Following work's: > any machine alone > mpi1 and kaak or mpi1 and ugh > but not all to gether except hostname,... > > This could be realated: > - demux ( have tried select and poll, whit poll have to restart slave > machine's) > - nfs4 ( some reason nfs4 must be manually mounted at moment after > restart at slave's) > - have changed --assert-level to 0 (default 2) > - blcr > > > ch3:socket setting: > > ============= > hosts file,.. > 192.168.0.41:2 <http://192.168.0.41:2> > 192.168.0.42:2 <http://192.168.0.42:2> > #192.168.0.43:2 <http://192.168.0.43:2> > ============= > summary.xml errors > <MPITEST> > <NAME>spawninfo1</NAME> > <NP>1</NP> > <WORKDIR>./spawn</WORKDIR> > <STATUS>fail</STATUS> > <TESTDIFF> > [mpiexec@mpi1] APPLICATION TIMED OUT > [proxy:0:0@mpi1] HYD_pmcd_pmip_control_cmd_cb > (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed > [proxy:0:0@mpi1] HYDT_dmxu_poll_wait_for_event > (./tools/demux/demux_poll.c:77): callback returned error status > [proxy:0:0@mpi1] main (./pm/pmiserv/pmip.c:206): demux engine error > waiting for event > [proxy:1:0@mpi1] HYD_pmcd_pmip_control_cmd_cb > (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed > [proxy:1:0@mpi1] HYDT_dmxu_poll_wait_for_event > (./tools/demux/demux_poll.c:77): callback returned error status > [proxy:1:0@mpi1] main (./pm/pmiserv/pmip.c:206): demux engine error > waiting for event > [mpiexec@mpi1] HYDT_bscu_wait_for_completion > (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes > terminated badly; aborting > [mpiexec@mpi1] HYDT_bsci_wait_for_completion > (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting > for completion > [mpiexec@mpi1] HYD_pmci_wait_for_completion > (./pm/pmiserv/pmiserv_pmci.c:188): launcher returned error waiting for > completion > [mpiexec@mpi1] main (./ui/mpich/mpiexec.c:331): process manager error > waiting for completion > </TESTDIFF> > </MPITEST> > <MPITEST> > <NAME>rdwrord</NAME> > <NP>4</NP> > <WORKDIR>./io</WORKDIR> > <STATUS>fail</STATUS> > <TESTDIFF> > Fatal error in PMPI_Bcast: Other MPI error > [cli_2]: aborting job: > Fatal error in PMPI_Bcast: Other MPI error > Fatal error in PMPI_Bcast: Other MPI error > [cli_0]: aborting job: > Fatal error in PMPI_Bcast: Other MPI error > > =================================================================================== > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = EXIT CODE: 1 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > </TESTDIFF> > </MPITEST> > <MPITEST> > <NAME>rdwrzero</NAME> > <NP>4</NP> > <WORKDIR>./io</WORKDIR> > <STATUS>fail</STATUS> > <TESTDIFF> > Fatal error in PMPI_Bcast: Other MPI error > [cli_2]: aborting job: > Fatal error in PMPI_Bcast: Other MPI error > Fatal error in PMPI_Bcast: Other MPI error > [cli_0]: aborting job: > Fatal error in PMPI_Bcast: Other MPI error > > =================================================================================== > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = EXIT CODE: 1 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > </TESTDIFF> > </MPITEST> > <MPITEST> > <NAME>getextent</NAME> > <NP>2</NP> > <WORKDIR>./io</WORKDIR> > <STATUS>pass</STATUS> > </MPITEST> > <MPITEST> > <NAME>setinfo</NAME> > <NP>4</NP> > <WORKDIR>./io</WORKDIR> > <STATUS>fail</STATUS> > <TESTDIFF> > Fatal error in PMPI_Bcast: Other MPI error > [cli_2]: aborting job: > Fatal error in PMPI_Bcast: Other MPI error > Fatal error in PMPI_Bcast: Other MPI error > [cli_0]: aborting job: > Fatal error in PMPI_Bcast: Other MPI error > > =================================================================================== > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = EXIT CODE: 1 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > </TESTDIFF> > </MPITEST> > <MPITEST> > <NAME>setviewcur</NAME> > <NP>4</NP> > <WORKDIR>./io</WORKDIR> > <STATUS>fail</STATUS> > <TESTDIFF> > Fatal error in PMPI_Bcast: Other MPI error > [cli_2]: aborting job: > Fatal error in PMPI_Bcast: Other MPI error > Fatal error in PMPI_Bcast: Other MPI error > [cli_0]: aborting job: > Fatal error in PMPI_Bcast: Other MPI error > > =================================================================================== > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = EXIT CODE: 1 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > </TESTDIFF> > </MPITEST> > > .... > .... > .... > > > ============ > ============ > hosts file,.. > 192.168.0.41:2 <http://192.168.0.41:2> > 192.168.0.42:2 <http://192.168.0.42:2> > 192.168.0.43:2 <http://192.168.0.43:2> > ============ > ============ > > When needed more than 4 process will hang,.. > > Unexpected output in allred3: [mpiexec@mpi1] APPLICATION TIMED OUT > Unexpected output in allred3: [proxy:0:0@mpi1] > HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886): assert > (!closed) failed > Unexpected output in allred3: [proxy:0:0@mpi1] > HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback > returned error status > Unexpected output in allred3: [proxy:0:0@mpi1] main > (./pm/pmiserv/pmip.c:206): demux engine error waiting for event > > ====== > > if useing ch3:nemesis hardisk is not running all the time as whit > sockets's,... > > > > ================================================ > This is read by rsh-redone-rsh for every process,... > ================================================ > #!/bin/bash > > # JPK-Integration for Ubuntu 12.4 LTS > # > https://sites.google.com/site/jpsdatareviewstheboy007/ubuntu-lts-12-4-compa… > # > # CMAKE goes loop and can not build, cmake build under development says > documentation > > # gcc 4.7 > # bdver1 optimization > > shopt -s expand_aliases > export JPK_MPI_DIR=/mpi3 # MAIN DIRECTORY, SUBDIRCTORYES: > export JPK_VER=C3 # BINARY CODE > export JPK_VER_S=S3 # SOURCE CODE > export JPK_VER_B=B3 # BASH FILES TO COMPILE AND CONFIGURE > export JPK_INS=$JPK_MPI_DIR/$JPK_VER > export JPK_BUI=$JPK_MPI_DIR/$JPK_VER_S > export JPK_ELMER=$JPK_INS/elmer_6283 #035 > export JPK_ELMER_S=$JPK_BUI/elmer_6283 > export JPK_NETGEN_S=$JPK_BUI/netgen_668 > export JPK_NETGEN=$JPK_INS/netgen_668 > > #GCC > #export JPK_FLAGS="-Wl,--no-as-needed -fPIC -DAdd_ -m64 -pthread -O3 > -fopenmp -lgomp -march=bdver1 -ftree-vectorize -funroll-loops" > #export CFLAGS="-Wl,--no-as-needed -fPIC -DAdd_ -m64 -pthread -fopenmp > -lgomp" > > # M A K E > > export JPK_JOBS=7 > > # O P E N MP > export OMP_NUM_THREADS=6 > > > # M P I C 3 > # http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager > export JPK_MPICH2_S=$JPK_BUI/mpich-3.0.4 > export JPK_MPICH2=$JPK_INS/mpich-3.0.4 > export PATH=$JPK_MPICH2/bin:$PATH > export MPI_HOME=$JPK_MPICH2 > export MPI_LIBS="-L$JPK_MPICH2/lib -lmpich -lmpichf90 -lmpl -lopa > -lmpichcxx" > export LD_LIBRARY_PATH=$JPK_MPICH2/lib:$JPK_MPICH2/bin # FIRST > > # M P I > > export MPI_IMPLEMENTATION=mpich > > export OMPI_CC=/$JPK_MPI_DIR/$JPK_VER/mpich-3.0.4/bin/mpicc > export OMPI_CXX=/$JPK_MPI_DIR/$JPK_VER/mpich-3.0.4/bin/mpicxx > export OMPI_77=/$JPK_MPI_DIR/$JPK_VER/mpich-3.0.4/bin/mpif77 > export OMPI_90=/$JPK_MPI_DIR/$JPK_VER/mpich-3.0.4/bin/mpif90 > > # http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager > export HYDRA_DEBUG=0 > export HYDRA_HOST_FILE=/mpi4/hosts > export HYDRA_LAUNCHER=rsh > #export HYDRA_LAUNCHER=ssh > #export HYDRA_LAUNCHER_EXEC=/usr/bin/netkit-rsh > export HYDRA_LAUNCHER_EXEC=/usr/bin/rsh-redone-rsh > #export HYDRA_LAUNCHER_EXEC=/usr/bin/ssh > export HYDRA_DEMUX=select > #export HYDRA_DEMUX=select #more porseses than core's > export HYDRA_PROXY_RETRY_COUNT=3 > #export HYDRA_RMK=pbs > #export HYDRA_DEFAULT_RMK=pbs > export HYDRA_ENV=all > export MPIEXEC_PORT_RANGE=7000:7500 > #mpirun -launcher rsh -launcher-exec /usr/bin/netkit-rsh -demux select > -n 21 ddd ./cpi > > # b l c r > > export HYDRA_CKPOINTLIB=blcr > export HYDRA_CKPOINT_PREFIX=/mpi3/chekpoint/default.chk > export HYDRA_CKPOINT_INTERVAL=10800 > export PATH=$JPK_INS/blcr-0.8.5/bin:$PATH > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JPK_INS/blcr-0.8.5/lib > #-ckpoint-num 5 > > # C M A K E - BUILD > export PATH=$JPK_INS/cmake-2.8.10.2/bin:$PATH > export CMAKE_COMMAND=$JPK_INS/cmake-2.8.10.2/bin > > # T O G L - UI netgen > > export JPK_TOGL="$JPK_INS/Togl-1.7" > export JPK_TOGL_S="$JPK_BUI/Togl-1.7" > > # OCC > export JPK_OCC=/usr/include/oce > > # M A T H > > export JPK_ARPACK_S=$JPK_BUI/ARPACK > export JPK_ARPACK=$JPK_INS/ARPACK > > export BLAS=$JPK_INS/acml5.3.1/gfortran64_mp > export BLAS32=$JPK_INS/acml5.3.1/gfortran64_mp > #export BLAS=$JPK_INS/clAmdBlas-1.10.321/lib64 > #export BLAS32=$JPK_INS/clAmdBlas-1.10.321/include > export FFT=$JPK_INS/fftw2 > export LACPACK=$BLAS > export SCALAPACK=$JPK_INS/scalapack-2.0.2 > export BLACS=$SCALAPACK > > export JPK_LMETISDIR_S=$JPK_BUI/ParMetis-3.2.0 > export JPK_LMETISDIR=$JPK_INS/ParMetis-3.2.0 > export JPK_LMETISDIR32=$JPK_LMETISDIR > export JPK_LMETISDIR_S5=$JPK_BUI/parmetis-4.0.2 > export JPK_LMETISDIR5=$JPK_INS/parmetis-4.0.2 > > export METIS_DIR="" #$JPK_LMETISDIR MUST BE EMPTY > export METIS_INCLUDE_DIR=$JPK_LMETISDIR > export METIS_LIBDIR=$JPK_LMETISDIR > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JPK_LMETISDIR:$BLAS/lib:$FTT/lib > #/mpi4/S/metis-5.0.2/GKlib > > export SCOTCHDIR=$JPK_INS/scotch_6.0.0 > export JPK_SCOTCHDIR_S=$JPK_BUI/scotch_6.0.0_esmumps > > export MUMPS_I=$JPK_INS/MUMPS_4.10.0 > export MUMPS=$JPK_BUI/MUMPS_4.10.0 > > export HYPRE=$JPK_INS/hypre-2.8.0b > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HYPRE/lib > export JPK_HYPRE_S=$JPK_BUI/hypre-2.8.0b > > export PARDISOLICMESSAGE=1 > export PARDISO=$JPK_INS/pardiso > export PARDISO_LIC_PATH=$PARDISO > export MKL_SERIAL=YES > > export > LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$SCOTCHDIR/lib:$MUMPS/lib:$BLAS/lib:$SCALAPACK/lib:$HYPRE/lib:$PARDISO:$METIS_LIBDIR:$JPK_ARPACK > > #HDF5 > #export JPK_HDF5_S=$JPK_BUI/hdf5-1.8.10-patch1 for vtk testing > #export JPK_HDF5=$JPK_INS/hdf5-1.8.10-patch1 > export JPK_HDF5_S=$JPK_BUI/hdf5-1.8.10-patch1 > export JPK_HDF5=$JPK_INS/hdf5-1.8.10-patch1 > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JPK_HDF5/lib > > # V T K > export JPK_VTK_DIR=$JPK_INS/VTK-5.8.0 > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JPK_VTK_DIR/lib/vtk-5.8 > export VTK_INCLUDEPATH=$JPK_VTK_DIR/include > > # Q T > export QT_QMAKE_EXECUTABLE=/usr/bin/qmake-qt4 > > # O P E N F O A M > # http://www.openfoam.org/download/source.php > > #export WM_SCHEDULER=wmakeScheduler > #export WM_HOSTS="192.168.0.41:6 <http://192.168.0.41:6> 192.168.0.42:6 > <http://192.168.0.42:6> 192.168.0.43:6 <http://192.168.0.43:6>" > #export WM_NCOMPPROCS=$($WM_SCHEDULER -count) > #export WM_COLOURS="black blue green cyan red magenta yellow" > > #export FOAM_INST_DIR=/mpi2/OpenFOAM > #foamDotFile=$FOAM_INST_DIR/OpenFOAM-2.1.x/etc/bashrc > #[ -f $foamDotFile ] && . $foamDotFile > #source /mpi3/OpenFOAM/OpenFOAM-2.1.x/etc/bashrc > > #export FOAM_RUN=/mpi2/om > #export OpenBIN=/mpi2/OpenFOAM/OpenFOAM-2.1.x/bin/tools > #export PATH=OpenBIN$:$PATH > #export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/apps/OpenFOAM/tools/lib64 > > #export > ParaView_DIR=/mpi2/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc/paraview-3.12.0 > #export PATH=$ParaView_DIR/bin:$PATH > #export PV_PLUGIN_PATH=$FOAM_LIBBIN/paraview-3.12 > > # E L M E R > export ELMER_HOME=$JPK_ELMER > export ELMER_LIB=$JPK_ELMER/share/elmersolver/lib > export PATH=$PATH:$ELMER_HOME/bin:$ELMER_HOME/lib > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ELMER_HOME/lib > export ELMERGUI_HOME=$ELMER_HOME/bin > export ELMER_POST_HOME=$ELMER_HOME/bin > > # S a l o m ï¿½ > #cd /mpi2/salome-meca/SALOME-MECA-2012.2-LGPL ; source envSalomeMeca.sh > #cd ~/ > > # Paraview > #export PATH=$PATH:$JPK_INS/ParaView-3.14.1-Linux-64bit > export PATH=$PATH:$JPK_INS/ParaView3 > > # N E T G E N P A R A L L E L $JPK_TCL/lib:$JPK_TK/lib: > #export > LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JPK_TOGL:$JPK_NETGEN\par/lib:/usr/lib/ > #export NETGENDIR=$JPK_NETGEN\par/bin > # NETGEN > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JPK_TOGL:$JPK_NETGEN/lib:/usr/lib/ > export NETGENDIR=$JPK_NETGEN/bin > > #crontab, ext editor > export EDITOR=nano > > #space ball > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib > > #vrpn & hidapi > export > LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/$JPK_MPI_DIR/$JPK_VER/vrpn/lib:/$JPK_MPI_DIR/$JPK_VER/hidapi/lib:/usr/include/libusb-1.0 > > > > > > 2013/8/27 Pavan Balaji <balaji(a)mcs.anl.gov <mailto:[email protected]>> > > > This is almost certainly a network issue with your third machine > (kaak, I presume?). > > Thanks for making sure "hostname" works fine on all machines. That > means that your ssh connections are setup correctly. But a non-MPI > program, such as hostname, does not check the connection from kaak > back to mpi1. > > Can you try a simple program like "examples/cpi" in the build > directory on all machines? Try it on 2 machines (mpiexec -np 4) and > 3 machines (mpiexec -np 6). > > If the third machine is in fact having problems running the application: > > 1. Make sure there's no firewall on the third machines. > > 2. Make sure the /etc/hosts file is consistent on both the machines > (mpi1 and kaak). > > -- Pavan > > > On 08/27/2013 06:46 AM, Joni-Pekka Kurronen wrote: > > > I have: > -Ubuntu 12.4 > -rsh-redo-rsh > -three machines > -mpich3 > -have tried export HYDRA_DEMUX=select / poll > -have tried ssh/rsh > -have added to LIBS: event_core event_pthreads > > I can run test at on to two machines whitout error but > when I take third machine to cluster demux engine goes mad,... > there is connection hanging,... and nothing happens,... > > > <MPITEST> > <NAME>uoplong</NAME> > <NP>11</NP> > <WORKDIR>./coll</WORKDIR> > <STATUS>fail</STATUS> > <TESTDIFF> > [mpiexec@mpi1] APPLICATION TIMED OUT > [proxy:0:0@mpi1] HYD_pmcd_pmip_control_cmd_cb > (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed > [proxy:0:0@mpi1] HYDT_dmxu_poll_wait_for_event > (./tools/demux/demux_poll.c:__77): callback returned error status > [proxy:0:0@mpi1] main (./pm/pmiserv/pmip.c:206): demux engine error > waiting for event > [mpiexec@mpi1] HYDT_bscu_wait_for_completion > (./tools/bootstrap/utils/bscu___wait.c:76): one of the processes > terminated badly; aborting > [mpiexec@mpi1] HYDT_bsci_wait_for_completion > (./tools/bootstrap/src/bsci___wait.c:23): launcher returned > error waiting > for completion > [mpiexec@mpi1] HYD_pmci_wait_for_completion > (./pm/pmiserv/pmiserv_pmci.c:__188): launcher returned error > waiting for > completion > [mpiexec@mpi1] main (./ui/mpich/mpiexec.c:331): process manager > error > waiting for completion > </TESTDIFF> > </MPITEST> > > Also I can run > joni@mpi1:/mpi3/S3/hpcc-1.4.2$ mpiexec -np 6 hostname > mpi1 > mpi1 > ugh > ugh > kaak > kaak > > but if I run > joni@mpi1:/mpi3/S3/hpcc-1.4.2$ mpiexec -np 6 ls > I get only one directory as output and > system will cease until I have re-started slave machines ! > > > > > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > > > > -- > Joni-Pekka Kurronen > Joni.Kurronen(a)gmail.com <mailto:[email protected]> > gsm. +358 50 521 2279 -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] ./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed
by Pavan Balaji 27 Aug '13

27 Aug '13

This is almost certainly a network issue with your third machine (kaak, I presume?). Thanks for making sure "hostname" works fine on all machines. That means that your ssh connections are setup correctly. But a non-MPI program, such as hostname, does not check the connection from kaak back to mpi1. Can you try a simple program like "examples/cpi" in the build directory on all machines? Try it on 2 machines (mpiexec -np 4) and 3 machines (mpiexec -np 6). If the third machine is in fact having problems running the application: 1. Make sure there's no firewall on the third machines. 2. Make sure the /etc/hosts file is consistent on both the machines (mpi1 and kaak). -- Pavan On 08/27/2013 06:46 AM, Joni-Pekka Kurronen wrote: > > I have: > -Ubuntu 12.4 > -rsh-redo-rsh > -three machines > -mpich3 > -have tried export HYDRA_DEMUX=select / poll > -have tried ssh/rsh > -have added to LIBS: event_core event_pthreads > > I can run test at on to two machines whitout error but > when I take third machine to cluster demux engine goes mad,... > there is connection hanging,... and nothing happens,... > > > <MPITEST> > <NAME>uoplong</NAME> > <NP>11</NP> > <WORKDIR>./coll</WORKDIR> > <STATUS>fail</STATUS> > <TESTDIFF> > [mpiexec@mpi1] APPLICATION TIMED OUT > [proxy:0:0@mpi1] HYD_pmcd_pmip_control_cmd_cb > (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed > [proxy:0:0@mpi1] HYDT_dmxu_poll_wait_for_event > (./tools/demux/demux_poll.c:77): callback returned error status > [proxy:0:0@mpi1] main (./pm/pmiserv/pmip.c:206): demux engine error > waiting for event > [mpiexec@mpi1] HYDT_bscu_wait_for_completion > (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes > terminated badly; aborting > [mpiexec@mpi1] HYDT_bsci_wait_for_completion > (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting > for completion > [mpiexec@mpi1] HYD_pmci_wait_for_completion > (./pm/pmiserv/pmiserv_pmci.c:188): launcher returned error waiting for > completion > [mpiexec@mpi1] main (./ui/mpich/mpiexec.c:331): process manager error > waiting for completion > </TESTDIFF> > </MPITEST> > > Also I can run > joni@mpi1:/mpi3/S3/hpcc-1.4.2$ mpiexec -np 6 hostname > mpi1 > mpi1 > ugh > ugh > kaak > kaak > > but if I run > joni@mpi1:/mpi3/S3/hpcc-1.4.2$ mpiexec -np 6 ls > I get only one directory as output and > system will cease until I have re-started slave machines ! > > > > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed
by Joni-Pekka Kurronen 27 Aug '13

27 Aug '13

I have: -Ubuntu 12.4 -rsh-redo-rsh -three machines -mpich3 -have tried export HYDRA_DEMUX=select / poll -have tried ssh/rsh -have added to LIBS: event_core event_pthreads I can run test at on to two machines whitout error but when I take third machine to cluster demux engine goes mad,... there is connection hanging,... and nothing happens,... <MPITEST> <NAME>uoplong</NAME> <NP>11</NP> <WORKDIR>./coll</WORKDIR> <STATUS>fail</STATUS> <TESTDIFF> [mpiexec@mpi1] APPLICATION TIMED OUT [proxy:0:0@mpi1] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed [proxy:0:0@mpi1] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status [proxy:0:0@mpi1] main (./pm/pmiserv/pmip.c:206): demux engine error waiting for event [mpiexec@mpi1] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting [mpiexec@mpi1] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion [mpiexec@mpi1] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:188): launcher returned error waiting for completion [mpiexec@mpi1] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion </TESTDIFF> </MPITEST> Also I can run joni@mpi1:/mpi3/S3/hpcc-1.4.2$ mpiexec -np 6 hostname mpi1 mpi1 ugh ugh kaak kaak but if I run joni@mpi1:/mpi3/S3/hpcc-1.4.2$ mpiexec -np 6 ls I get only one directory as output and system will cease until I have re-started slave machines ! -- Joni-Pekka Kurronen AMRINA, Insinï¿½ï¿½ri AMK veneala

1 0

Re: [mpich-discuss] Single slow mpi process
by Pavan Balaji 26 Aug '13

26 Aug '13

This is very surprising. We try to keep the mpicc compile mode as similar to the native compiler as possible. What's the output of: % mpicc -show Are the compile flags very different? -- Pavan On 08/26/2013 05:29 AM, Bob Ilgner wrote: > Dear all, > When adding #include "mpi.h" and building a single process with mpi, I > find that the single process mpi job runs very much slower than a single > thread process built with gcc. i.e. 150 to 200% slower, and run as a > standalone executable. > That is, for example > gcc mycode.c -o mycode.exe > ./mycode > will run in 100 seconds. > Adding #include "mpi.h" to mycode.c > mpicc mycode.c -o mycode.exe > mpirun -n 1 mycode.exe > will take 180 to 400 seconds to run. > This varies from system to system on intel, Sparc or IBM based systems. > Can anyone explain this offhand ? Has anyone else experienced this ? > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Single slow mpi process
by Reuti 26 Aug '13

26 Aug '13

Ups - sorry. Mixed the two MPI lists I subscribed to... Please ignore this post. Am 26.08.2013 um 12:43 schrieb Reuti: > Am 26.08.2013 um 12:29 schrieb Bob Ilgner: > >> Dear all, >> >> When adding #include "mpi.h" and building a single process with mpi, I find that the single process mpi job runs very much slower than a single thread process built with gcc. i.e. 150 to 200% slower, and run as a standalone executable. >> >> That is, for example >> >> gcc mycode.c -o mycode.exe >> ./mycode >> >> >> will run in 100 seconds. >> >> >> Adding #include "mpi.h" to mycode.c >> >> mpicc mycode.c -o mycode.exe > > `mpicc -showme ...` will show the options (and used compiler). Maybe the optimization options are set differently > > How fast is the application in case you start the Open MPI compilation without `mpirun` but just execute it in the usual way? > > -- Reuti > > >> mpirun -n 1 mycode.exe >> >> >> will take 180 to 400 seconds to run. >> >> This varies from system to system on intel, Sparc or IBM based systems. >> >> Can anyone explain this offhand ? Has anyone else experienced this ? >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >

1 0