April 2016 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] ImportError: No module named mpdlib
by Halim Amer 28 Apr '16

28 Apr '16

John, MPD has been deprecated for a long time and got completely deleted since MPICH 3.1.1. I guess mpiexec at <somedir>/Contents/engine/bin/mpiexec comes from an old MPICH that uses MPD. The version you compiled should have been built by default with the Hydra process manager but it was installed in a different folder (/somedir/mpich-3.0.4-shared) than the one showing bugs. You can change your environment to prioritize loading /somedir/mpich-3.0.4-shared over <somedir>/Contents/engine/bin/ or wipe out the MPICH installation in the latter folder. To make sure that MPD is not being built, you can specify --with-pm=hydra during configure or just use a more recent MPICH version >= 3.1.1 or the most recent MPICH 3.2. --Halim www.mcs.anl.gov/~aamer On 4/28/16 5:42 PM, JR Cary wrote: > One of our users saw: > > Traceback (most recent call last): > > File "<somedir>/Contents/engine/bin/mpiexec", line 186, in > import mpdlib > ImportError: No module named mpdlib > > Anyone seen this before? > > mpich-3.0.4, compiled with > > > ../configure' \ > --prefix=/somedir/mpich-3.0.4-shared \ > --enable-shared \ > --disable-static \ > CC='/usr/bin/gcc' \ > CXX='/usr/bin/g++' \ > FC='/usr/bin/gfortran' \ > F77='/usr/bin/gfortran' \ > CFLAGS='-pipe -fPIC' \ > CXXFLAGS='-pipe -fPIC' \ > FFLAGS='-fPIC' \ > FCFLAGS='-fPIC' > > mpiexec is a softlink to mpiexec.hydra. > > Thx....John Cary > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

ImportError: No module named mpdlib
by JR Cary 28 Apr '16

28 Apr '16

One of our users saw: Traceback (most recent call last): File "<somedir>/Contents/engine/bin/mpiexec", line 186, in import mpdlib ImportError: No module named mpdlib Anyone seen this before? mpich-3.0.4, compiled with ../configure' \ --prefix=/somedir/mpich-3.0.4-shared \ --enable-shared \ --disable-static \ CC='/usr/bin/gcc' \ CXX='/usr/bin/g++' \ FC='/usr/bin/gfortran' \ F77='/usr/bin/gfortran' \ CFLAGS='-pipe -fPIC' \ CXXFLAGS='-pipe -fPIC' \ FFLAGS='-fPIC' \ FCFLAGS='-fPIC' mpiexec is a softlink to mpiexec.hydra. Thx....John Cary _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] runtime error for mpich-master-v3.2-247-g1aec69b70951 with Solaris Sparc
by Siegmar Gross 27 Apr '16

27 Apr '16

Hi Min, thank you very much for your help. I'm waiting for your message that the patch has been added into the master branch. Kind regards Siegmar Am 27.04.2016 um 15:52 schrieb Min Si: > Hi Siegmar, > > I think this is the same issue as you reported several month before. This > error is caused by unaligned memory access in MPICH internal code, which is > not allowed on SPARC machines. We have already finished a fix patch, but the > patch is still in review processing, so it is not added into MPICH master > branch yet. I will let you know once we have it in master branch. > > Min > > On 4/21/16 9:21 PM, Siegmar Gross wrote: >> Hi, >> >> I have built mpich-master-v3.2-247-g1aec69b70951 on my machines >> (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 x86_64) >> with gcc-5.1.0 and Sun C 5.13. I get the following errors for both >> compilers if I run small programs that spawn processes on two Sparc >> machines. Everything works fine if I use Linux and/or Solaris x86_64. >> "mpiexec" is aliased to 'mpiexec -genvnone'. I get different errors, >> if I run the same command several times as you can see below (sometimes >> it even works as expected). >> >> >> tyr spawn 119 mpichversion >> MPICH Version: 3.2 >> MPICH Release date: Tue Apr 19 00:00:44 CDT 2016 >> MPICH Device: ch3:nemesis >> MPICH configure: --prefix=/usr/local/mpich-3.2.1_64_gcc >> --libdir=/usr/local/mpich-3.2.1_64_gcc/lib64 >> --includedir=/usr/local/mpich-3.2.1_64_gcc/include64 CC=gcc CXX=g++ >> F77=gfortran FC=gfortran CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 FCFLAGS=-m64 >> LDFLAGS=-m64 -L/usr/lib/sparcv9 -Wl,-rpath -Wl,/usr/lib/sparcv9 >> --enable-fortran=yes --enable-cxx --enable-romio --enable-debuginfo >> --enable-smpcoll --enable-threads=multiple --with-thread-package=posix >> --enable-shared >> MPICH CC: gcc -m64 -O2 >> MPICH CXX: g++ -m64 -O2 >> MPICH F77: gfortran -m64 -O2 >> MPICH FC: gfortran -m64 -O2 >> >> >> tyr spawn 120 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester spawn_master >> >> Parent process 0 running on tyr.informatik.hs-fulda.de >> I create 4 slave processes >> >> Fatal error in MPI_Comm_spawn: Unknown error class, error stack: >> MPI_Comm_spawn(144)...................: MPI_Comm_spawn(cmd="spawn_slave", >> argv=0, maxprocs=4, MPI_INFO_NULL, root=0, MPI_COMM_WORLD, >> intercomm=ffffffff7fffdf58, errors=0) failed >> MPIDI_Comm_spawn_multiple(274)........: >> MPID_Comm_accept(153).................: >> MPIDI_Comm_accept(1039)...............: >> MPIDU_Complete_posted_with_error(1137): Process failed >> >> =================================================================================== >> >> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >> = PID 3182 RUNNING AT tyr >> = EXIT CODE: 10 >> = CLEANING UP REMAINING PROCESSES >> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >> =================================================================================== >> >> >> >> >> tyr spawn 121 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester spawn_master >> >> Parent process 0 running on tyr.informatik.hs-fulda.de >> I create 4 slave processes >> >> Parent process 0: tasks in MPI_COMM_WORLD: 1 >> tasks in COMM_CHILD_PROCESSES local group: 1 >> tasks in COMM_CHILD_PROCESSES remote group: 4 >> >> Slave process 3 of 4 running on ruester.informatik.hs-fulda.de >> Slave process 2 of 4 running on ruester.informatik.hs-fulda.de >> spawn_slave 2: argv[0]: spawn_slave >> spawn_slave 3: argv[0]: spawn_slave >> Slave process 0 of 4 running on tyr.informatik.hs-fulda.de >> spawn_slave 0: argv[0]: spawn_slave >> Slave process 1 of 4 running on tyr.informatik.hs-fulda.de >> spawn_slave 1: argv[0]: spawn_slave >> >> >> >> tyr spawn 122 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester spawn_master >> >> Parent process 0 running on tyr.informatik.hs-fulda.de >> I create 4 slave processes >> >> >> >> tyr spawn 123 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester spawn_master >> >> Parent process 0 running on tyr.informatik.hs-fulda.de >> I create 4 slave processes >> >> Fatal error in MPI_Comm_spawn: Unknown error class, error stack: >> MPI_Comm_spawn(144)...................: MPI_Comm_spawn(cmd="spawn_slave", >> argv=0, maxprocs=4, MPI_INFO_NULL, root=0, MPI_COMM_WORLD, >> intercomm=ffffffff7fffdf58, errors=0) failed >> MPIDI_Comm_spawn_multiple(274)........: >> MPID_Comm_accept(153).................: >> MPIDI_Comm_accept(1039)...............: >> MPIDU_Complete_posted_with_error(1137): Process failed >> tyr spawn 124 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester spawn_master >> >> Parent process 0 running on tyr.informatik.hs-fulda.de >> I create 4 slave processes >> >> Fatal error in MPI_Comm_spawn: Unknown error class, error stack: >> MPI_Comm_spawn(144)...................: MPI_Comm_spawn(cmd="spawn_slave", >> argv=0, maxprocs=4, MPI_INFO_NULL, root=0, MPI_COMM_WORLD, >> intercomm=ffffffff7fffdf58, errors=0) failed >> MPIDI_Comm_spawn_multiple(274)........: >> MPID_Comm_accept(153).................: >> MPIDI_Comm_accept(1039)...............: >> MPIDU_Complete_posted_with_error(1137): Process failed >> >> =================================================================================== >> >> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >> = PID 3466 RUNNING AT tyr >> = EXIT CODE: 10 >> = CLEANING UP REMAINING PROCESSES >> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >> =================================================================================== >> >> [proxy:0:[email protected]] HYD_pmcd_pmip_control_cmd_cb >> (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/pm/pmiserv/pmip_cb.c:886): >> assert (!closed) failed >> [proxy:0:[email protected][proxy:1:[email protected]] >> HYD_pmcd_pmip_control_cmd_cb >> (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/pm/pmiserv/pmip_cb.c] >> HYDT_dmxu_poll_wait_for_event >> (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/tools/demux/demux_poll.c:77): >> callback returned error status >> [proxy:0:[email protected]] main >> (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/pm/pmiserv/pmip.c:202): >> demux engine error waiting for event >> [mpiexec(a)tyr.informatik.hs-fulda.de] HYDT_bscu_wait_for_completion >> (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/tools/bootstrap/utils/bscu_wait.c:76): >> one of the processes terminated badly; aborting >> [mpiexec(a)tyr.informatik.hs-fulda.de] HYDT_bsci_wait_for_completion >> (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/tools/bootstrap/src/bsci_wait.c:23): >> launcher returned error waiting for completion >> [mpiexec(a)tyr.informatik.hs-fulda.de] HYD_pmci_wait_for_completion >> (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/pm/pmiserv/pmiserv_pmci.c:218): >> launcher returned error waiting for completion >> [mpiexec(a)tyr.informatik.hs-fulda.de] main >> (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/ui/mpich/mpiexec.c:340): >> process manager error waiting for completion >> tyr spawn 125 >> >> >> I would be grateful if somebody can fix the problem. Thank you very >> much for any help in advance. >> >> >> Kind regards >> >> Siegmar >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] runtime error for mpich-master-v3.2-247-g1aec69b70951 with Solaris Sparc
by Min Si 27 Apr '16

27 Apr '16

Hi Siegmar, I think this is the same issue as you reported several month before. This error is caused by unaligned memory access in MPICH internal code, which is not allowed on SPARC machines. We have already finished a fix patch, but the patch is still in review processing, so it is not added into MPICH master branch yet. I will let you know once we have it in master branch. Min On 4/21/16 9:21 PM, Siegmar Gross wrote: > Hi, > > I have built mpich-master-v3.2-247-g1aec69b70951 on my machines > (Solaris 10 Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 x86_64) > with gcc-5.1.0 and Sun C 5.13. I get the following errors for both > compilers if I run small programs that spawn processes on two Sparc > machines. Everything works fine if I use Linux and/or Solaris x86_64. > "mpiexec" is aliased to 'mpiexec -genvnone'. I get different errors, > if I run the same command several times as you can see below (sometimes > it even works as expected). > > > tyr spawn 119 mpichversion > MPICH Version: 3.2 > MPICH Release date: Tue Apr 19 00:00:44 CDT 2016 > MPICH Device: ch3:nemesis > MPICH configure: --prefix=/usr/local/mpich-3.2.1_64_gcc > --libdir=/usr/local/mpich-3.2.1_64_gcc/lib64 > --includedir=/usr/local/mpich-3.2.1_64_gcc/include64 CC=gcc CXX=g++ > F77=gfortran FC=gfortran CFLAGS=-m64 CXXFLAGS=-m64 FFLAGS=-m64 > FCFLAGS=-m64 LDFLAGS=-m64 -L/usr/lib/sparcv9 -Wl,-rpath > -Wl,/usr/lib/sparcv9 --enable-fortran=yes --enable-cxx --enable-romio > --enable-debuginfo --enable-smpcoll --enable-threads=multiple > --with-thread-package=posix --enable-shared > MPICH CC: gcc -m64 -O2 > MPICH CXX: g++ -m64 -O2 > MPICH F77: gfortran -m64 -O2 > MPICH FC: gfortran -m64 -O2 > > > tyr spawn 120 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester > spawn_master > > Parent process 0 running on tyr.informatik.hs-fulda.de > I create 4 slave processes > > Fatal error in MPI_Comm_spawn: Unknown error class, error stack: > MPI_Comm_spawn(144)...................: > MPI_Comm_spawn(cmd="spawn_slave", argv=0, maxprocs=4, MPI_INFO_NULL, > root=0, MPI_COMM_WORLD, intercomm=ffffffff7fffdf58, errors=0) failed > MPIDI_Comm_spawn_multiple(274)........: > MPID_Comm_accept(153).................: > MPIDI_Comm_accept(1039)...............: > MPIDU_Complete_posted_with_error(1137): Process failed > > =================================================================================== > > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = PID 3182 RUNNING AT tyr > = EXIT CODE: 10 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > > > > > tyr spawn 121 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester > spawn_master > > Parent process 0 running on tyr.informatik.hs-fulda.de > I create 4 slave processes > > Parent process 0: tasks in MPI_COMM_WORLD: 1 > tasks in COMM_CHILD_PROCESSES local group: 1 > tasks in COMM_CHILD_PROCESSES remote group: 4 > > Slave process 3 of 4 running on ruester.informatik.hs-fulda.de > Slave process 2 of 4 running on ruester.informatik.hs-fulda.de > spawn_slave 2: argv[0]: spawn_slave > spawn_slave 3: argv[0]: spawn_slave > Slave process 0 of 4 running on tyr.informatik.hs-fulda.de > spawn_slave 0: argv[0]: spawn_slave > Slave process 1 of 4 running on tyr.informatik.hs-fulda.de > spawn_slave 1: argv[0]: spawn_slave > > > > tyr spawn 122 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester > spawn_master > > Parent process 0 running on tyr.informatik.hs-fulda.de > I create 4 slave processes > > > > tyr spawn 123 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester > spawn_master > > Parent process 0 running on tyr.informatik.hs-fulda.de > I create 4 slave processes > > Fatal error in MPI_Comm_spawn: Unknown error class, error stack: > MPI_Comm_spawn(144)...................: > MPI_Comm_spawn(cmd="spawn_slave", argv=0, maxprocs=4, MPI_INFO_NULL, > root=0, MPI_COMM_WORLD, intercomm=ffffffff7fffdf58, errors=0) failed > MPIDI_Comm_spawn_multiple(274)........: > MPID_Comm_accept(153).................: > MPIDI_Comm_accept(1039)...............: > MPIDU_Complete_posted_with_error(1137): Process failed > tyr spawn 124 mpiexec -np 1 --host tyr,tyr,tyr,ruester,ruester > spawn_master > > Parent process 0 running on tyr.informatik.hs-fulda.de > I create 4 slave processes > > Fatal error in MPI_Comm_spawn: Unknown error class, error stack: > MPI_Comm_spawn(144)...................: > MPI_Comm_spawn(cmd="spawn_slave", argv=0, maxprocs=4, MPI_INFO_NULL, > root=0, MPI_COMM_WORLD, intercomm=ffffffff7fffdf58, errors=0) failed > MPIDI_Comm_spawn_multiple(274)........: > MPID_Comm_accept(153).................: > MPIDI_Comm_accept(1039)...............: > MPIDU_Complete_posted_with_error(1137): Process failed > > =================================================================================== > > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = PID 3466 RUNNING AT tyr > = EXIT CODE: 10 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > > [proxy:0:[email protected]] HYD_pmcd_pmip_control_cmd_cb > (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/pm/pmiserv/pmip_cb.c:886): > assert (!closed) failed > [proxy:0:[email protected][proxy:1:[email protected]] > HYD_pmcd_pmip_control_cmd_cb > (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/pm/pmiserv/pmip_cb.c] > HYDT_dmxu_poll_wait_for_event > (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/tools/demux/demux_poll.c:77): > callback returned error status > [proxy:0:[email protected]] main > (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/pm/pmiserv/pmip.c:202): > demux engine error waiting for event > [mpiexec(a)tyr.informatik.hs-fulda.de] HYDT_bscu_wait_for_completion > (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/tools/bootstrap/utils/bscu_wait.c:76): > one of the processes terminated badly; aborting > [mpiexec(a)tyr.informatik.hs-fulda.de] HYDT_bsci_wait_for_completion > (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/tools/bootstrap/src/bsci_wait.c:23): > launcher returned error waiting for completion > [mpiexec(a)tyr.informatik.hs-fulda.de] HYD_pmci_wait_for_completion > (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/pm/pmiserv/pmiserv_pmci.c:218): > launcher returned error waiting for completion > [mpiexec(a)tyr.informatik.hs-fulda.de] main > (../../../../mpich-master-v3.2-247-g1aec69b70951/src/pm/hydra/ui/mpich/mpiexec.c:340): > process manager error waiting for completion > tyr spawn 125 > > > I would be grateful if somebody can fix the problem. Thank you very > much for any help in advance. > > > Kind regards > > Siegmar > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Running mpich without mpiexec
by Dmitriy Lyubimov 26 Apr '16

26 Apr '16

Thank you Kenneth. I figured as much. No matter how long i searched, it seems this question keeps popping up and there is no solution to date. On the bright side i figured, oh well, we can launch the separate workers, configure them into one intra communicator via MPI_Comm_accept() and intracomm_merge() and pass in the task via shared memory. It seems to be working although direct in-process control over task data hand-off would be nicer still. Thank you for the issue tracker! -dmitriy On Tue, Apr 26, 2016 at 8:18 AM, Kenneth Raffenetti <raffenet(a)mcs.anl.gov> wrote: > Hi Dmitriy, > > mpiexec is still needed for this type of operation with MPICH today. This > ticket has some of the previous discussion about singleton init and spawn, > but there hasn't been much demand for the feature. > https://trac.mpich.org/projects/mpich/ticket/1074 > > Ken > > > On 04/21/2016 12:18 AM, Dmitriy Lyubimov wrote: > >> Hi, >> >> I guess i need to be asking more specific questions. >> >> I saw the same issue has been raised a few years ago and apparently >> there was no solution for it back then, so i thought i'd check again. >> >> The question is about bypassing mpiexec and trying to connect already >> running processes (which already have some data in memory to share with >> mpi tasks) into a single communicator without having to spawn yet >> another process with mpiexec. >> >> Suppose one of this processes uses MPI_Comm_accept, and the other >> somehow know the mpi port name and would use MPI_Comm_connect() to >> connect to it. >> >> Everything works as long as both server and client started with mpiexec >> -n 1. However, without mpiexec, the same attempts cause errors and >> connection never happens: >> >> mpiexec@Intel-Kubu] match_arg (utils/args/args.c:159): unrecognized >> argument pmi_args >> [mpiexec@Intel-Kubu] HYDU_parse_array (utils/args/args.c:174): argument >> matching returned error >> [mpiexec@Intel-Kubu] parse_args (ui/mpich/utils.c:1596): error parsing >> input array >> [mpiexec@Intel-Kubu] HYD_uii_mpx_get_parameters (ui/mpich/utils.c:1648): >> unable to parse user arguments >> [mpiexec@Intel-Kubu] main (ui/mpich/mpiexec.c:153): error parsing >> parameters >> >> >> So, there's no way of working around the necessity to spawn an entirely >> new process with mpiexec? >> >> Thank you very much for your help. >> -Dmitriy >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Running mpich without mpiexec
by Kenneth Raffenetti 26 Apr '16

26 Apr '16

Hi Dmitriy, mpiexec is still needed for this type of operation with MPICH today. This ticket has some of the previous discussion about singleton init and spawn, but there hasn't been much demand for the feature. https://trac.mpich.org/projects/mpich/ticket/1074 Ken On 04/21/2016 12:18 AM, Dmitriy Lyubimov wrote: > Hi, > > I guess i need to be asking more specific questions. > > I saw the same issue has been raised a few years ago and apparently > there was no solution for it back then, so i thought i'd check again. > > The question is about bypassing mpiexec and trying to connect already > running processes (which already have some data in memory to share with > mpi tasks) into a single communicator without having to spawn yet > another process with mpiexec. > > Suppose one of this processes uses MPI_Comm_accept, and the other > somehow know the mpi port name and would use MPI_Comm_connect() to > connect to it. > > Everything works as long as both server and client started with mpiexec > -n 1. However, without mpiexec, the same attempts cause errors and > connection never happens: > > mpiexec@Intel-Kubu] match_arg (utils/args/args.c:159): unrecognized > argument pmi_args > [mpiexec@Intel-Kubu] HYDU_parse_array (utils/args/args.c:174): argument > matching returned error > [mpiexec@Intel-Kubu] parse_args (ui/mpich/utils.c:1596): error parsing > input array > [mpiexec@Intel-Kubu] HYD_uii_mpx_get_parameters (ui/mpich/utils.c:1648): > unable to parse user arguments > [mpiexec@Intel-Kubu] main (ui/mpich/mpiexec.c:153): error parsing parameters > > > So, there's no way of working around the necessity to spawn an entirely > new process with mpiexec? > > Thank you very much for your help. > -Dmitriy > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Isend and Test
by Zhen Wang 26 Apr '16

26 Apr '16

Yanfei, Thank you. If I change usleep to 2000 or 10000000, 2 MPI_Test's are called. This is quoted from Junchao's reply: "My understanding is that: In eager mode, sender does not need a Ack message from receiver (to know recv buf is ready), but in rendezvous mode, it does. In your case, if it is in rendezvous mode, Isend() issues the request. But since it is a nonblocking call, before sender gets the Ack message, it goes into sleep. The asynchronous progress engine does not progress until MPI_Wait() is called. At that time, the Ack message is got, and message passing happens. Therefore, MPI_Recv() finishes after MPI_Wait()." So my understanding is there's a handshake in rendezvous mode, and the handshake is performed by MPI_Test and MPI_Recv (or in the case above, MPI_Wait gets the Ack message). Please correct me if I'm wrong. Thanks. Best regards, Zhen On Mon, Apr 25, 2016 at 6:45 PM, Guo, Yanfei <yguo(a)anl.gov> wrote: > Hi, > > The MPI_Test call has nothing to do with “let P1 receive the data”. Even > without calling MPI_Test, the communication should eventually complete. > MPI_Test in your code is only for the sender to check whether the > communication is actually done. The 2 MPI_Test calls in your test is > because the completion of the communication takes longer time than the > usleep. If you reduce the time in usleep, you might get even more MPI_Test > calls. I don’t think it is relevant to performance. > > Yanfei Guo > Postdoctoral Researcher > MCS Division, ANL > > > > > > > > On 4/25/16, 3:25 PM, "Zhen Wang" <toddwz(a)gmail.com> wrote: > > >Rob, > > > > > >Thanks for your reply. Let me rephrase my question. I'm simulating what I > do in a complicate code. P0 Isends data to P1, and does computations. P1 > needs data to do computation, so it calls Recv. As you said, P0 is calling > MPI_Test periodically. What puzzles > > me is why 2 calls of MPI_Test are needed to let P1 receive the data. > (With Intel MPI, the number jumps to around 6.) The number of MPI_Test > calls has influence on performance. Thanks. > > > > > >Best regards, > >Zhen > > > > > > > > > >On Mon, Apr 25, 2016 at 4:13 PM, Rob Latham > ><robl(a)mcs.anl.gov> wrote: > > > > > > > >On 04/25/2016 02:57 PM, Zhen Wang wrote: > > > >Hi, > > > >I have questions regarding MPI_Isend and MPI_Test. A sample code is > >attached, output is as follows. > > > >For instance, both Isend and Recv of 0 start at 15:46:14, but 2 MPI_Test > >are called before Recv of 0 finishes. My understanding is first Test > >receives the signal that receiver is ready, and data transfer follows. > >Then the second Test will see the transfer is complete, and free > >MPI_Request. Is my understanding correct? Thanks. > > > > > > > >MPI_Test() is non-blocking. MPI_Test() is not going to wait for anything > to happen -- that's what MPI_Wait() will do. > > > >under the hood, MPI_Test "kicks the progress engine", which means > everything that can execute right now will happen. If there is anything > that requires code to wait (to receive a signal, in your case), then test > will return. > > > >A data transfer might take several calls to MPI_Test to complete. Or, > there's a background progress thread that's churning along and you only > need to make one call to MPI_Test. It's implementation-dependent. > > > >You are asking the wrong question, though. At this level, it doesn't > matter what MPI_Test does. If your code has something it can do while the > isend progresses, then go do that and call MPI_Test periodically. If you > have nothing productive you can do, call > > MPI_Wait() (or one of the variants). > > > >==rob > > > > > > > >Best regards, > >Zhen > > > >MPI 1: Recv of 0 started at 15:46:14. > >MPI 0: Isend of 0 started at 15:46:14. > >MPI 0: Isend of 1 started at 15:46:14. > >MPI 0: Isend of 2 started at 15:46:14. > >MPI 0: Isend of 3 started at 15:46:14. > >MPI 0: Isend of 4 started at 15:46:14. > >MPI 0: MPI_Test of 0 at 15:46:16. > >MPI 0: MPI_Test of 0 at 15:46:18. > >MPI 0: Isend of 0 finished at 15:46:18. > >MPI 1: Recv of 0 finished at 15:46:18. > >MPI 1: Recv of 1 started at 15:46:18. > >MPI 0: MPI_Test of 1 at 15:46:20. > >MPI 0: MPI_Test of 1 at 15:46:22. > >MPI 0: Isend of 1 finished at 15:46:22. > >MPI 1: Recv of 1 finished at 15:46:22. > >MPI 1: Recv of 2 started at 15:46:22. > >MPI 0: MPI_Test of 2 at 15:46:24. > >MPI 0: MPI_Test of 2 at 15:46:26. > >MPI 0: Isend of 2 finished at 15:46:26. > >MPI 1: Recv of 2 finished at 15:46:26. > >MPI 1: Recv of 3 started at 15:46:26. > >MPI 0: MPI_Test of 3 at 15:46:28. > >MPI 0: MPI_Test of 3 at 15:46:30. > >MPI 0: Isend of 3 finished at 15:46:30. > >MPI 1: Recv of 3 finished at 15:46:30. > >MPI 1: Recv of 4 started at 15:46:30. > >MPI 0: MPI_Test of 4 at 15:46:32. > >MPI 0: MPI_Test of 4 at 15:46:34. > >MPI 0: Isend of 4 finished at 15:46:34. > >MPI 1: Recv of 4 finished at 15:46:34. > > > > > > > > > > > >_______________________________________________ > >discuss mailing list discuss(a)mpich.org > >To manage subscription options or unsubscribe: > >https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > >_______________________________________________ > >discuss mailing list discuss(a)mpich.org > >To manage subscription options or unsubscribe: > >https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH2-1.4.1 on Windows Server, Issues Running on More than One Node
by Joshua Moore 25 Apr '16

25 Apr '16

Hello, This is a follow-up to my message below. I am running MPICH2-1.4.1 on Windows Server 2102r2 using the job scheduler with command line through PowerShell. I found this page which explains my issue https://technet.microsoft.com/pt-pt/library/cc720141(v=ws.10).aspx "An exception is the mpiexec option *-n*. Using this option will cause all of the processes to be run on the same processor, regardless of how many processors have been specified using */numprocessors:*." It's an older version, but I think is the cause of my problem. Can I use MPICH2 instead of Windows MPI with the Windows Job Scheduler? If I don't specify any "-n" parameter, I only get mpiexec to execute a single task, despite the number of processors I am trying to request I am trying to use $j=New-HpcJob $j | Add-HpcTask -numcores $NUMCORES -command "$MPIEXEC -n $NUMCORES lmp_mpi.exe -in in.lammps" $j | Submit-HpcJob where $MPIEXEC is the full path to mpiexec.exe. I am using MPICH2-1.4.1 because it works with LAMMPS and I am told Windows MPI does not because of Visual C++ compatibility with LAMMPS. I can successfully run the cpi.exe example outside the job scheduler on all cores and nodes. I can use a machinefile as well but when I use it, the cores and nodes that mpiexec executes don't follow the allocated nodes from the job scheduler. Is there anyway around using MPICH2 with the -n option in Windows Job Scheduler getting jobs to run on more than one node? Thanks again for any response I get. I've spent days trying to figure this out so would be grateful for any advice. Josh On Sun, Apr 24, 2016 at 11:49 PM, Joshua Moore <jdmoore(a)ncsu.edu> wrote: > Hello, > > I am having issues running mpiexec when I run on more than one node. > > I am using v 1.4.1 because it is compatible with LAMMPS software I am > using. > > I am running on Windows Server 2012r2 and trying to get MPICH2 to play > nice with Window's batch server. > > I've installed mpich2 on each of the nodes and the head node. > > 1) *msiexec /i mpich2-1.2.1-win-ia32.msi* > *2) mpiexec -register (to register username and password)* > *3) smpd -install (to start spmd server on **each of the nodes and the > head node)* > *4) On the head node, I've used smpd -sethosts hostname1 hostname2 ... to > add each of the hosts. I haven't done this on the compute nodes.* > > When I execute a job through Windows batch system through PowerShell, my > job executes but puts all of the executions on the same node. So if for > example my nodes are 16 cores, and I ask for 32 cores, 32 separate > processes are run on the first node and none on the second, even though > Window's scheduler is allocated 2 nodes. > > It's like mpiexec is ignoring the node list that the smpd hosts is setting. > > I can use a machinefile with mpiexec and this will allow me to execute on > multiple nodes but it doesn't seem to follow the nodes that Windows > allocates in their batch server. > > I should also add that when I try to request more than one node through > Window's batch server with -requestednodes "host1 host2" with new-hpcjob, > Windows tells me that I can't do this because I have zero cores available. > I can ask for only 1 node and up to the maximum number of cores and it is > ok with this. > > Any suggestions? > > Thank you. > > Josh > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Isend and Test
by Guo, Yanfei 25 Apr '16

25 Apr '16

Hi, The MPI_Test call has nothing to do with “let P1 receive the data”. Even without calling MPI_Test, the communication should eventually complete. MPI_Test in your code is only for the sender to check whether the communication is actually done. The 2 MPI_Test calls in your test is because the completion of the communication takes longer time than the usleep. If you reduce the time in usleep, you might get even more MPI_Test calls. I don’t think it is relevant to performance. Yanfei Guo Postdoctoral Researcher MCS Division, ANL On 4/25/16, 3:25 PM, "Zhen Wang" <toddwz(a)gmail.com> wrote: >Rob, > > >Thanks for your reply. Let me rephrase my question. I'm simulating what I do in a complicate code. P0 Isends data to P1, and does computations. P1 needs data to do computation, so it calls Recv. As you said, P0 is calling MPI_Test periodically. What puzzles > me is why 2 calls of MPI_Test are needed to let P1 receive the data. (With Intel MPI, the number jumps to around 6.) The number of MPI_Test calls has influence on performance. Thanks. > > >Best regards, >Zhen > > > > >On Mon, Apr 25, 2016 at 4:13 PM, Rob Latham ><robl(a)mcs.anl.gov> wrote: > > > >On 04/25/2016 02:57 PM, Zhen Wang wrote: > >Hi, > >I have questions regarding MPI_Isend and MPI_Test. A sample code is >attached, output is as follows. > >For instance, both Isend and Recv of 0 start at 15:46:14, but 2 MPI_Test >are called before Recv of 0 finishes. My understanding is first Test >receives the signal that receiver is ready, and data transfer follows. >Then the second Test will see the transfer is complete, and free >MPI_Request. Is my understanding correct? Thanks. > > > >MPI_Test() is non-blocking. MPI_Test() is not going to wait for anything to happen -- that's what MPI_Wait() will do. > >under the hood, MPI_Test "kicks the progress engine", which means everything that can execute right now will happen. If there is anything that requires code to wait (to receive a signal, in your case), then test will return. > >A data transfer might take several calls to MPI_Test to complete. Or, there's a background progress thread that's churning along and you only need to make one call to MPI_Test. It's implementation-dependent. > >You are asking the wrong question, though. At this level, it doesn't matter what MPI_Test does. If your code has something it can do while the isend progresses, then go do that and call MPI_Test periodically. If you have nothing productive you can do, call > MPI_Wait() (or one of the variants). > >==rob > > > >Best regards, >Zhen > >MPI 1: Recv of 0 started at 15:46:14. >MPI 0: Isend of 0 started at 15:46:14. >MPI 0: Isend of 1 started at 15:46:14. >MPI 0: Isend of 2 started at 15:46:14. >MPI 0: Isend of 3 started at 15:46:14. >MPI 0: Isend of 4 started at 15:46:14. >MPI 0: MPI_Test of 0 at 15:46:16. >MPI 0: MPI_Test of 0 at 15:46:18. >MPI 0: Isend of 0 finished at 15:46:18. >MPI 1: Recv of 0 finished at 15:46:18. >MPI 1: Recv of 1 started at 15:46:18. >MPI 0: MPI_Test of 1 at 15:46:20. >MPI 0: MPI_Test of 1 at 15:46:22. >MPI 0: Isend of 1 finished at 15:46:22. >MPI 1: Recv of 1 finished at 15:46:22. >MPI 1: Recv of 2 started at 15:46:22. >MPI 0: MPI_Test of 2 at 15:46:24. >MPI 0: MPI_Test of 2 at 15:46:26. >MPI 0: Isend of 2 finished at 15:46:26. >MPI 1: Recv of 2 finished at 15:46:26. >MPI 1: Recv of 3 started at 15:46:26. >MPI 0: MPI_Test of 3 at 15:46:28. >MPI 0: MPI_Test of 3 at 15:46:30. >MPI 0: Isend of 3 finished at 15:46:30. >MPI 1: Recv of 3 finished at 15:46:30. >MPI 1: Recv of 4 started at 15:46:30. >MPI 0: MPI_Test of 4 at 15:46:32. >MPI 0: MPI_Test of 4 at 15:46:34. >MPI 0: Isend of 4 finished at 15:46:34. >MPI 1: Recv of 4 finished at 15:46:34. > > > > > >_______________________________________________ >discuss mailing list discuss(a)mpich.org >To manage subscription options or unsubscribe: >https://lists.mpich.org/mailman/listinfo/discuss > > > >_______________________________________________ >discuss mailing list discuss(a)mpich.org >To manage subscription options or unsubscribe: >https://lists.mpich.org/mailman/listinfo/discuss > > > > > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Isend and Test
by Zhen Wang 25 Apr '16

25 Apr '16

Rob, Thanks for your reply. Let me rephrase my question. I'm simulating what I do in a complicate code. P0 Isends data to P1, and does computations. P1 needs data to do computation, so it calls Recv. As you said, P0 is calling MPI_Test periodically. What puzzles me is why 2 calls of MPI_Test are needed to let P1 receive the data. (With Intel MPI, the number jumps to around 6.) The number of MPI_Test calls has influence on performance. Thanks. Best regards, Zhen On Mon, Apr 25, 2016 at 4:13 PM, Rob Latham <robl(a)mcs.anl.gov> wrote: > > > On 04/25/2016 02:57 PM, Zhen Wang wrote: > >> Hi, >> >> I have questions regarding MPI_Isend and MPI_Test. A sample code is >> attached, output is as follows. >> >> For instance, both Isend and Recv of 0 start at 15:46:14, but 2 MPI_Test >> are called before Recv of 0 finishes. My understanding is first Test >> receives the signal that receiver is ready, and data transfer follows. >> Then the second Test will see the transfer is complete, and free >> MPI_Request. Is my understanding correct? Thanks. >> > > MPI_Test() is non-blocking. MPI_Test() is not going to wait for anything > to happen -- that's what MPI_Wait() will do. > > under the hood, MPI_Test "kicks the progress engine", which means > everything that can execute right now will happen. If there is anything > that requires code to wait (to receive a signal, in your case), then test > will return. > > A data transfer might take several calls to MPI_Test to complete. Or, > there's a background progress thread that's churning along and you only > need to make one call to MPI_Test. It's implementation-dependent. > > You are asking the wrong question, though. At this level, it doesn't > matter what MPI_Test does. If your code has something it can do while the > isend progresses, then go do that and call MPI_Test periodically. If you > have nothing productive you can do, call MPI_Wait() (or one of the > variants). > > ==rob > > >> >> Best regards, >> Zhen >> >> MPI 1: Recv of 0 started at 15:46:14. >> MPI 0: Isend of 0 started at 15:46:14. >> MPI 0: Isend of 1 started at 15:46:14. >> MPI 0: Isend of 2 started at 15:46:14. >> MPI 0: Isend of 3 started at 15:46:14. >> MPI 0: Isend of 4 started at 15:46:14. >> MPI 0: MPI_Test of 0 at 15:46:16. >> MPI 0: MPI_Test of 0 at 15:46:18. >> MPI 0: Isend of 0 finished at 15:46:18. >> MPI 1: Recv of 0 finished at 15:46:18. >> MPI 1: Recv of 1 started at 15:46:18. >> MPI 0: MPI_Test of 1 at 15:46:20. >> MPI 0: MPI_Test of 1 at 15:46:22. >> MPI 0: Isend of 1 finished at 15:46:22. >> MPI 1: Recv of 1 finished at 15:46:22. >> MPI 1: Recv of 2 started at 15:46:22. >> MPI 0: MPI_Test of 2 at 15:46:24. >> MPI 0: MPI_Test of 2 at 15:46:26. >> MPI 0: Isend of 2 finished at 15:46:26. >> MPI 1: Recv of 2 finished at 15:46:26. >> MPI 1: Recv of 3 started at 15:46:26. >> MPI 0: MPI_Test of 3 at 15:46:28. >> MPI 0: MPI_Test of 3 at 15:46:30. >> MPI 0: Isend of 3 finished at 15:46:30. >> MPI 1: Recv of 3 finished at 15:46:30. >> MPI 1: Recv of 4 started at 15:46:30. >> MPI 0: MPI_Test of 4 at 15:46:32. >> MPI 0: MPI_Test of 4 at 15:46:34. >> MPI 0: Isend of 4 finished at 15:46:34. >> MPI 1: Recv of 4 finished at 15:46:34. >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0