March 2016 - discuss - mailman3.cels.anl.gov

ssh-less MPI using docker exec?
by Christian Kniep 31 Mar '16

31 Mar '16

Hey there, I tinkered around with a ssh-less openmpi and I was wondering if it could be implemented in a nicer way. http://qnib.org/2016/03/31/dssh/ I just used -mca plm_rsh_agent /opt/qnib/src/dssh with a rather simple script to use docker exec: #!/bin/bash REMOTE_HOST=$1 shift set -x docker -H unix:///var/run/docker.sock exec -i -u ${USER} ${REMOTE_HOST} $@ Cheers Christian _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] uninitialized MPI_Status object
by Kenneth Raffenetti 30 Mar '16

30 Mar '16

Try "mpiexec -n 1 ./get_count". On 03/30/2016 08:46 AM, יוסף אלון wrote: > i don't know why, but the first command works and the second command isnt. > i have the following error: > תמונה מוטבעת 1 > > what this means? > > 2016-03-28 21:15 GMT+03:00 Jeff Hammond <jeff.science(a)gmail.com > <mailto:[email protected]>>: > > I created a ticket for this: > https://trac.mpich.org/projects/mpich/ticket/2332. > > Jeff > > On Mon, Mar 28, 2016 at 10:26 AM, Wei-keng Liao > <wkliao(a)eecs.northwestern.edu <mailto:[email protected]>> > wrote: > > My test program can be compiled with command "mpicc get_count.c > -o get_count" > and run "mpiexec -n 4 get_count". For correct (expected) result, > nothing > will be printed on stdout. Otherwise, error messages will be > printed. > > My point is MPI_Get_count does not report correct result because > MPI collective I/O call fails to initialize the MPI_Status object. > > I found that OpenMPI and earlier version of MPICH (2-1.2.1) can run > this test code correctly. > > The test failed when I ran MPICH 3.1.4 and the current from GIT > repo. > > Wei-keng > > On Mar 26, 2016, at 3:21 PM, יוסף אלון wrote: > > > בס"ד > > > > becuse all the command confusion. > > which command should i put to run the above code? > > > > 2016-03-26 3:31 GMT+03:00 Wei-keng Liao > <wkliao(a)eecs.northwestern.edu > <mailto:[email protected]>>: > > > > I notice an uninitialized MPI_Status object can make > MPI_Get_count return a wrong result, > > when the data amount in the MPI operation is zero (although I > only checked MPI-IO). > > Attached is a test program that uses MPI collective read > where only root process has > > non-zero length data to read. The expected result from > MPI_Get_count is 0 for all > > non-root process. To mimic an uninitialized MPI_Status > object, I call memset to make > > the object non-zero. > > > > Here is the code fragment. > > > > if (rank == 0) len = 10; > > else len = 0; > > > > MPI_File_read_all(fh, buf, len, MPI_BYTE, &status); > > > > MPI_Get_count(&status, MPI_BYTE, &get_size); > > > > For process rank > 0, get_size may not be 0. > > > > > > Wei-keng > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > -- > > בברכה, יוסף אלון > > 050-4916740 > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com <mailto:[email protected]> > http://jeffhammond.github.io/ > > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > > -- > בברכה, יוסף אלון > 050-4916740 > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] uninitialized MPI_Status object
by יוסף אלון 30 Mar '16

30 Mar '16

i don't know why, but the first command works and the second command isnt. i have the following error: [image: תמונה מוטבעת 1] what this means? 2016-03-28 21:15 GMT+03:00 Jeff Hammond <jeff.science(a)gmail.com>: > I created a ticket for this: > https://trac.mpich.org/projects/mpich/ticket/2332. > > Jeff > > On Mon, Mar 28, 2016 at 10:26 AM, Wei-keng Liao < > wkliao(a)eecs.northwestern.edu> wrote: > >> My test program can be compiled with command "mpicc get_count.c -o >> get_count" >> and run "mpiexec -n 4 get_count". For correct (expected) result, nothing >> will be printed on stdout. Otherwise, error messages will be printed. >> >> My point is MPI_Get_count does not report correct result because >> MPI collective I/O call fails to initialize the MPI_Status object. >> >> I found that OpenMPI and earlier version of MPICH (2-1.2.1) can run >> this test code correctly. >> >> The test failed when I ran MPICH 3.1.4 and the current from GIT repo. >> >> Wei-keng >> >> On Mar 26, 2016, at 3:21 PM, יוסף אלון wrote: >> >> > בס"ד >> > >> > becuse all the command confusion. >> > which command should i put to run the above code? >> > >> > 2016-03-26 3:31 GMT+03:00 Wei-keng Liao <wkliao(a)eecs.northwestern.edu>: >> > >> > I notice an uninitialized MPI_Status object can make MPI_Get_count >> return a wrong result, >> > when the data amount in the MPI operation is zero (although I only >> checked MPI-IO). >> > Attached is a test program that uses MPI collective read where only >> root process has >> > non-zero length data to read. The expected result from MPI_Get_count is >> 0 for all >> > non-root process. To mimic an uninitialized MPI_Status object, I call >> memset to make >> > the object non-zero. >> > >> > Here is the code fragment. >> > >> > if (rank == 0) len = 10; >> > else len = 0; >> > >> > MPI_File_read_all(fh, buf, len, MPI_BYTE, &status); >> > >> > MPI_Get_count(&status, MPI_BYTE, &get_size); >> > >> > For process rank > 0, get_size may not be 0. >> > >> > >> > Wei-keng >> > >> > >> > _______________________________________________ >> > discuss mailing list discuss(a)mpich.org >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/discuss >> > >> > >> > >> > -- >> > בברכה, יוסף אלון >> > 050-4916740 >> > _______________________________________________ >> > discuss mailing list discuss(a)mpich.org >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- בברכה, יוסף אלון 050-4916740 _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] uninitialized MPI_Status object
by Jeff Hammond 28 Mar '16

28 Mar '16

I created a ticket for this: https://trac.mpich.org/projects/mpich/ticket/2332. Jeff On Mon, Mar 28, 2016 at 10:26 AM, Wei-keng Liao < wkliao(a)eecs.northwestern.edu> wrote: > My test program can be compiled with command "mpicc get_count.c -o > get_count" > and run "mpiexec -n 4 get_count". For correct (expected) result, nothing > will be printed on stdout. Otherwise, error messages will be printed. > > My point is MPI_Get_count does not report correct result because > MPI collective I/O call fails to initialize the MPI_Status object. > > I found that OpenMPI and earlier version of MPICH (2-1.2.1) can run > this test code correctly. > > The test failed when I ran MPICH 3.1.4 and the current from GIT repo. > > Wei-keng > > On Mar 26, 2016, at 3:21 PM, יוסף אלון wrote: > > > בס"ד > > > > becuse all the command confusion. > > which command should i put to run the above code? > > > > 2016-03-26 3:31 GMT+03:00 Wei-keng Liao <wkliao(a)eecs.northwestern.edu>: > > > > I notice an uninitialized MPI_Status object can make MPI_Get_count > return a wrong result, > > when the data amount in the MPI operation is zero (although I only > checked MPI-IO). > > Attached is a test program that uses MPI collective read where only root > process has > > non-zero length data to read. The expected result from MPI_Get_count is > 0 for all > > non-root process. To mimic an uninitialized MPI_Status object, I call > memset to make > > the object non-zero. > > > > Here is the code fragment. > > > > if (rank == 0) len = 10; > > else len = 0; > > > > MPI_File_read_all(fh, buf, len, MPI_BYTE, &status); > > > > MPI_Get_count(&status, MPI_BYTE, &get_size); > > > > For process rank > 0, get_size may not be 0. > > > > > > Wei-keng > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > -- > > בברכה, יוסף אלון > > 050-4916740 > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] uninitialized MPI_Status object
by Wei-keng Liao 28 Mar '16

28 Mar '16

My test program can be compiled with command "mpicc get_count.c -o get_count" and run "mpiexec -n 4 get_count". For correct (expected) result, nothing will be printed on stdout. Otherwise, error messages will be printed. My point is MPI_Get_count does not report correct result because MPI collective I/O call fails to initialize the MPI_Status object. I found that OpenMPI and earlier version of MPICH (2-1.2.1) can run this test code correctly. The test failed when I ran MPICH 3.1.4 and the current from GIT repo. Wei-keng On Mar 26, 2016, at 3:21 PM, יוסף אלון wrote: > בס"ד > > becuse all the command confusion. > which command should i put to run the above code? > > 2016-03-26 3:31 GMT+03:00 Wei-keng Liao <wkliao(a)eecs.northwestern.edu>: > > I notice an uninitialized MPI_Status object can make MPI_Get_count return a wrong result, > when the data amount in the MPI operation is zero (although I only checked MPI-IO). > Attached is a test program that uses MPI collective read where only root process has > non-zero length data to read. The expected result from MPI_Get_count is 0 for all > non-root process. To mimic an uninitialized MPI_Status object, I call memset to make > the object non-zero. > > Here is the code fragment. > > if (rank == 0) len = 10; > else len = 0; > > MPI_File_read_all(fh, buf, len, MPI_BYTE, &status); > > MPI_Get_count(&status, MPI_BYTE, &get_size); > > For process rank > 0, get_size may not be 0. > > > Wei-keng > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > בברכה, יוסף אלון > 050-4916740 > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] 3.2 build help
by Balaji, Pavan 28 Mar '16

28 Mar '16

Michael, MPICH IB support is through MXM, so it won't work directly with native verbs. So your configure option below is essentially falling back to "tcp". With respect to the error, what's "mpi_script_launcher.run"? Can you try using "mpiexec" and see if the error still shows up? -- Pavan > On Mar 28, 2016, at 11:33 AM, Galloway, Michael D. <gallowaymd(a)ornl.gov> wrote: > > Good Day All, > > We’re trying to get a build of 3.2 for our centos7 hpc environment using IB. we don’t have mxm installed so I’m trying this: > > ./configure --prefix=/software/tools/apps/mpich/gnu/3.2 --with-pm=hydra -with-device=ch3:nemesis --with-ibverbs=/usr --with-pbs=/opt/torque > > But we end up with backtraces of the form: > > mpi_script_launcher.run:17440 terminated with signal 7 at PC=7f8caca067f4 SP=7fff7c4ffb20. Backtrace: > mpi_script_launcher.run:17441 terminated with signal 7 at PC=7fece77297f4 SP=7ffd3b5ff100. Backtrace: > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_nem_init+0x964)[0x7f8caca067f4] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_nem_init+0x964)[0x7fece77297f4] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIDI_CH3_Init+0x29)[0x7f8cac9f7609] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIDI_CH3_Init+0x29)[0x7fece771a609] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_Init+0x18b)[0x7f8cac9ececb] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_Init+0x18b)[0x7fece770fecb] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIR_Init_thread+0x34c)[0x7f8cac95345c] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIR_Init_thread+0x34c)[0x7fece767645c] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPI_Init+0x7e)[0x7f8cac952ede] > /home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4008d0] > /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPI_Init+0x7e)[0x7fece7675ede] > /home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4008d0] > /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f8cac4fcb15] > /home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4007c9] > /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fece721fb15] > /home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4007c9] > > > Similarly with mvapich we get failures of this form: > > mpirun -n 4 /home/m8a/mpi_script_launcher/MVAPICH/mpi_script_launcher.run /home/m8a/mpi_script_launcher/mpi_bash_script_example.sh > [cli_0]: aborting job: > Fatal error in MPI_Init: > Other MPI error, error stack: > MPIR_Init_thread(514)..........: > MPID_Init(365).................: channel initialization failed > MPIDI_CH3_Init(495)............: > MPIDI_CH3I_SHMEM_Helper_fn(921): write: Success > =================================================================================== > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = PID 17901 RUNNING AT mod-condo-login02.ornl.gov > = EXIT CODE: 1 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > > > I suspect we are doing something silly here but I’m not sure what, and openmpi code on the same cluster runs fine. > > Is there a current recommendation for IB/pbs/torque build flags? > > — michael > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

3.2 build help
by Galloway, Michael D. 28 Mar '16

28 Mar '16

Good Day All, We’re trying to get a build of 3.2 for our centos7 hpc environment using IB. we don’t have mxm installed so I’m trying this: ./configure --prefix=/software/tools/apps/mpich/gnu/3.2 --with-pm=hydra -with-device=ch3:nemesis --with-ibverbs=/usr --with-pbs=/opt/torque But we end up with backtraces of the form: mpi_script_launcher.run:17440 terminated with signal 7 at PC=7f8caca067f4 SP=7fff7c4ffb20. Backtrace: mpi_script_launcher.run:17441 terminated with signal 7 at PC=7fece77297f4 SP=7ffd3b5ff100. Backtrace: /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_nem_init+0x964)[0x7f8caca067f4] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_nem_init+0x964)[0x7fece77297f4] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIDI_CH3_Init+0x29)[0x7f8cac9f7609] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIDI_CH3_Init+0x29)[0x7fece771a609] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_Init+0x18b)[0x7f8cac9ececb] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPID_Init+0x18b)[0x7fece770fecb] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIR_Init_thread+0x34c)[0x7f8cac95345c] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPIR_Init_thread+0x34c)[0x7fece767645c] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPI_Init+0x7e)[0x7f8cac952ede] /home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4008d0] /software/tools/apps/mpich/gnu/3.2/lib/libmpi.so.12(MPI_Init+0x7e)[0x7fece7675ede] /home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4008d0] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f8cac4fcb15] /home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4007c9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fece721fb15] /home/m8a/mpi_script_launcher/mpi_script_launcher.run[0x4007c9] Similarly with mvapich we get failures of this form: mpirun -n 4 /home/m8a/mpi_script_launcher/MVAPICH/mpi_script_launcher.run /home/m8a/mpi_script_launcher/mpi_bash_script_example.sh [cli_0]: aborting job: Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(514)..........: MPID_Init(365).................: channel initialization failed MPIDI_CH3_Init(495)............: MPIDI_CH3I_SHMEM_Helper_fn(921): write: Success =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 17901 RUNNING AT mod-condo-login02.ornl.gov = EXIT CODE: 1 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== I suspect we are doing something silly here but I’m not sure what, and openmpi code on the same cluster runs fine. Is there a current recommendation for IB/pbs/torque build flags? — michael _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] uninitialized MPI_Status object
by יוסף אלון 26 Mar '16

26 Mar '16

בס"ד becuse all the command confusion. which command should i put to run the above code? 2016-03-26 3:31 GMT+03:00 Wei-keng Liao <wkliao(a)eecs.northwestern.edu>: > > I notice an uninitialized MPI_Status object can make MPI_Get_count return > a wrong result, > when the data amount in the MPI operation is zero (although I only checked > MPI-IO). > Attached is a test program that uses MPI collective read where only root > process has > non-zero length data to read. The expected result from MPI_Get_count is 0 > for all > non-root process. To mimic an uninitialized MPI_Status object, I call > memset to make > the object non-zero. > > Here is the code fragment. > > if (rank == 0) len = 10; > else len = 0; > > MPI_File_read_all(fh, buf, len, MPI_BYTE, &status); > > MPI_Get_count(&status, MPI_BYTE, &get_size); > > For process rank > 0, get_size may not be 0. > > > Wei-keng > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- בברכה, יוסף אלון 050-4916740 _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

uninitialized MPI_Status object
by Wei-keng Liao 25 Mar '16

25 Mar '16

1 0

compile code with mpiexec in mingw on windows
by Yi Pan 24 Mar '16

24 Mar '16

Dear all, I have been trying to compile some code (adcirc+swan) in mingw on windows, one win 10 and one win 7. I failed on both of them. For win 10, I even could not run the example case cpi by mpiexec -n 4 cpi. it says "error while connecting to host, no connection could be made because the target machine actively refused it. (10061)". I tried many solutions, including turning off the firewall, turning off the UAC, add a environment variable MPIR_CVAR_CH3_PORT_RANGE to specify the port to the same port that the mpi listening to... Non of them work. For win 8, don't know what reason I could successfully run the example case cpi by mpiexec -n 4 cpi. However, when I try to compile the code there's another error: launch failed: CreatProcess(work -I ../prep -O2 "-mcmodel=medium" -ffixed-line-length-none "-march+cori7-avx" -m64 -static -DREAL8 ............some flages here............ padcirc odir4/mkdir.o) on 'GodMapleTree' failed, error 5 - assess denied. I tried something, e.g., turn off the UAC, give the administrator assess, ... I just won't work. I know very little on the MPI. Could someone give me some tapes? -- Best regards, Yi Pan, PhD, Assistant Professor, College of Harbor, Coastal and Offshore Engineering, Hohai University, 1 Xikang Rd., Nanjing, Jiangsu 210098, China _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0