April 2016 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPI comm attributes not working?
by Balaji, Pavan 05 Apr '16

05 Apr '16

You are printing a number that's generated by the MPI implementation. It's value will be different for different MPI implementations. -- Pavan > On Apr 4, 2016, at 11:56 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > I am trying to understand how communicator attributes work, so I found http://stackoverflow.com/questions/25198353/callbacks-provided-in-mpi-comm-…, which contains a nice example (the code in http://stackoverflow.com/a/25205611/2189128) > > However, the posted output of this code is: > > $ mpirun -np 1 ./comm-attr > In ibcast_copy: keyval = 10 > In ibcast_delete: keyval = 10 > In ibcast_delete: keyval = 10 > > Whereas I observe: > > $ mpirun -np 1 ./a.out > In ibcast_copy: keyval = -1539309568 > In ibcast_delete: keyval = -1539309568 > > Can anyone help me understand what the correct output of this program is? If it is not what I am seeing, is this a bug in MPICH or am I doing something wrong? > > I am using MPICH git from November 2015, so I will git pull && ./autogen.sh && $CONFIGUREMAKEINSTALL and try again just to be sure. > > Thanks, > > Jeff > > $ mpirun --version > HYDRA build details: > Version: 3.2 > Release Date: unreleased development copy > CC: icc > CXX: icpc > F77: ifort > F90: ifort > Configure options: '--disable-option-checking' '--prefix=/opt/mpich/dev/intel/default' 'CC=icc' 'CXX=icpc' 'FC=ifort' 'F77=ifort' '--enable-cxx' '--enable-fortran' '--enable-threads=runtime' '--enable-g=dbg' '--with-pm=hydra' '--enable-wrapper-rpath' '--enable-static' '--enable-shared' '--cache-file=/dev/null' '--srcdir=../../../../src/pm/hydra' 'CFLAGS= -g -O2' 'LDFLAGS=' 'LIBS=-lpthread ' 'CPPFLAGS= -I/Users/jrhammon/Work/MPI/MPICH/git/build/src/mpl/include -I/Users/jrhammon/Work/MPI/MPICH/git/src/mpl/include -I/Users/jrhammon/Work/MPI/MPICH/git/src/openpa/src -I/Users/jrhammon/Work/MPI/MPICH/git/build/src/openpa/src -D_REENTRANT -I/Users/jrhammon/Work/MPI/MPICH/git/build/src/mpi/romio/include' > Process Manager: pmi > Launchers available: ssh rsh fork slurm ll lsf sge manual persist > Topology libraries available: hwloc > Resource management kernels available: user slurm ll lsf sge pbs cobalt > Checkpointing libraries available: > Demux engines available: poll select > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPI comm attributes not working?
by Jeff Hammond 04 Apr '16

04 Apr '16

I am trying to understand how communicator attributes work, so I found http://stackoverflow.com/questions/25198353/callbacks-provided-in-mpi-comm-…, which contains a nice example (the code in http://stackoverflow.com/a/25205611/2189128) However, the posted output of this code is: $ mpirun -np 1 ./comm-attr In ibcast_copy: keyval = 10 In ibcast_delete: keyval = 10 In ibcast_delete: keyval = 10 Whereas I observe: $ mpirun -np 1 ./a.out In ibcast_copy: keyval = -1539309568 In ibcast_delete: keyval = -1539309568 Can anyone help me understand what the correct output of this program is? If it is not what I am seeing, is this a bug in MPICH or am I doing something wrong? I am using MPICH git from November 2015, so I will git pull && ./autogen.sh && $CONFIGUREMAKEINSTALL and try again just to be sure. Thanks, Jeff $ mpirun --version HYDRA build details: Version: 3.2 Release Date: unreleased development copy CC: icc CXX: icpc F77: ifort F90: ifort Configure options: '--disable-option-checking' '--prefix=/opt/mpich/dev/intel/default' 'CC=icc' 'CXX=icpc' 'FC=ifort' 'F77=ifort' '--enable-cxx' '--enable-fortran' '--enable-threads=runtime' '--enable-g=dbg' '--with-pm=hydra' '--enable-wrapper-rpath' '--enable-static' '--enable-shared' '--cache-file=/dev/null' '--srcdir=../../../../src/pm/hydra' 'CFLAGS= -g -O2' 'LDFLAGS=' 'LIBS=-lpthread ' 'CPPFLAGS= -I/Users/jrhammon/Work/MPI/MPICH/git/build/src/mpl/include -I/Users/jrhammon/Work/MPI/MPICH/git/src/mpl/include -I/Users/jrhammon/Work/MPI/MPICH/git/src/openpa/src -I/Users/jrhammon/Work/MPI/MPICH/git/build/src/openpa/src -D_REENTRANT -I/Users/jrhammon/Work/MPI/MPICH/git/build/src/mpi/romio/include' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs cobalt Checkpointing libraries available: Demux engines available: poll select -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Checkpoint/Restart problem
by Husen R 04 Apr '16

04 Apr '16

Dear Pavan, Thank you for your reply and information. I know you are one of the authorities of MPICH as your name is written in the userguide. in my previous email, I just want to make sure. Unfortunately, I really need this feature. Do you have any suggestion besides SCR. as far as i know, SCR needs modification to the code. Thank you in advance. Regards, Husen On Mon, Apr 4, 2016 at 11:54 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > I'm sure about the fact that MPICH doesn't support BLCR anymore. :-) > I'll make sure it's removed in the future versions of the userguide. > Thanks for pointing it out. > > BLCR kernel support was from my experience about an year ago, so things > might have changed in that. > > -- Pavan > > > On Apr 3, 2016, at 8:36 PM, Husen R <hus3nr(a)gmail.com> wrote: > > > > Hello Pavan, > > > > Thank you for your reply, > > > > Are you sure about that ? > > I have downloaded MPICH-3.2 (The latest version) . According to the > userguide, MPICH supports checkpoint/rollback fault tolerance and currently > only the BLCR checkpointing library is supported. We can see this > information at chapter 8 in this userguide > https://www.mpich.org/static/downloads/3.2/mpich-3.2-userguide.pdf. > > > > In addition, I also have downloaded the latest version of BLCR that > supports recent linux kernels. > > > > -Husen > > > > On Sun, Apr 3, 2016 at 10:20 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > Hello, > > > > BLCR checkpointing is no longer supported in MPICH. The BLCR kernel > module hasn't been updated to recent linux kernels, and move folks have > moved to alternate checkpointing infrastructure such as FTI or SCR. You > might want to consider doing that as well. > > > > -- Pavan > > > > > On Apr 1, 2016, at 9:51 PM, Husen R <hus3nr(a)gmail.com> wrote: > > > > > > Dear all, > > > > > > Please anyone tell me how to manually checkpoint mpiexec ? > > > I have followed the instruction in this link > https://wiki.mpich.org/mpich/index.php/Checkpointing. > > > I used the following command to send SIGUSR1. nothing happened. > > > > > > kill -SIGUSR1 [pid of mpiexec] > > > > > > thank you in advance > > > > > > regards, > > > > > > > > > Husen > > > _______________________________________________ > > > discuss mailing list discuss(a)mpich.org > > > To manage subscription options or unsubscribe: > > > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] ssh-less MPI using docker exec?
by Balaji, Pavan 04 Apr '16

04 Apr '16

Christian, Supporting a new launcher is fairly trivial in Hydra (mpich's process manager). You'd need to create something similar to 'ssh' in here: http://git.mpich.org/mpich.git/tree/HEAD:/src/pm/hydra/tools/bootstrap/exte… In fact, based on a google search, I found this: https://docs.docker.com/engine/reference/commandline/exec/ It looks like you can simply replace "ssh" with "docker exec" to get the required behavior? If that is correct, can you try launching mpich this way: % HYDRA_LAUNCHER_EXTRA_ARGS=exec mpiexec -hosts host1,host2 -launcher-exec 'docker' ./foo -- Pavan > On Apr 3, 2016, at 4:56 AM, Christian Kniep <christian(a)qnib.org> wrote: > > Hey Pavan, > > I sparked a discussion on the github repo of Open MPI: https://github.com/open-mpi/ompi/issues/1511 > But I assume that a docker-native execution model would also do for MPICH. > > Cheers > Christian > > On 3 April 2016 at 05:18:36, Balaji, Pavan (balaji(a)anl.gov) wrote: > >> >> Did you mean to send this to the Open MPI mailing list? >> >> -- Pavan >> >> > On Mar 31, 2016, at 12:56 PM, Christian Kniep <christian(a)qnib.org> wrote: >> > >> > Hey there, >> > >> > I tinkered around with a ssh-less openmpi and I was wondering if it could be implemented in a nicer way. http://qnib.org/2016/03/31/dssh/ >> > >> > I just used -mca plm_rsh_agent /opt/qnib/src/dssh with a rather simple script to use docker exec: >> > >> > #!/bin/bash >> > >> > REMOTE_HOST=$1 >> > shift >> > set -x >> > docker -H >> > unix:///var/run/docker.sock >> > exec -i -u ${USER} ${REMOTE_HOST} $@ >> > >> > Cheers Christian >> > >> > >> > >> > >> > >> > >> > _______________________________________________ >> > discuss mailing list discuss(a)mpich.org >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Checkpoint/Restart problem
by Balaji, Pavan 04 Apr '16

04 Apr '16

I'm sure about the fact that MPICH doesn't support BLCR anymore. :-) I'll make sure it's removed in the future versions of the userguide. Thanks for pointing it out. BLCR kernel support was from my experience about an year ago, so things might have changed in that. -- Pavan > On Apr 3, 2016, at 8:36 PM, Husen R <hus3nr(a)gmail.com> wrote: > > Hello Pavan, > > Thank you for your reply, > > Are you sure about that ? > I have downloaded MPICH-3.2 (The latest version) . According to the userguide, MPICH supports checkpoint/rollback fault tolerance and currently only the BLCR checkpointing library is supported. We can see this information at chapter 8 in this userguide https://www.mpich.org/static/downloads/3.2/mpich-3.2-userguide.pdf. > > In addition, I also have downloaded the latest version of BLCR that supports recent linux kernels. > > -Husen > > On Sun, Apr 3, 2016 at 10:20 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > Hello, > > BLCR checkpointing is no longer supported in MPICH. The BLCR kernel module hasn't been updated to recent linux kernels, and move folks have moved to alternate checkpointing infrastructure such as FTI or SCR. You might want to consider doing that as well. > > -- Pavan > > > On Apr 1, 2016, at 9:51 PM, Husen R <hus3nr(a)gmail.com> wrote: > > > > Dear all, > > > > Please anyone tell me how to manually checkpoint mpiexec ? > > I have followed the instruction in this link https://wiki.mpich.org/mpich/index.php/Checkpointing. > > I used the following command to send SIGUSR1. nothing happened. > > > > kill -SIGUSR1 [pid of mpiexec] > > > > thank you in advance > > > > regards, > > > > > > Husen > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Checkpoint/Restart problem
by Husen R 03 Apr '16

03 Apr '16

Hello Pavan, Thank you for your reply, Are you sure about that ? I have downloaded MPICH-3.2 (The latest version) . According to the userguide, MPICH supports checkpoint/rollback fault tolerance and currently only the BLCR checkpointing library is supported. We can see this information at chapter 8 in this userguide https://www.mpich.org/static/downloads/3.2/mpich-3.2-userguide.pdf. In addition, I also have downloaded the latest version of BLCR that supports recent linux kernels. -Husen On Sun, Apr 3, 2016 at 10:20 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > Hello, > > BLCR checkpointing is no longer supported in MPICH. The BLCR kernel > module hasn't been updated to recent linux kernels, and move folks have > moved to alternate checkpointing infrastructure such as FTI or SCR. You > might want to consider doing that as well. > > -- Pavan > > > On Apr 1, 2016, at 9:51 PM, Husen R <hus3nr(a)gmail.com> wrote: > > > > Dear all, > > > > Please anyone tell me how to manually checkpoint mpiexec ? > > I have followed the instruction in this link > https://wiki.mpich.org/mpich/index.php/Checkpointing. > > I used the following command to send SIGUSR1. nothing happened. > > > > kill -SIGUSR1 [pid of mpiexec] > > > > thank you in advance > > > > regards, > > > > > > Husen > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] ssh-less MPI using docker exec?
by Christian Kniep 03 Apr '16

03 Apr '16

Hey Pavan, I sparked a discussion on the github repo of Open MPI: https://github.com/open-mpi/ompi/issues/1511 But I assume that a docker-native execution model would also do for MPICH. Cheers Christian On 3 April 2016 at 05:18:36, Balaji, Pavan (balaji(a)anl.gov) wrote: Did you mean to send this to the Open MPI mailing list? -- Pavan > On Mar 31, 2016, at 12:56 PM, Christian Kniep <christian(a)qnib.org> wrote: > > Hey there, > > I tinkered around with a ssh-less openmpi and I was wondering if it could be implemented in a nicer way. http://qnib.org/2016/03/31/dssh/ > > I just used -mca plm_rsh_agent /opt/qnib/src/dssh with a rather simple script to use docker exec: > > #!/bin/bash > > REMOTE_HOST=$1 > shift > set -x > docker -H > unix:///var/run/docker.sock > exec -i -u ${USER} ${REMOTE_HOST} $@ > > Cheers Christian > > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Checkpoint/Restart problem
by Balaji, Pavan 02 Apr '16

02 Apr '16

Hello, BLCR checkpointing is no longer supported in MPICH. The BLCR kernel module hasn't been updated to recent linux kernels, and move folks have moved to alternate checkpointing infrastructure such as FTI or SCR. You might want to consider doing that as well. -- Pavan > On Apr 1, 2016, at 9:51 PM, Husen R <hus3nr(a)gmail.com> wrote: > > Dear all, > > Please anyone tell me how to manually checkpoint mpiexec ? > I have followed the instruction in this link https://wiki.mpich.org/mpich/index.php/Checkpointing. > I used the following command to send SIGUSR1. nothing happened. > > kill -SIGUSR1 [pid of mpiexec] > > thank you in advance > > regards, > > > Husen > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] ssh-less MPI using docker exec?
by Balaji, Pavan 02 Apr '16

02 Apr '16

Did you mean to send this to the Open MPI mailing list? -- Pavan > On Mar 31, 2016, at 12:56 PM, Christian Kniep <christian(a)qnib.org> wrote: > > Hey there, > > I tinkered around with a ssh-less openmpi and I was wondering if it could be implemented in a nicer way. http://qnib.org/2016/03/31/dssh/ > > I just used -mca plm_rsh_agent /opt/qnib/src/dssh with a rather simple script to use docker exec: > > #!/bin/bash > > REMOTE_HOST=$1 > shift > set -x > docker -H > unix:///var/run/docker.sock > exec -i -u ${USER} ${REMOTE_HOST} $@ > > Cheers Christian > > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] uninitialized MPI_Status object
by יוסף אלון 02 Apr '16

02 Apr '16

בס"ד first thank you! your program works with the command: "mpiexec -n 1 ./get_count" but when i put: "mpiexec -n 4 ./get_count" i receive: [image: תמונה מוטבעת 1] what does it means? 2016-03-30 16:56 GMT+03:00 Kenneth Raffenetti <raffenet(a)mcs.anl.gov>: > Try "mpiexec -n 1 ./get_count". > > On 03/30/2016 08:46 AM, יוסף אלון wrote: > >> i don't know why, but the first command works and the second command isnt. >> i have the following error: >> תמונה מוטבעת 1 >> >> what this means? >> >> 2016-03-28 21:15 GMT+03:00 Jeff Hammond <jeff.science(a)gmail.com >> <mailto:[email protected]>>: >> >> I created a ticket for this: >> https://trac.mpich.org/projects/mpich/ticket/2332. >> >> Jeff >> >> On Mon, Mar 28, 2016 at 10:26 AM, Wei-keng Liao >> <wkliao(a)eecs.northwestern.edu <mailto:[email protected]>> >> wrote: >> >> My test program can be compiled with command "mpicc get_count.c >> -o get_count" >> and run "mpiexec -n 4 get_count". For correct (expected) result, >> nothing >> will be printed on stdout. Otherwise, error messages will be >> printed. >> >> My point is MPI_Get_count does not report correct result because >> MPI collective I/O call fails to initialize the MPI_Status object. >> >> I found that OpenMPI and earlier version of MPICH (2-1.2.1) can >> run >> this test code correctly. >> >> The test failed when I ran MPICH 3.1.4 and the current from GIT >> repo. >> >> Wei-keng >> >> On Mar 26, 2016, at 3:21 PM, יוסף אלון wrote: >> >> > בס"ד >> > >> > becuse all the command confusion. >> > which command should i put to run the above code? >> > >> > 2016-03-26 3:31 GMT+03:00 Wei-keng Liao >> <wkliao(a)eecs.northwestern.edu >> <mailto:[email protected]>>: >> > >> > I notice an uninitialized MPI_Status object can make >> MPI_Get_count return a wrong result, >> > when the data amount in the MPI operation is zero (although I >> only checked MPI-IO). >> > Attached is a test program that uses MPI collective read >> where only root process has >> > non-zero length data to read. The expected result from >> MPI_Get_count is 0 for all >> > non-root process. To mimic an uninitialized MPI_Status >> object, I call memset to make >> > the object non-zero. >> > >> > Here is the code fragment. >> > >> > if (rank == 0) len = 10; >> > else len = 0; >> > >> > MPI_File_read_all(fh, buf, len, MPI_BYTE, &status); >> > >> > MPI_Get_count(&status, MPI_BYTE, &get_size); >> > >> > For process rank > 0, get_size may not be 0. >> > >> > >> > Wei-keng >> > >> > >> > _______________________________________________ >> > discuss mailing list discuss(a)mpich.org <mailto: >> discuss(a)mpich.org> >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/discuss >> > >> > >> > >> > -- >> > בברכה, יוסף אלון >> > 050-4916740 >> > _______________________________________________ >> > discuss mailing list discuss(a)mpich.org <mailto: >> discuss(a)mpich.org> >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com <mailto:[email protected]> >> http://jeffhammond.github.io/ >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> >> >> -- >> בברכה, יוסף אלון >> 050-4916740 >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- בברכה, יוסף אלון 050-4916740 _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0