October 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] new communicator value differs for myid=0
by Rajeev Thakur 14 Oct '13

14 Oct '13

The communicator is an opaque object. The value of the new communicator variable is not useful. It is a local variable and can be different on each process. As long as the new communicator behaves the way it should, the program is ok. Rajeev On Oct 14, 2013, at 2:50 PM, Shan-ho Tsai wrote: > > Hello, > I am using MPICH2 3.0.2 with gfortran 4.4.7 on a 64-bit > Linux machine. The program uses MPI_Comm_split to > create a new communicator. All processes print the same > value for MPI_COMM_WORLD, but for the new communicator, > all processes with myid greater than zero print the same value, > but myid=0 prints a different value. Why is that? I thought > all processes should print the same number as well. > > Simple test code: > =================================================== > PROGRAM Test > IMPLICIT NONE > include "mpif.h" > INTEGER :: nBins=8, nWalkers=8 > INTEGER :: ierr,numprocs,myid,energy_comm > INTEGER :: energy_id,numranks,iBin,iWalker > INTEGER :: mpistatus(MPI_STATUS_SIZE) > > CALL MPI_INIT(ierr) > CALL MPI_COMM_SIZE(MPI_COMM_WORLD,numprocs,ierr) > CALL MPI_COMM_RANK(MPI_COMM_WORLD,myid,ierr) > > iBin = myid/nWalkers > iWalker = MOD(myid,nWalkers) > CALL MPI_Comm_split(MPI_COMM_WORLD,iBin,iWalker,energy_comm,ierr) > CALL MPI_COMM_SIZE(energy_comm,numranks,ierr) > CALL MPI_COMM_RANK(energy_comm,energy_id,ierr) > > write(*,*)myid,MPI_COMM_WORLD,energy_comm > > CALL MPI_FINALIZE(ierr) > > END PROGRAM Test > ========================================================= > > Compile with > > mpif90 test.f90 > > Run with > > mpirun -n 4 -f host.txt ./a.out > > Generates > 0 1140850688 -2080374780 > 2 1140850688 -2080374782 > 1 1140850688 -2080374782 > 3 1140850688 -2080374782 > > "mpif90 -v" returns: > > mpif90 for MPICH version 3.0.2 > Using built-in specs. > Target: x86_64-redhat-linux6E > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --disable-gnu-unique-object --with-as=/usr/libexec/binutils220/as --enable-languages=c,c++,fortran --disable-libgcj --with-mpfr=/builddir/build/BUILD/gcc-4.4.7-20120601/obj-x86_64-redhat-linux6E/mpfr-install/ --with-ppl=/builddir/build/BUILD/gcc-4.4.7-20120601/obj-x86_64-redhat-linux6E/ppl-install --with-cloog=/builddir/build/BUILD/gcc-4.4.7-20120601/obj-x86_64-redhat-linux6E/cloog-install --with-tune=generic --with-arch_32=i586 --build=x86_64-redhat-linux6E > Thread model: posix > gcc version 4.4.7 20120313 (Red Hat 4.4.7-1) (GCC) > > I get the same output for MPICH2 1.4.1p1. > Any ideas what is happening? Or shouldn't the new communicator also > have the same value for all myids? > > Thank you very much for any suggestions. > Shan-Ho > > ---------------------------------------------------- > Shan-Ho Tsai > University of Georgia > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

new communicator value differs for myid=0
by Shan-ho Tsai 14 Oct '13

14 Oct '13

Hello, I am using MPICH2 3.0.2 with gfortran 4.4.7 on a 64-bit Linux machine. The program uses MPI_Comm_split to create a new communicator. All processes print the same value for MPI_COMM_WORLD, but for the new communicator, all processes with myid greater than zero print the same value, but myid=0 prints a different value. Why is that? I thought all processes should print the same number as well. Simple test code: =================================================== PROGRAM Test IMPLICIT NONE include "mpif.h" INTEGER :: nBins=8, nWalkers=8 INTEGER :: ierr,numprocs,myid,energy_comm INTEGER :: energy_id,numranks,iBin,iWalker INTEGER :: mpistatus(MPI_STATUS_SIZE) CALL MPI_INIT(ierr) CALL MPI_COMM_SIZE(MPI_COMM_WORLD,numprocs,ierr) CALL MPI_COMM_RANK(MPI_COMM_WORLD,myid,ierr) iBin = myid/nWalkers iWalker = MOD(myid,nWalkers) CALL MPI_Comm_split(MPI_COMM_WORLD,iBin,iWalker,energy_comm,ierr) CALL MPI_COMM_SIZE(energy_comm,numranks,ierr) CALL MPI_COMM_RANK(energy_comm,energy_id,ierr) write(*,*)myid,MPI_COMM_WORLD,energy_comm CALL MPI_FINALIZE(ierr) END PROGRAM Test ========================================================= Compile with mpif90 test.f90 Run with mpirun -n 4 -f host.txt ./a.out Generates 0 1140850688 -2080374780 2 1140850688 -2080374782 1 1140850688 -2080374782 3 1140850688 -2080374782 "mpif90 -v" returns: mpif90 for MPICH version 3.0.2 Using built-in specs. Target: x86_64-redhat-linux6E Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --disable-gnu-unique-object --with-as=/usr/libexec/binutils220/as --enable-languages=c,c++,fortran --disable-libgcj --with-mpfr=/builddir/build/BUILD/gcc-4.4.7-20120601/obj-x86_64-redhat-linux6E/mpfr-install/ --with-ppl=/builddir/build/BUILD/gcc-4.4.7-20120601/obj-x86_64-redhat-linux6E/ppl-install --with-cloog=/builddir/build/BUILD/gcc-4.4.7-20120601/obj-x86_64-redhat-linux6E/cloog-install --with-tune=generic --with-arch_32=i586 --build=x86_64-redhat-linux6E Thread model: posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-1) (GCC) I get the same output for MPICH2 1.4.1p1. Any ideas what is happening? Or shouldn't the new communicator also have the same value for all myids? Thank you very much for any suggestions. Shan-Ho ---------------------------------------------------- Shan-Ho Tsai University of Georgia

1 0

Re: [mpich-discuss] How to set mpif90 compiler
by Jeff Hammond 14 Oct '13

14 Oct '13

The error is obvious and has nothing to do with MPI; you are using Fortran flags that only work with the Intel compiler: "> gfortran: error: unrecognized command line option ‘-mp1’" Furthermore, you shouldn't even be using that flag with ifort anymore: software.intel.com/en-us/forums/topic/268838‎ Change your flags appropriately and your application may build. Jeff On Mon, Oct 14, 2013 at 9:47 AM, Hana Milani <hana.milani(a)yahoo.com> wrote: > Hello, > > Sorry if this is not directly related to MPICH. > > The file I have enclosed is a shell script as a part of a package that > should be modified according to compiler settings in my computer to get the > code run. > > I have MPICH installed with mpif90 and mpicc both in the following path: > /usr/local/bin/mpif90\mpicc which I have already mentioned in the script > replacing the ifort. > > However, the "make" command is giving me the following error: > > $ make > /bin/sh getmachine gulp > make[1]: Entering directory `/home/Packages/GULP/Src/Linux' > /usr/local/bin/mpif90 -O3 -mp1 -I.. -DFLUSH -c ../datatypes.F90 > gfortran: error: unrecognized command line option ‘-mp1’ > make[1]: *** [datatypes.o] Error 1 > make[1]: Leaving directory `/home/Packages/GULP/Src/Linux' > mv: cannot stat ‘gulp’: No such file or directory > make: *** [gulp] Error 1 > > I'd be really grateful if you take a look at the script and tell me where I > am going wrong. > Regards, > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com

1 0

How to set mpif90 compiler
by Hana Milani 14 Oct '13

14 Oct '13

Hello, Sorry if this is not directly related to MPICH. The file I have enclosed is a shell script as a part of a package that should be modified according to compiler settings in my computer to get the code run. I have MPICH installed with mpif90 and mpicc both in the following path: /usr/local/bin/mpif90\mpicc which I have already mentioned in the script replacing the ifort. However, the "make" command is giving me the following error: $ make /bin/sh getmachine gulp make[1]: Entering directory `/home/Packages/GULP/Src/Linux' /usr/local/bin/mpif90 -O3 -mp1 -I.. -DFLUSH -c ../datatypes.F90 gfortran: error: unrecognized command line option ‘-mp1’ make[1]: *** [datatypes.o] Error 1 make[1]: Leaving directory `/home/Packages/GULP/Src/Linux' mv: cannot stat ‘gulp’: No such file or directory make: *** [gulp] Error 1 I'd be really grateful if you take a look at the script and tell me where I am going wrong. Regards,

1 0

Bugs in Hydra-KVS
by Jan Bierbaum 14 Oct '13

14 Oct '13

Hello! Doing some experiments with an older version of MPICH I stumbled upon two bugs in 'HYD_pmcd_pmi_add_kvs', both of which are still present according to Git [1]: The loop traverses all KVS entries and checks for one with a duplicate key. However, it will not check the last entry (Bug 1). When a duplicate entry is found the loop is aborted. But the newly created 'key_pair' is never freed (Bug 2). I attached a patch that fixes both issues. Also, newer versions of MPICH do not check anymore whether a duplicate KVS entry was encountered - 'HYD_status fn_put' ignores 'ret' completely [2] whereas it did some error checking and reporting in older versions. This change was introduced in commit b98c7fd361541659d7e8fb6ef7fc76547024ad75 [3] and I am not sure whether this check was removed intentionally or rather by accident. Last but not least the README in the release tarball is a bit outdated and still refers to the old site, mailing list and SVN repository. I attach another patch that updates the file. I also propose to add a link to the Git Wiki page [4] directly at the end of the download page [5]. It took me quiet a while to realize the repository was publicly available. Regards, Jan [1] http://git.mpich.org/mpich.git/blob/HEAD:/src/pm/hydra/pm/pmiserv/common.c#… [2] http://git.mpich.org/mpich.git/blob/HEAD:/src/pm/hydra/pm/pmiserv/pmiserv_p… [3] http://git.mpich.org/mpich.git/commit/b98c7fd361541659d7e8fb6ef7fc76547024a… [4] https://wiki.mpich.org/mpich/index.php/Getting_And_Building_MPICH [5] https://www.mpich.org/downloads/

1 0

Re: [mpich-discuss] how to make mpich crash hard so I can gdb bt it?
by Jeff Hammond 11 Oct '13

11 Oct '13

Thanks for all the pointers. Jed's response indicates that I need to learn how to use gdb better, but I'll cache all of these suggestions since I imagine they are all useful in some situations. Best, Jeff On Fri, Oct 11, 2013 at 5:22 PM, David Goodell (dgoodell) <dgoodell(a)cisco.com> wrote: > On Oct 11, 2013, at 4:44 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > >> I apologize if this is a stupid question, but I would like MPICH to >> crash and burn rather than terminate gently when an error occurs so >> that I can gdb bt it to know where the error is in my program, since I >> am absolutely certain that this error is in the application and not >> MPICH. >> >> Ideally, I would be able to select the failure mode at runtime, e.g. >> MPICH_FAILURE_MODE={hard,soft}, so that I can get either the nice >> MPICH trace when I think MPICH is the problem and a brutal >> light-the-machine-room-on-fire-and-abort-the-universe error when I >> think it's my fault. > > For assertion failures, I think I used to change MPIR_Assert_fail/MPIR_Assert_fail_fmt to just call "abort()" instead of MPID_Abort. As long as my ulimit settings were correct then I'd get a core file that I could load in the debugger. The process manager will take care of cleaning up the other processes anyway. > > IIRC, you can perform a similar trick with MPIR_Err_create_code to catch error codes at the time they are created, though that may not be exactly the same spot that you want to examine. > > -Dave > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] how to make mpich crash hard so I can gdb bt it?
by David Goodell (dgoodell) 11 Oct '13

11 Oct '13

On Oct 11, 2013, at 4:44 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > I apologize if this is a stupid question, but I would like MPICH to > crash and burn rather than terminate gently when an error occurs so > that I can gdb bt it to know where the error is in my program, since I > am absolutely certain that this error is in the application and not > MPICH. > > Ideally, I would be able to select the failure mode at runtime, e.g. > MPICH_FAILURE_MODE={hard,soft}, so that I can get either the nice > MPICH trace when I think MPICH is the problem and a brutal > light-the-machine-room-on-fire-and-abort-the-universe error when I > think it's my fault. For assertion failures, I think I used to change MPIR_Assert_fail/MPIR_Assert_fail_fmt to just call "abort()" instead of MPID_Abort. As long as my ulimit settings were correct then I'd get a core file that I could load in the debugger. The process manager will take care of cleaning up the other processes anyway. IIRC, you can perform a similar trick with MPIR_Err_create_code to catch error codes at the time they are created, though that may not be exactly the same spot that you want to examine. -Dave

1 0

Re: [mpich-discuss] how to make mpich crash hard so I can gdb bt it?
by Jed Brown 11 Oct '13

11 Oct '13

Ken Raffenetti <raffenet(a)mcs.anl.gov> writes: > MPI_Comm_set_errhandler(MPI_COMM_WORLD, MPI_ERRORS_RETURN); This is less useful in a debugger because you don't get to see the MPICH stack that may give more information about what caused the function to fail. You can always walk up the stack in the debugger.

1 0

Re: [mpich-discuss] how to make mpich crash hard so I can gdb bt it?
by Ken Raffenetti 11 Oct '13

11 Oct '13

Try: MPI_Comm_set_errhandler(MPI_COMM_WORLD, MPI_ERRORS_RETURN); Or similar for your communicator at runtime. ----- Original Message ----- > From: "Jeff Hammond" <jeff.science(a)gmail.com> > To: "MPICH" <discuss(a)mpich.org> > Sent: Friday, October 11, 2013 4:44:52 PM > Subject: [mpich-discuss] how to make mpich crash hard so I can gdb bt it? > > I apologize if this is a stupid question, but I would like MPICH to > crash and burn rather than terminate gently when an error occurs so > that I can gdb bt it to know where the error is in my program, since > I > am absolutely certain that this error is in the application and not > MPICH. > > Ideally, I would be able to select the failure mode at runtime, e.g. > MPICH_FAILURE_MODE={hard,soft}, so that I can get either the nice > MPICH trace when I think MPICH is the problem and a brutal > light-the-machine-room-on-fire-and-abort-the-universe error when I > think it's my fault. > > Thanks, > > Jeff > > <running in gdb> > Assertion failed in file ../src/mpi/coll/helper_fns.c at line 81: > FALSE > memcpy argument memory ranges overlap, dst_=0x101401000 > src_=0x101401000 len_=400 > > internal ABORT - process 0 > Fatal error in MPI_Win_free: Other MPI error, error stack: > MPI_Win_free(134)..........: MPI_Win_free(win=0x10000b394) failed > MPIDI_CH3_SHM_Win_free(103): pthread mutex routine failed (Resource > busy) > > Program exited with code 01. > (gdb) bt > > $ mpichversion > MPICH Version: 3.1b2 > MPICH Release date: unreleased development copy > MPICH Device: ch3:nemesis > MPICH configure: CC=clang CXX=clang++ FC=/bin/false --disable-fc > --disable-f77 --enable-threads --enable-g --with-pm=hydra > --prefix=/opt/mpich/dev/clang > MPICH CC: clang -g -O2 > MPICH CXX: clang++ -g -O2 > MPICH F77: no -g > MPICH FC: no -g > > -- > Jeff Hammond > jeff.science(a)gmail.com > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] how to make mpich crash hard so I can gdb bt it?
by Jed Brown 11 Oct '13

11 Oct '13

Jeff Hammond <jeff.science(a)gmail.com> writes: > I apologize if this is a stupid question, but I would like MPICH to > crash and burn rather than terminate gently when an error occurs so > that I can gdb bt it to know where the error is in my program, since I > am absolutely certain that this error is in the application and not > MPICH. Why not just set your breakpoint? (gdb) b MPID_Abort

1 0