April 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Memory leak comparing communicators
by "Antonio J. Peña" 28 Apr '14

28 Apr '14

Hi Joost, Can you share the smallest fragment of code with which we can reproduce this? Antonio On 04/28/2014 11:25 AM, VandeVondele Joost wrote: > Hi, > > using mpich 3.1 I notice that my application leaks memory. Compiling it with '-O1 -g -fno-omit-frame-pointer -fsanitize=leak' using gcc 4.9 it appears the sources is comparing communicators, for example: > > [0] Direct leak of 38400 byte(s) in 600 object(s) allocated from: > [0] #0 0x7f84298d53a8 in __interceptor_malloc ../../../../gcc/libsanitizer/lsan/lsan_interceptors.cc:66 > [0] #1 0x7f842aabaf6b in MPIR_Group_create /data/vjoost/mpich-3.1/src/mpi/group/grouputil.c:77 > [0] #2 0x7f842ab88314 in MPIR_Comm_group_impl /data/vjoost/mpich-3.1/src/mpi/comm/comm_group.c:44 > [0] #3 0x7f842ab840d3 in PMPI_Comm_compare /data/vjoost/mpich-3.1/src/mpi/comm/comm_compare.c:122 > [0] #4 0x7f842aac8e84 in pmpi_comm_compare_ /data/vjoost/mpich-3.1/src/binding/f77/comm_comparef.c:267 > [0] #5 0x1bfd447 in __message_passing_MOD_mp_comm_compare /data/vjoost/clean/cp2k/cp2k/src/common/message_passing.F:1084 > > I'm not seeing other leaks, so is guess something particular is going on here. Any suggestions on a possible fix ? > > Thanks, > > Joost > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Antonio J. Peï¿½a Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya

1 0

Memory leak comparing communicators
by VandeVondele Joost 28 Apr '14

28 Apr '14

Hi, using mpich 3.1 I notice that my application leaks memory. Compiling it with '-O1 -g -fno-omit-frame-pointer -fsanitize=leak' using gcc 4.9 it appears the sources is comparing communicators, for example: [0] Direct leak of 38400 byte(s) in 600 object(s) allocated from: [0] #0 0x7f84298d53a8 in __interceptor_malloc ../../../../gcc/libsanitizer/lsan/lsan_interceptors.cc:66 [0] #1 0x7f842aabaf6b in MPIR_Group_create /data/vjoost/mpich-3.1/src/mpi/group/grouputil.c:77 [0] #2 0x7f842ab88314 in MPIR_Comm_group_impl /data/vjoost/mpich-3.1/src/mpi/comm/comm_group.c:44 [0] #3 0x7f842ab840d3 in PMPI_Comm_compare /data/vjoost/mpich-3.1/src/mpi/comm/comm_compare.c:122 [0] #4 0x7f842aac8e84 in pmpi_comm_compare_ /data/vjoost/mpich-3.1/src/binding/f77/comm_comparef.c:267 [0] #5 0x1bfd447 in __message_passing_MOD_mp_comm_compare /data/vjoost/clean/cp2k/cp2k/src/common/message_passing.F:1084 I'm not seeing other leaks, so is guess something particular is going on here. Any suggestions on a possible fix ? Thanks, Joost

1 0

mpi windows with groups mpich2
by Helen Kershaw 28 Apr '14

28 Apr '14

Hi, I'm trying to use MPI one-sided communication with sub-communicators. I've put my test code at the end of the email. It runs on 8 processors, creates two communicators, makes a window on the communicators, then tries to do some one-sided communication. I can use groups OR windows without a problem, but when I use them together (in the test code) I get a coredump from the first 4 tasks. I think the one-sided communication may be using the local rank in the sub-communicator (mpi_comm_grid) as if it is the global rank in mpi_comm_world. I can run the test code ok using open-mpi, but not mpich2. I might be missing something obvious. Thanks for your help, Helen ----- program test_group_win use mpi implicit none ! mpi integer :: rank, size, ierr, status(mpi_status_size) ! groups integer, parameter :: group_size = 4 integer :: group_members(group_size) integer :: group_all ! mpi_comm_world group integer :: subgroup integer :: mpi_comm_grid integer :: local_rank ! rank within subgroup ! window integer :: win integer ::bytesize !> size in bytes of each element in the window integer(KIND=MPI_ADDRESS_KIND) :: window_size pointer(aa, duplicate) real :: duplicate(*) integer(KIND=MPI_ADDRESS_KIND) :: target_disp !> displacement in window integer :: owner real :: result call mpi_init(ierr) call mpi_comm_size(mpi_comm_world, size, ierr) call mpi_comm_rank(mpi_comm_world, rank, ierr) ! create groups call mpi_comm_group(mpi_comm_world, group_all, ierr) if (rank < 4 ) then group_members = (/0,1,2,3/) else group_members = (/4,5,6,7/) endif call mpi_group_incl(group_all, group_size, group_members, subgroup, ierr) call mpi_comm_create(mpi_comm_world, subgroup, mpi_comm_grid, ierr) call mpi_comm_rank(mpi_comm_grid, local_rank, ierr) ! rank within group ! create window call mpi_type_size(mpi_real, bytesize, ierr) window_size = bytesize ! one element in the window aa = malloc(1) call MPI_ALLOC_MEM(window_size, mpi_info_null, aa, ierr) duplicate(1) = rank call mpi_win_create(duplicate, window_size, bytesize, MPI_INFO_NULL, mpi_comm_grid, win, ierr) ! grabbing data from rank above if (rank < 3 ) then owner = rank + 1 elseif (rank < 7) then owner = rank + 1 - 4 else owner = 0 endif print*, 'rank owner', rank, owner target_disp = 0 call mpi_win_lock(MPI_LOCK_SHARED, owner, 0, win, ierr) call mpi_get(result, 1, mpi_real, owner, target_disp, 1, mpi_real, win, ierr) call mpi_win_unlock(owner, win, ierr) print*, 'rank, result ', rank, result ! free window call mpi_win_free(win, ierr) call mpi_free_mem(duplicate, ierr) call mpi_finalize(ierr) end program test_group_win ----

1 0

Re: [mpich-discuss] mpi_put works only with target_displacement=0
by Ted Sariyski 27 Apr '14

27 Apr '14

Thanks Rajeev! It works. --Ted On 04/26/2014 10:47 PM, Rajeev Thakur wrote: > The size parameter to MPI_Win_create should be the size in bytes, and the disp_unit parameter should be sizeofreal, not mpi_real. > > Rajeev > > On Apr 26, 2014, at 9:08 PM, Ted Sariyski <tsariysk(a)craft-tech.com> > wrote: > >> Attached is a test program (mpi_alloc_mem is called from alloc.c). >> 'run' will compile tst.f90 and alloc.c, and run the code on two cpus. If line 99, "target_displacement=0", is commented out, code breaks. >> >> /> mpif90 --version >> GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4) >> >> Thanks for the prompt response. >> --Ted >> >>> Can you send a small test program that can be compiled and run? >>> Rajeev >> >> >> On 04/26/2014 11:03 AM, Ted Sariyski wrote: >>> HI, >>> I am trying to apply RMA so that each process evaluates its segment of data and PUTs it at process 0. Here is what I do (see attachment): >>> >>> 1. Process 0 creates a window win0 with size=GLOBAL_SIZE of the array >>> 2. Rest of processes create NULL() win0 with size 0. >>> 3. Eeach process evaluates it portion dat with size myCount >>> 4. All create mpi_shared_lock on 0 (dat on different process are strictly non-overlapping) >>> 5. Put dat on 0 with appropriate offset (myOffset) >>> >>> Code works on two cpus ONLY when target_displacement=0. If target_displacement=myOffset, I get: >>> >>>> rank 0 in job 383 craft28.craft-tech.com_57409 caused collective abort of all ranks >>>> exit status of rank 0: killed by signal 11 >>> What do I miss here? >>> >>> Any help will be highly appreciated. Thank in advance, >>> --Ted >>> >>> P.S. Mixed C and fortran90 with mpich2.1.5 on Linux/x86_64 (CentOS 6.5) >>> >>> >>> ... >>> integer(mpi_address_kind) :: size=0,target_displacement=0 >>> if ( myid == 0 ) then >>> call mpi_win_create(mdat,size,mpi_realsize,info,comm,win0,ierr) >>> else >>> call mpi_win_create(NULL(),0,1,info,comm,win0,ierr) >>> end if >>> ... >>> target_rank=0 >>> call mpi_win_lock(mpi_lock_shared,target_rank,mpi_mode_nocheck,win0,ierr) >>> ... >>> origin_count=myCount >>> target_count=myCount >>> target_displacement=myOffset >>> ... >>> call mpi_put(ldat,origin_count,mpi_real,target_rank,target_displacement,target_count,mpi_real,win0,ierr) >>> ...

1 0

Re: [mpich-discuss] mpi_put works only with target_displacement=0
by Rajeev Thakur 26 Apr '14

26 Apr '14

The size parameter to MPI_Win_create should be the size in bytes, and the disp_unit parameter should be sizeofreal, not mpi_real. Rajeev On Apr 26, 2014, at 9:08 PM, Ted Sariyski <tsariysk(a)craft-tech.com> wrote: > Attached is a test program (mpi_alloc_mem is called from alloc.c). > 'run' will compile tst.f90 and alloc.c, and run the code on two cpus. If line 99, "target_displacement=0", is commented out, code breaks. > > /> mpif90 --version > GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4) > > Thanks for the prompt response. > --Ted > >> Can you send a small test program that can be compiled and run? >> Rajeev > > > > On 04/26/2014 11:03 AM, Ted Sariyski wrote: >> HI, >> I am trying to apply RMA so that each process evaluates its segment of data and PUTs it at process 0. Here is what I do (see attachment): >> >> 1. Process 0 creates a window win0 with size=GLOBAL_SIZE of the array >> 2. Rest of processes create NULL() win0 with size 0. >> 3. Eeach process evaluates it portion dat with size myCount >> 4. All create mpi_shared_lock on 0 (dat on different process are strictly non-overlapping) >> 5. Put dat on 0 with appropriate offset (myOffset) >> >> Code works on two cpus ONLY when target_displacement=0. If target_displacement=myOffset, I get: >> >> > rank 0 in job 383 craft28.craft-tech.com_57409 caused collective abort of all ranks >> > exit status of rank 0: killed by signal 11 >> >> What do I miss here? >> >> Any help will be highly appreciated. Thank in advance, >> --Ted >> >> P.S. Mixed C and fortran90 with mpich2.1.5 on Linux/x86_64 (CentOS 6.5) >> >> >> ... >> integer(mpi_address_kind) :: size=0,target_displacement=0 >> if ( myid == 0 ) then >> call mpi_win_create(mdat,size,mpi_realsize,info,comm,win0,ierr) >> else >> call mpi_win_create(NULL(),0,1,info,comm,win0,ierr) >> end if >> ... >> target_rank=0 >> call mpi_win_lock(mpi_lock_shared,target_rank,mpi_mode_nocheck,win0,ierr) >> ... >> origin_count=myCount >> target_count=myCount >> target_displacement=myOffset >> ... >> call mpi_put(ldat,origin_count,mpi_real,target_rank,target_displacement,target_count,mpi_real,win0,ierr) >> ... > > > -- > Ted Sariyski > Senior Research Scientist > Combustion Research & Flow Technology, Inc. (CRAFT Tech) > 6210 Keller's Church Road, Pipersville, PA 18947-1020 > Voice: (215) 766-1520 x14; Fax: (215) 766-1524 > http://www.craft-tech.com > > <tst.gz>_______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpi_put works only with target_displacement=0
by Ted Sariyski 26 Apr '14

26 Apr '14

Attached is a test program (mpi_alloc_mem is called from alloc.c). 'run' will compile tst.f90 and alloc.c, and run the code on two cpus. If line 99, "target_displacement=0", is commented out, code breaks. /> mpif90 --version GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4) Thanks for the prompt response. --Ted > Can you send a small test program that can be compiled and run? > Rajeev On 04/26/2014 11:03 AM, Ted Sariyski wrote: > HI, > I am trying to apply RMA so that each process evaluates its segment of > data and PUTs it at process 0. Here is what I do (see attachment): > > 1. Process 0 creates a window win0 with size=GLOBAL_SIZE of the array > 2. Rest of processes create NULL() win0 with size 0. > 3. Eeach process evaluates it portion dat with size myCount > 4. All create mpi_shared_lock on 0 (dat on different process are > strictly non-overlapping) > 5. Put dat on 0 with appropriate offset (myOffset) > > Code works on two cpus ONLY when target_displacement=0. If > target_displacement=myOffset, I get: > > > rank 0 in job 383 craft28.craft-tech.com_57409 caused collective > abort of all ranks > > exit status of rank 0: killed by signal 11 > > What do I miss here? > > Any help will be highly appreciated. Thank in advance, > --Ted > > P.S. Mixed C and fortran90 with mpich2.1.5 on Linux/x86_64 (CentOS > 6.5) > > > ... > integer(mpi_address_kind) :: size=0,target_displacement=0 > if ( myid == 0 ) then > call mpi_win_create(mdat,size,mpi_realsize,info,comm,win0,ierr) > else > call mpi_win_create(NULL(),0,1,info,comm,win0,ierr) > end if > ... > target_rank=0 > call > mpi_win_lock(mpi_lock_shared,target_rank,mpi_mode_nocheck,win0,ierr) > ... > origin_count=myCount > target_count=myCount > target_displacement=myOffset > ... > call > mpi_put(ldat,origin_count,mpi_real,target_rank,target_displacement,target_count,mpi_real,win0,ierr) > ... -- Ted Sariyski Senior Research Scientist Combustion Research & Flow Technology, Inc. (CRAFT Tech) 6210 Keller's Church Road, Pipersville, PA 18947-1020 Voice: (215) 766-1520 x14; Fax: (215) 766-1524 http://www.craft-tech.com

1 0

Re: [mpich-discuss] mpi_put works only with target_displacement=0
by Balaji, Pavan 26 Apr '14

26 Apr '14

As Rajeev mentioned, a test program is most useful. The most common error in these kind of programs is the displacement unit provided to win_create. Remember that the offset given to PUT is multiplied with the displacement unit given to win_create. — Pavan On Apr 26, 2014, at 10:58 AM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > Can you send a small test program that can be compiled and run? > > Rajeev > > On Apr 26, 2014, at 10:03 AM, Ted Sariyski <tsariysk(a)craft-tech.com> wrote: > >> HI, >> I am trying to apply RMA so that each process evaluates its segment of data and PUTs it at process 0. Here is what I do (see attachment): >> >> 1. Process 0 creates a window win0 with size=GLOBAL_SIZE of the array >> 2. Rest of processes create NULL() win0 with size 0. >> 3. Eeach process evaluates it portion dat with size myCount >> 4. All create mpi_shared_lock on 0 (dat on different process are strictly non-overlapping) >> 5. Put dat on 0 with appropriate offset (myOffset) >> >> Code works on two cpus ONLY when target_displacement=0. If target_displacement=myOffset, I get: >> >>> rank 0 in job 383 craft28.craft-tech.com_57409 caused collective abort of all ranks >>> exit status of rank 0: killed by signal 11 >> >> What do I miss here? >> >> Any help will be highly appreciated. Thank in advance, >> --Ted >> >> P.S. Mixed C and fortran90 with mpich2.1.5 on Linux/x86_64 (CentOS 6.5) >> >> >> ... >> integer(mpi_address_kind) :: size=0,target_displacement=0 >> if ( myid == 0 ) then >> call mpi_win_create(mdat,size,mpi_realsize,info,comm,win0,ierr) >> else >> call mpi_win_create(NULL(),0,1,info,comm,win0,ierr) >> end if >> ... >> target_rank=0 >> call mpi_win_lock(mpi_lock_shared,target_rank,mpi_mode_nocheck,win0,ierr) >> ... >> origin_count=myCount >> target_count=myCount >> target_displacement=myOffset >> ... >> call mpi_put(ldat,origin_count,mpi_real,target_rank,target_displacement,target_count,mpi_real,win0,ierr) >> ... >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpi_put works only with target_displacement=0
by Rajeev Thakur 26 Apr '14

26 Apr '14

Can you send a small test program that can be compiled and run? Rajeev On Apr 26, 2014, at 10:03 AM, Ted Sariyski <tsariysk(a)craft-tech.com> wrote: > HI, > I am trying to apply RMA so that each process evaluates its segment of data and PUTs it at process 0. Here is what I do (see attachment): > > 1. Process 0 creates a window win0 with size=GLOBAL_SIZE of the array > 2. Rest of processes create NULL() win0 with size 0. > 3. Eeach process evaluates it portion dat with size myCount > 4. All create mpi_shared_lock on 0 (dat on different process are strictly non-overlapping) > 5. Put dat on 0 with appropriate offset (myOffset) > > Code works on two cpus ONLY when target_displacement=0. If target_displacement=myOffset, I get: > > > rank 0 in job 383 craft28.craft-tech.com_57409 caused collective abort of all ranks > > exit status of rank 0: killed by signal 11 > > What do I miss here? > > Any help will be highly appreciated. Thank in advance, > --Ted > > P.S. Mixed C and fortran90 with mpich2.1.5 on Linux/x86_64 (CentOS 6.5) > > > ... > integer(mpi_address_kind) :: size=0,target_displacement=0 > if ( myid == 0 ) then > call mpi_win_create(mdat,size,mpi_realsize,info,comm,win0,ierr) > else > call mpi_win_create(NULL(),0,1,info,comm,win0,ierr) > end if > ... > target_rank=0 > call mpi_win_lock(mpi_lock_shared,target_rank,mpi_mode_nocheck,win0,ierr) > ... > origin_count=myCount > target_count=myCount > target_displacement=myOffset > ... > call mpi_put(ldat,origin_count,mpi_real,target_rank,target_displacement,target_count,mpi_real,win0,ierr) > ... > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

mpi_put works only with target_displacement=0
by Ted Sariyski 26 Apr '14

26 Apr '14

HI, I am trying to apply RMA so that each process evaluates its segment of data and PUTs it at process 0. Here is what I do (see attachment): 1. Process 0 creates a window win0 with size=GLOBAL_SIZE of the array 2. Rest of processes create NULL() win0 with size 0. 3. Eeach process evaluates it portion dat with size myCount 4. All create mpi_shared_lock on 0 (dat on different process are strictly non-overlapping) 5. Put dat on 0 with appropriate offset (myOffset) Code works on two cpus ONLY when target_displacement=0. If target_displacement=myOffset, I get: > rank 0 in job 383 craft28.craft-tech.com_57409 caused collective abort of all ranks > exit status of rank 0: killed by signal 11 What do I miss here? Any help will be highly appreciated. Thank in advance, --Ted P.S. Mixed C and fortran90 with mpich2.1.5 on Linux/x86_64 (CentOS 6.5) ... integer(mpi_address_kind) :: size=0,target_displacement=0 if ( myid == 0 ) then call mpi_win_create(mdat,size,mpi_realsize,info,comm,win0,ierr) else call mpi_win_create(NULL(),0,1,info,comm,win0,ierr) end if ... target_rank=0 call mpi_win_lock(mpi_lock_shared,target_rank,mpi_mode_nocheck,win0,ierr) ... origin_count=myCount target_count=myCount target_displacement=myOffset ... call mpi_put(ldat,origin_count,mpi_real,target_rank,target_displacement,target_count,mpi_real,win0,ierr) ...

1 0

Re: [mpich-discuss] how to declare ranks IDs on each machine?
by ehsan saei 25 Apr '14

25 Apr '14

Thanks for your guide. That solved my problem. On Fri, Apr 25, 2014 at 3:17 PM, Lu, Huiwei <huiweilu(a)mcs.anl.gov> wrote: > You can determine the rank order of your MPI processes in your > machinefile. For example, if you create a machinefile like: > > machine1 > machine1 > machine1 > machine1 > machine2 > machine1 > > Using ‘mpiexec -f machinefile -n 6 ./example/cpi’, then rank 0-3 and rank > 5 will run on machine1, and rank 4 will run on machine2. > > — > Huiwei Lu > Postdoc Appointee > Mathematics and Computer Science Division, Argonne National Laboratory > http://www.mcs.anl.gov/~huiweilu/ > > On Apr 25, 2014, at 2:42 AM, ehsan saei <ehsan.saei(a)gmail.com> wrote: > > > Dear all, > > > > I want to run my code on two machines and I want to declare rank IDs > manually. > > for example I want to rank=0 on machine 1 and rank= 4 on machine 2 and > rank=5 on machine 1 > > > > how can I do that? > > > > Thanks in advance, > > Ehsan > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0