September 2016 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPICH-3.2: SIGSEGV in MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101
by Jeff Hammond 08 Sep '16

08 Sep '16

On Wed, Sep 7, 2016 at 3:25 PM, Halim Amer <aamer(a)anl.gov> wrote: > > Regrading missing symbols from dynamic libraries when using gdb on Mac OS, I personally never managed to make it work, not just with MPICH, but also with other libraries. I consistently do static builds to avoid this problem. If you are able to see symbols from other libraries, we might want to look into this as well. For what it's worth, I've found that lldb works a lot better on Mac, which is not too surprising, since that is what Apple ships. I recall that I need to compile with "-Wl,-pie" to get the best debugging experience. The commands for LLDB are a bit different from GDB but they are intuitive ( http://lldb.llvm.org/tutorial.html) I'm not sure if it helps, but this is what I observed just now... I built my code with these options: $ make CC=/opt/mpich/dev/gcc/default/bin/mpicc CFLAGS="-g -O2 -Wall -Wl,-pie" MPICH is included as a shared library: $ otool -L win_fence.x win_fence.x: /opt/mpich/dev/gcc/default/lib/libmpi.0.dylib (compatibility version 1.0.0, current version 1.0.0) /opt/mpich/dev/gcc/default/lib/libpmpi.0.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1) /usr/local/lib/gcc/6/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) MPICH is built with GCC 6.1.0 with standard options (and debug symbols): $ /opt/mpich/dev/gcc/default/bin/mpicc -show gcc-6 -g -Wl,-flat_namespace -Wl,-commons,use_dylibs -I/opt/mpich/dev/gcc/default/include -L/opt/mpich/dev/gcc/default/lib -lmpi -lpmpi $ /opt/mpich/dev/gcc/default/bin/mpichversion MPICH Version: 3.3a1 MPICH Release date: unreleased development copy MPICH Device: ch3:nemesis MPICH configure: CC=gcc-6 CXX=g++-6 FC=gfortran-6 F77=gfortran-6 --enable-cxx --enable-fortran --enable-threads=runtime --enable-g=dbg --with-pm=hydra --prefix=/opt/mpich/dev/gcc/default --enable-wrapper-rpath --disable-static --enable-shared MPICH CC: gcc-6 -g -O2 MPICH CXX: g++-6 -g MPICH F77: gfortran-6 -g MPICH FC: gfortran-6 -g MPICH Custom Information: The following shows the debug symbols including line numbers into MPICH source: $ lldb ./win_fence.x (lldb) target create "./win_fence.x" Current executable set to './win_fence.x' (x86_64). (lldb) breakpoint set -n MPIR_Barrier_impl Breakpoint 1: where = libpmpi.0.dylib`MPIR_Barrier_impl + 1 at barrier.c:306, address = 0x000000000000af71 (lldb) run Process 87575 launched: './win_fence.x' (x86_64) SUCCESS! Process 87575 stopped * thread #1: tid = 0x2c927c, 0x0000000100144f71 libpmpi.0.dylib`MPIR_Barrier_impl(comm_ptr=0x0000000100391c60, errflag=0x00007fff5fbfef5c) + 1 at barrier.c:306, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 frame #0: 0x0000000100144f71 libpmpi.0.dylib`MPIR_Barrier_impl(comm_ptr=0x0000000100391c60, errflag=0x00007fff5fbfef5c) + 1 at barrier.c:306 303 int MPIR_Barrier_impl(MPIR_Comm *comm_ptr, MPIR_Errflag_t *errflag) 304 { 305 int mpi_errno = MPI_SUCCESS; -> 306 if (comm_ptr->coll_fns != NULL && comm_ptr->coll_fns->Barrier != NULL) 307 { 308 /* --BEGIN USEREXTENSION-- */ 309 mpi_errno = comm_ptr->coll_fns->Barrier(comm_ptr, errflag); (lldb) bt * thread #1: tid = 0x2c927c, 0x0000000100144f71 libpmpi.0.dylib`MPIR_Barrier_impl(comm_ptr=0x0000000100391c60, errflag=0x00007fff5fbfef5c) + 1 at barrier.c:306, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 * frame #0: 0x0000000100144f71 libpmpi.0.dylib`MPIR_Barrier_impl(comm_ptr=0x0000000100391c60, errflag=0x00007fff5fbfef5c) + 1 at barrier.c:306 frame #1: 0x0000000100276788 libpmpi.0.dylib`MPID_Win_free(win_ptr=0x00007fff5fbfef88) + 328 at mpidi_rma.c:181 frame #2: 0x00000001000655f1 libmpi.0.dylib`MPI_Win_free(win=0x00007fff5fbfeff0) + 513 at win_free.c:119 frame #3: 0x0000000100000e20 win_fence.x`main(argc=<unavailable>, argv=<unavailable>) + 336 at win_fence.c:64 frame #4: 0x00007fff8afd45ad libdyld.dylib`start + 1 Jeff -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH-3.2: SIGSEGV in MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101
by Halim Amer 07 Sep '16

07 Sep '16

Mark, Thanks for the toy program. I could reproduce the problem locally. We also experienced a similar bug with a PathScale compiler. We are still working on a fix, though. You can follow the progress and potentially provide more input in the ticket I created to track this issue (https://trac.mpich.org/projects/mpich/ticket/2350) Regrading missing symbols from dynamic libraries when using gdb on Mac OS, I personally never managed to make it work, not just with MPICH, but also with other libraries. I consistently do static builds to avoid this problem. If you are able to see symbols from other libraries, we might want to look into this as well. --Halim www.mcs.anl.gov/~aamer On 8/17/16 3:29 PM, Mark Davis wrote: > Hello, > > Sure. Please see the attached which is a simple single-threaded test > that simply loops through many broadcasts in a row. Note that when I > run this with NPROCS=2 it runs fine, but when I turn up the NPROCS to > 6, about 50% of the time it seems to hang for a couple seconds, and > then SEGV (backtrace below). Again, this is a transient issue and > seems to happen on every MPI program I try, assuming the NPROCS is > large enough; sometimes it runs fine with no "hanging". Note that I'm > running on an 8-core macbook pro. (When I run the same application on > my Linux cluster, I never have this problem.) > > Secondly, note that I used the recommendation of compiling with > --enable-g=most,mem and that did allow me to compile master HEAD. > However, debug symbols aren't being either generated or at least not > loaded by gdb. I did notice that no libpmpi.12.dylib.dSYM directories > (with DWARF debug format info) were created when I built master from > source, despite my --enable-g=most,mem flag. I'm pretty sure when I > built MPICH 3.2 release it did create these. Is there something else I > can do during the build to force debug symbols? > > Thank you, > Mark > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x1313 of process 62381] > MPID_Request_init (req=0x105be2098) at src/mpid/ch3/src/ch3u_request.c:56 > 56 req->dev.ext_hdr_ptr = NULL; > (gdb) bt full > #0 MPID_Request_init (req=0x105be2098) at src/mpid/ch3/src/ch3u_request.c:56 > No locals. > #1 0x000000010036aeea in ?? () from /Users/m/local/lib/libpmpi.12.dylib > No symbol table info available. > #2 0x0000000100445550 in ?? () from /Users/m/local/lib/libpmpi.12.dylib > No symbol table info available. > #3 0x0000000000000000 in ?? () > No symbol table info available. > > On Tue, Aug 16, 2016 at 7:32 PM, Halim Amer <aamer(a)anl.gov> wrote: >> Can you send us a toy program that reproduces this problem? >> >> --Halim >> www.mcs.anl.gov/~aamer >> >> >> On 8/16/16 6:04 PM, Mark Davis wrote: >>> >>> I pulled from master (at d8bb1df from yesterday) again and then >>> recompiled with the --enable-g=most,mem flag instead of just >>> --enable-g=most. >>> >>> The good news is that the --enable-g=most,mem flag compiled successfully. >>> >>> The bad news is two-fold: >>> >>> 1. I believe I'm still getting the same SEGV as I was getting before >>> related to req->dev.ext_hdr_ptr = NULL; (Although it's now >>> pointing to a different line in src/mpid/ch3/src/ch3u_request.c (line >>> 56 instead of line 101 as before). I'm not sure if the line is >>> relevant; some other things may have moved around in that file since >>> then the 3.2 release version. >>> >>> 2. I no longer have debugging symbols in my library, so my backtraces >>> are not helpful. It's possible these two issues are related? >>> >>> I did double check that I rebuilt my application from scratch so it >>> linked in the new library and that the library was indeed rebuilt (by >>> looking at file creation timestamps). >>> >>> Any ideas about these two issues? Thank you. >>> >>> Program received signal SIGSEGV, Segmentation fault. >>> [Switching to Thread 0x1313 of process 62381] >>> MPID_Request_init (req=0x105be2098) at src/mpid/ch3/src/ch3u_request.c:56 >>> 56 req->dev.ext_hdr_ptr = NULL; >>> (gdb) bt full >>> #0 MPID_Request_init (req=0x105be2098) at >>> src/mpid/ch3/src/ch3u_request.c:56 >>> No locals. >>> #1 0x000000010036aeea in ?? () from /Users/m/local/lib/libpmpi.12.dylib >>> No symbol table info available. >>> #2 0x0000000100445550 in ?? () from /Users/m/local/lib/libpmpi.12.dylib >>> No symbol table info available. >>> #3 0x0000000000000000 in ?? () >>> No symbol table info available. >>> >>> On Mon, Aug 15, 2016 at 5:32 PM, Halim Amer <aamer(a)anl.gov> wrote: >>>> >>>> Good catch! the `most` option implies `mem`, but the root configure >>>> failed >>>> to forward the `mem` option to the MPL software layer. We will push a >>>> fix, >>>> but meanwhile you can specify `--enable-g=most,mem` to get the desired >>>> behavior. >>>> >>>> --Halim >>>> www.mcs.anl.gov/~aamer >>>> >>>> >>>> On 8/12/16 9:37 PM, Mark Davis wrote: >>>>> >>>>> >>>>> I've tried both git HEAD (3ea7589) as well as the August 1 master >>>>> snapshot and am having trouble building it; I'm getting the same error >>>>> in both cases. I've configured with --enable-g=most but otherwise it's >>>>> all default. I'm running on OSX (Darwin - 15.6.0) and clang 3.8.1. >>>>> >>>>> It's erroring on compiling src/mpi/attr/lib_libpmpi_la-attr_delete.lo >>>>> due to an issue with the macro MPL_free >>>>> >>>>> Has anyone seen this before? I'm including the full error trace below: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Making all in . >>>>> CC src/mpi/attr/lib_libpmpi_la-attr_delete.lo >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:217: >>>>> ./src/include/mpir_request.h:281:13: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> MPL_free(req->u.ureq.greq_fns); >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:217: >>>>> ./src/include/mpir_request.h:281:13: warning: character constant too >>>>> long for its type >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:217: >>>>> ./src/include/mpir_request.h:281:13: error: expected ';' after >>>>> expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:50: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:217: >>>>> ./src/include/mpir_request.h:281:13: error: expected expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:51: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:225: >>>>> In file included from ./src/include/mpir_cvars.h:17: >>>>> In file included from ./src/include/mpitimpl.h:18: >>>>> ./src/include/mpir_utarray.h:238:43: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> *_dst = (*_src == NULL) ? NULL : (char*)utarray_strdup_(*_src); >>>>> ^ >>>>> ./src/include/mpir_utarray.h:56:33: note: expanded from macro >>>>> 'utarray_strdup_' >>>>> #define utarray_strdup_(x_) MPL_strdup(x_) >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:17:20: >>>>> note: expanded from macro 'MPL_strdup' >>>>> #define MPL_strdup strdup >>>>> ^ >>>>> ./src/include/mpir_mem.h:100:27: note: expanded from macro 'strdup' >>>>> #define strdup(a) 'Error use MPL_strdup' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:225: >>>>> In file included from ./src/include/mpir_cvars.h:17: >>>>> In file included from ./src/include/mpitimpl.h:18: >>>>> ./src/include/mpir_utarray.h:238:43: warning: character constant too >>>>> long for its type >>>>> ./src/include/mpir_utarray.h:56:33: note: expanded from macro >>>>> 'utarray_strdup_' >>>>> #define utarray_strdup_(x_) MPL_strdup(x_) >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:17:20: >>>>> note: expanded from macro 'MPL_strdup' >>>>> #define MPL_strdup strdup >>>>> ^ >>>>> ./src/include/mpir_mem.h:100:27: note: expanded from macro 'strdup' >>>>> #define strdup(a) 'Error use MPL_strdup' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:225: >>>>> In file included from ./src/include/mpir_cvars.h:17: >>>>> In file included from ./src/include/mpitimpl.h:18: >>>>> ./src/include/mpir_utarray.h:238:43: error: expected ';' after >>>>> expression >>>>> ./src/include/mpir_utarray.h:56:33: note: expanded from macro >>>>> 'utarray_strdup_' >>>>> #define utarray_strdup_(x_) MPL_strdup(x_) >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:17:20: >>>>> note: expanded from macro 'MPL_strdup' >>>>> #define MPL_strdup strdup >>>>> ^ >>>>> ./src/include/mpir_mem.h:100:50: note: expanded from macro 'strdup' >>>>> #define strdup(a) 'Error use MPL_strdup' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:225: >>>>> In file included from ./src/include/mpir_cvars.h:17: >>>>> In file included from ./src/include/mpitimpl.h:18: >>>>> ./src/include/mpir_utarray.h:238:43: error: expected expression >>>>> ./src/include/mpir_utarray.h:56:33: note: expanded from macro >>>>> 'utarray_strdup_' >>>>> #define utarray_strdup_(x_) MPL_strdup(x_) >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:17:20: >>>>> note: expanded from macro 'MPL_strdup' >>>>> #define MPL_strdup strdup >>>>> ^ >>>>> ./src/include/mpir_mem.h:100:51: note: expanded from macro 'strdup' >>>>> #define strdup(a) 'Error use MPL_strdup' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:225: >>>>> In file included from ./src/include/mpir_cvars.h:17: >>>>> In file included from ./src/include/mpitimpl.h:18: >>>>> ./src/include/mpir_utarray.h:242:14: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> if (*eltc) utarray_free_(*eltc); >>>>> ^ >>>>> ./src/include/mpir_utarray.h:54:33: note: expanded from macro >>>>> 'utarray_free_' >>>>> #define utarray_free_(x_) MPL_free(x_) >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:225: >>>>> In file included from ./src/include/mpir_cvars.h:17: >>>>> In file included from ./src/include/mpitimpl.h:18: >>>>> ./src/include/mpir_utarray.h:242:14: warning: character constant too >>>>> long for its type >>>>> ./src/include/mpir_utarray.h:54:33: note: expanded from macro >>>>> 'utarray_free_' >>>>> #define utarray_free_(x_) MPL_free(x_) >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:225: >>>>> In file included from ./src/include/mpir_cvars.h:17: >>>>> In file included from ./src/include/mpitimpl.h:18: >>>>> ./src/include/mpir_utarray.h:242:14: error: expected ';' after >>>>> expression >>>>> ./src/include/mpir_utarray.h:54:33: note: expanded from macro >>>>> 'utarray_free_' >>>>> #define utarray_free_(x_) MPL_free(x_) >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:50: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:225: >>>>> In file included from ./src/include/mpir_cvars.h:17: >>>>> In file included from ./src/include/mpitimpl.h:18: >>>>> ./src/include/mpir_utarray.h:242:14: error: expected expression >>>>> ./src/include/mpir_utarray.h:54:33: note: expanded from macro >>>>> 'utarray_free_' >>>>> #define utarray_free_(x_) MPL_free(x_) >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:51: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:61:29: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> nIndirect = (int *) MPL_calloc(objmem->indirect_size, >>>>> sizeof(int)); >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:27: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:61:29: warning: character constant too >>>>> long for its type >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:27: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:61:29: error: expected ';' after >>>>> expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:50: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:61:29: error: expected expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:51: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:117:9: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> MPL_free(nIndirect); >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:117:9: warning: character constant too >>>>> long for its type >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:117:9: error: expected ';' after >>>>> expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:50: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:117:9: error: expected expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:51: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:179:9: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> MPL_free((*indirect)[i]); >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:179:9: warning: character constant too >>>>> long for its type >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:179:9: error: expected ';' after >>>>> expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:50: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:179:9: error: expected expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:51: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:182:9: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> MPL_free(indirect); >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:182:9: warning: character constant too >>>>> long for its type >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:27: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:182:9: error: expected ';' after >>>>> expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:50: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:182:9: error: expected expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:110:26: >>>>> note: expanded from macro 'MPL_free' >>>>> #define MPL_free(a) free((void *)(a)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:92:51: note: expanded from macro 'free' >>>>> #define free(a) 'Error use MPL_free' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:249:30: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> *indirect = (void *) MPL_calloc(indirect_num_blocks, sizeof(void >>>>> *)); >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:27: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:249:30: warning: character constant too >>>>> long for its type >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:27: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:249:30: error: expected ';' after >>>>> expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:50: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:249:30: error: expected expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:51: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:264:26: warning: multi-character >>>>> character constant [-Wmultichar] >>>>> block_ptr = (void *) MPL_calloc(indirect_num_indices, obj_size); >>>>> ^ >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:27: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:264:26: warning: character constant too >>>>> long for its type >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:27: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:264:26: error: expected ';' after >>>>> expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:50: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> In file included from src/mpi/attr/attr_delete.c:8: >>>>> In file included from ./src/include/mpiimpl.h:228: >>>>> ./src/include/mpir_handlemem.h:264:26: error: expected expression >>>>> >>>>> >>>>> /Users/m/local/src/mpich-master-v3.2-370-g0d6412303488/src/mpl/include/mpl_trmem.h:109:26: >>>>> note: expanded from macro 'MPL_calloc' >>>>> #define MPL_calloc(a,b) calloc((size_t)(a),(size_t)(b)) >>>>> ^ >>>>> ./src/include/mpir_mem.h:91:51: note: expanded from macro 'calloc' >>>>> #define calloc(a,b) 'Error use MPL_calloc' ::: >>>>> ^ >>>>> 18 warnings and 18 errors generated. >>>>> make[2]: *** [src/mpi/attr/lib_libpmpi_la-attr_delete.lo] Error 1 >>>>> make[1]: *** [all-recursive] Error 1 >>>>> Makefile:10270: recipe for target 'all' failed >>>>> gmake: *** [all] Error 2 >>>>> >>>>> On Thu, Aug 11, 2016 at 5:21 PM, Halim Amer <aamer(a)anl.gov> wrote: >>>>>> >>>>>> >>>>>> This should be related to the alignment problem reported before >>>>>> (http://lists.mpich.org/pipermail/discuss/2016-May/004764.html) >>>>>> >>>>>> We plan to include a fix in the 3.2.x bug fix release series. >>>>>> Meanwhile, >>>>>> please try the repo version (git.mpich.org/mpich.git) which should not >>>>>> suffer from this problem. >>>>>> >>>>>> --Halim >>>>>> www.mcs.anl.gov/~aamer >>>>>> >>>>>> >>>>>> On 8/11/16 8:48 AM, Mark Davis wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hello, I'm running into a segfault when I run some relatively simple >>>>>>> MPI programs. In this particular case, I'm running a small program in >>>>>>> a loop that does MPI_Bcast, once per loop, within MPI_COMM_WORLD. The >>>>>>> buffer consists of just 7 doubles. I'm running with 6 procs on a >>>>>>> machine with 8 cores on OSX (Darwin - 15.6.0 Darwin Kernel Version >>>>>>> 15.6.0: Thu Jun 23 18:25:34 PDT 2016; >>>>>>> root:xnu-3248.60.10~1/RELEASE_X86_64 x86_64). When I run the same >>>>>>> program with a smaller number of procs, the error usually doesn't show >>>>>>> up. My compiler (both for compiling the MPICH source as well as my >>>>>>> application) is clang 3.8.1. >>>>>>> >>>>>>> When I run the same program on linux, also with MPICH-3.2 (I believe >>>>>>> the same exact source), compiled with gcc 5.3, I do not get this >>>>>>> error. This seems to be something I get only with >>>>>>> >>>>>>> gdb shows the following stack trace. I have a feeling that this has >>>>>>> something to do with my toolchain and/or libraries on my system given >>>>>>> that I never get this error on my other system (linux). However, it's >>>>>>> possible that there's an application bug as well. >>>>>>> >>>>>>> I'm running the MPICH-3.2 stable release; I haven't tried anything >>>>>>> from the repository yet. >>>>>>> >>>>>>> Does anyone have any ideas about what's going on here? I'm happy to >>>>>>> provide more details. >>>>>>> >>>>>>> Thank you, >>>>>>> Mark >>>>>>> >>>>>>> >>>>>>> Program received signal SIGSEGV, Segmentation fault. >>>>>>> MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 >>>>>>> 101 req->dev.ext_hdr_ptr = NULL; >>>>>>> (gdb) bt full >>>>>>> #0 MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 >>>>>>> No locals. >>>>>>> #1 0x00000001003ac4c9 in MPIDI_CH3U_Recvq_FDP_or_AEU >>>>>>> (match=<optimized out>, foundp=0x7fff5fbfe2bc) at >>>>>>> src/mpid/ch3/src/ch3u_recvq.c:830 >>>>>>> proc_failure_bit_masked = <error reading variable >>>>>>> proc_failure_bit_masked (Cannot access memory at address 0x1)> >>>>>>> error_bit_masked = <error reading variable error_bit_masked >>>>>>> (Cannot access memory at address 0x1)> >>>>>>> prev_rreq = <optimized out> >>>>>>> channel_matched = <optimized out> >>>>>>> rreq = <optimized out> >>>>>>> #2 0x00000001003d1ffe in MPIDI_CH3_PktHandler_EagerSend >>>>>>> (vc=<optimized out>, pkt=0x1004b3fd8 <MPIU_DBG_MaxLevel>, >>>>>>> buflen=0x7fff5fbfe440, rreqp=0x7fff5fbfe438) at >>>>>>> src/mpid/ch3/src/ch3u_eager.c:629 >>>>>>> mpi_errno = <error reading variable mpi_errno (Cannot access >>>>>>> memory at address 0x0)> >>>>>>> found = <error reading variable found (Cannot access memory at >>>>>>> address 0xefefefefefefefef)> >>>>>>> rreq = <optimized out> >>>>>>> data_len = <optimized out> >>>>>>> complete = <optimized out> >>>>>>> #3 0x00000001003f6045 in MPID_nem_handle_pkt (vc=<optimized out>, >>>>>>> buf=0x102ad07e0 "", buflen=<optimized out>) at >>>>>>> src/mpid/ch3/channels/nemesis/src/ch3_progress.c:760 >>>>>>> len = 140734799800192 >>>>>>> mpi_errno = <optimized out> >>>>>>> complete = <error reading variable complete (Cannot access >>>>>>> memory at address 0x1)> >>>>>>> rreq = <optimized out> >>>>>>> #4 0x00000001003f4e41 in MPIDI_CH3I_Progress >>>>>>> (progress_state=0x7fff5fbfe750, is_blocking=1) at >>>>>>> src/mpid/ch3/channels/nemesis/src/ch3_progress.c:570 >>>>>>> payload_len = 4299898840 >>>>>>> cell_buf = <optimized out> >>>>>>> rreq = <optimized out> >>>>>>> vc = 0x102ad07e8 >>>>>>> made_progress = <error reading variable made_progress (Cannot >>>>>>> access memory at address 0x0)> >>>>>>> mpi_errno = <optimized out> >>>>>>> #5 0x000000010035386d in MPIC_Wait (request_ptr=<optimized out>, >>>>>>> errflag=<optimized out>) at src/mpi/coll/helper_fns.c:225 >>>>>>> progress_state = {ch = {completion_count = -1409286143}} >>>>>>> mpi_errno = <error reading variable mpi_errno (Cannot access >>>>>>> memory at address 0x0)> >>>>>>> #6 0x0000000100353b10 in MPIC_Send (buf=0x100917c30, >>>>>>> count=4299945096, datatype=-1581855963, dest=<optimized out>, >>>>>>> tag=4975608, comm_ptr=0x1004b3fd8 <MPIU_DBG_MaxLevel>, >>>>>>> errflag=<optimized out>) at src/mpi/coll/helper_fns.c:302 >>>>>>> mpi_errno = <optimized out> >>>>>>> request_ptr = 0x1004bf7e0 <MPID_Request_direct+1760> >>>>>>> #7 0x0000000100246031 in MPIR_Bcast_binomial (buffer=<optimized out>, >>>>>>> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >>>>>>> comm_ptr=<optimized out>, errflag=<optimized out>) at >>>>>>> src/mpi/coll/bcast.c:280 >>>>>>> nbytes = <optimized out> >>>>>>> mpi_errno_ret = <optimized out> >>>>>>> mpi_errno = 0 >>>>>>> comm_size = <optimized out> >>>>>>> rank = 2 >>>>>>> type_size = <optimized out> >>>>>>> tmp_buf = 0x0 >>>>>>> position = <optimized out> >>>>>>> relative_rank = <optimized out> >>>>>>> mask = <optimized out> >>>>>>> src = <optimized out> >>>>>>> status = <optimized out> >>>>>>> recvd_size = <optimized out> >>>>>>> dst = <optimized out> >>>>>>> #8 0x00000001002455a3 in MPIR_SMP_Bcast (buffer=<optimized out>, >>>>>>> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >>>>>>> comm_ptr=<optimized out>, errflag=<optimized out>) at >>>>>>> src/mpi/coll/bcast.c:1087 >>>>>>> mpi_errno_ = <error reading variable mpi_errno_ (Cannot access >>>>>>> memory at address 0x0)> >>>>>>> mpi_errno = <optimized out> >>>>>>> mpi_errno_ret = <optimized out> >>>>>>> nbytes = <optimized out> >>>>>>> type_size = <optimized out> >>>>>>> status = <optimized out> >>>>>>> recvd_size = <optimized out> >>>>>>> #9 MPIR_Bcast_intra (buffer=0x100917c30, count=<optimized out>, >>>>>>> datatype=<optimized out>, root=1, comm_ptr=<optimized out>, >>>>>>> errflag=<optimized out>) at src/mpi/coll/bcast.c:1245 >>>>>>> nbytes = <optimized out> >>>>>>> mpi_errno_ret = <error reading variable mpi_errno_ret (Cannot >>>>>>> access memory at address 0x0)> >>>>>>> mpi_errno = <optimized out> >>>>>>> type_size = <optimized out> >>>>>>> comm_size = <optimized out> >>>>>>> #10 0x000000010024751e in MPIR_Bcast (buffer=<optimized out>, >>>>>>> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >>>>>>> comm_ptr=0x0, errflag=<optimized out>) at src/mpi/coll/bcast.c:1475 >>>>>>> mpi_errno = <optimized out> >>>>>>> #11 MPIR_Bcast_impl (buffer=0x1004bf7e0 <MPID_Request_direct+1760>, >>>>>>> count=-269488145, datatype=-16, root=0, comm_ptr=0x0, >>>>>>> errflag=0x1004bf100 <MPID_Request_direct>) at >>>>>>> src/mpi/coll/bcast.c:1451 >>>>>>> mpi_errno = <optimized out> >>>>>>> #12 0x00000001000f3c24 in MPI_Bcast (buffer=<optimized out>, count=7, >>>>>>> datatype=1275069445, root=1, comm=<optimized out>) at >>>>>>> src/mpi/coll/bcast.c:1585 >>>>>>> errflag = 2885681152 >>>>>>> mpi_errno = <optimized out> >>>>>>> comm_ptr = <optimized out> >>>>>>> #13 0x0000000100001df7 in run_test<int> (my_rank=2, >>>>>>> num_ranks=<optimized out>, count=<optimized out>, root_rank=1, >>>>>>> datatype=@0x7fff5fbfeaec: 1275069445, iterations=<optimized out>) at >>>>>>> bcast_test.cpp:83 >>>>>>> No locals. >>>>>>> #14 0x00000001000019cd in main (argc=<optimized out>, argv=<optimized >>>>>>> out>) at bcast_test.cpp:137 >>>>>>> root_rank = <optimized out> >>>>>>> count = <optimized out> >>>>>>> iterations = <optimized out> >>>>>>> my_rank = 4978656 >>>>>>> num_errors = <optimized out> >>>>>>> runtime_ns = <optimized out> >>>>>>> stats = {<std::__1::__basic_string_common<true>> = {<No data >>>>>>> fields>}, __r_ = >>>>>>> {<std::__1::__libcpp_compressed_pair_imp<std::__1::basic_string<char, >>>>>>> std::__1::char_traits<char>, std::__1::allocator<char> >::__rep, >>>>>>> std::__1::allocator<char>, 2>> = {<std::__1::allocator<char>> = {<No >>>>>>> data fields>}, __first_ = {{__l = {__cap_ = 17289301308300324847, >>>>>>> __size_ = 17289301308300324847, __data_ = 0xefefefefefefefef <error: >>>>>>> Cannot access memory at address 0xefefefefefefefef>} >>>>>>> _______________________________________________ >>>>>>> discuss mailing list discuss(a)mpich.org >>>>>>> To manage subscription options or unsubscribe: >>>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_F08_status
by Kenneth Raffenetti 07 Sep '16

07 Sep '16

Thanks, James. This indeed looks like a bug MPICH. I will work up a patch for inclusion in master soon. Ken On 09/07/2016 09:46 AM, James Custer wrote: > Hi all, > > It looks like MPICH defines MPI_F08_Status, but the standard says: > > "Using the mpi_f08 Fortran module, a status is declared as TYPE(MPI_Status). The C type MPI_F08_status can be used to pass a Fortran TYPE(MPI_Status) argument into a C routine. Figure 17.1 illustrates all status conversion routines. Some are only available in C, some in both C and Fortran." > > That is, MPI_F08_Status should be MPI_F08_status. Am I correct here, or am I missing something? > > Regards, > James Custer > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_Send and MPI_Bcast
by Min Si 07 Sep '16

07 Sep '16

Hi Alaa, Some information is still missing. - Number of processes per node - The word document shows you are comparing three algorithms, but you only mentioned two of them. What is the third one ? - What is the size of small/medium/large arrays in your experiments ? - I see Bcast speedup is better than Send in Figure (c), but the numbers in the table do not show that. What is the number of processes corresponding to each result in the table ? Since we do not support MPICH on Windows anymore, perhaps you would like to use MS-MPI for better performance. If you still observe the same issue, you can discuss with the Microsoft MPI team. https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_Why_ca… Min On 9/7/16 8:59 AM, alaa nashar wrote: > Hi Min > Thanks a lot for your fast response. > The file attached contains the required information. Regards > > Alaa > > > > On Wednesday, September 7, 2016 4:21 PM, msi <msi(a)anl.gov> wrote: > > > Hi Alaa, > > Generally, for small number of processes, the performance of MPI_Bcast > is usually equal to MPI_Isend/MPI_Recv, for large number of processes, > broadcast should be better. > > Could you please try the lateat version of MPICH on you system ? > MPICH2 is very old. > > Please also give us following information. > Number of processes > Number of processes per node > The execution time of each algorithm > > Min > > > > Sent via my cell phone. > > > -------- Original message -------- > From: alaa nashar <nashar_al(a)yahoo.com> > Date: 9/7/2016 7:04 AM (GMT-06:00) > To: discuss(a)mpich.org > Subject: [mpich-discuss] MPI_Send and MPI_Bcast > > Dear all > I have implemented the following two algorithms on my home Ethernet > LAN containing 2/ and three heterogeneous devices using MPICH2. > Algorithm1: > 1- Root process reads the contents of an input array. > 2- It then sends the array data to all other processes. > 3- All processes including the root process perform specific > computations on their array copies. > 4- Once the computations within each process are finished, > the generated arrays are directly written by the process that > performed the computations to separate files. > Algorithm2: > Same as Algorithm 1 except step 2 is replace by: > 2- Root process broadcast the array data to all other processes. > The implementation of both algorithms work fine and give the expected > results but Algorithm2 that uses MIPI_Bcast is slower than Algorithm1 > that uses MPI_Send > For my knowledge, MIPI_Bcast is faster than MPI_Send. > Please, would you guide me to know if there is any conflict of > misunderstand. > Thanks a lot > Alaa > > > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPI_F08_status
by James Custer 07 Sep '16

07 Sep '16

Hi all, It looks like MPICH defines MPI_F08_Status, but the standard says: "Using the mpi_f08 Fortran module, a status is declared as TYPE(MPI_Status). The C type MPI_F08_status can be used to pass a Fortran TYPE(MPI_Status) argument into a C routine. Figure 17.1 illustrates all status conversion routines. Some are only available in C, some in both C and Fortran." That is, MPI_F08_Status should be MPI_F08_status. Am I correct here, or am I missing something? Regards, James Custer _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_Send and MPI_Bcast
by msi 07 Sep '16

07 Sep '16

Hi Alaa, Generally, for small number of processes, the performance of MPI_Bcast is usually equal to MPI_Isend/MPI_Recv, for large number of processes, broadcast should be better. Could you please try the lateat version of MPICH on you system ? MPICH2 is very old. Please also give us following information.Number of processesNumber of processes per nodeThe execution time of each algorithm Min Sent via my cell phone. -------- Original message -------- From: alaa nashar <nashar_al(a)yahoo.com> Date: 9/7/2016 7:04 AM (GMT-06:00) To: discuss(a)mpich.org Subject: [mpich-discuss] MPI_Send and MPI_Bcast Dear all I have implemented the following two algorithms on my home Ethernet LAN containing 2/ and three heterogeneous devices using MPICH2. Algorithm1: 1- Root process reads the contents of an input array. 2- It then sends the array data to all other processes. 3- All processes including the root process perform specific computations on their array copies. 4- Once the computations within each process are finished, the generated arrays are directly written by the process that performed the computations to separate files. Algorithm2: Same as Algorithm 1 except step 2 is replace by: 2- Root process broadcast the array data to all other processes. The implementation of both algorithms work fine and give the expected results but Algorithm2 that uses MIPI_Bcast is slower than Algorithm1 that uses MPI_Send For my knowledge, MIPI_Bcast is faster than MPI_Send. Please, would you guide me to know if there is any conflict of misunderstand. Thanks a lot Alaa _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPI_Send and MPI_Bcast
by alaa nashar 07 Sep '16

07 Sep '16

Dearall I haveimplemented the following two algorithms on my home Ethernet LAN containing 2/and three heterogeneous devices using MPICH2. Algorithm1: 1- Rootprocess reads the contents of an input array. 2- Itthen sends the array data to all other processes. 3- Allprocesses including the root process perform specific computations on their array copies. 4- Oncethe computations within each process are finished, the generated arrays are directly written by the process that performed the computations to separate files. Algorithm2: Same asAlgorithm 1 except step 2 is replace by: 2- Root process broadcast the array data to all other processes. Theimplementation of both algorithms work fine and give the expected resultsbut Algorithm2 that uses MIPI_Bcast is slower than Algorithm1 that usesMPI_Send For myknowledge, MIPI_Bcast is faster than MPI_Send. Please,would you guide me to know if there is any conflict of misunderstand. Thanksa lot Alaa _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] What does mpich mpiexec do with signals?
by Kenneth Raffenetti 06 Sep '16

06 Sep '16

I can't speak for Windows, but on Unix platforms Hydra (mpiexec) will detect a Ctrl-C and send it along to the MPI processes. Ken On 09/06/2016 03:47 PM, JR Cary wrote: > In particular, if I send ctrl-c or ctrl-break (on Windows) to mpiexec, > does it pass it to the application? > > Can I get it to do so? > > Thx...John > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] What does mpich mpiexec do with signals?
by 任龙 06 Sep '16

06 Sep '16

You can make a test to check whether the order works. At 2016-09-07 04:47:12, "JR Cary" <cary(a)colorado.edu> wrote: >In particular, if I send ctrl-c or ctrl-break (on Windows) to mpiexec, >does it pass it to the application? > >Can I get it to do so? > >Thx...John >_______________________________________________ >discuss mailing list discuss(a)mpich.org >To manage subscription options or unsubscribe: >https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

What does mpich mpiexec do with signals?
by JR Cary 06 Sep '16

06 Sep '16

In particular, if I send ctrl-c or ctrl-break (on Windows) to mpiexec, does it pass it to the application? Can I get it to do so? Thx...John _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0