August 2016 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPICH-3.2: SIGSEGV in MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101
by Eric A. Borisch 11 Aug '16

11 Aug '16

Or install from macports; is has the patch. On Thursday, August 11, 2016, Kenneth Raffenetti <raffenet(a)mcs.anl.gov> wrote: > Or a snapshot tarball: http://www.mpich.org/static/do > wnloads/nightly/master/mpich/ > > On 08/11/2016 04:21 PM, Halim Amer wrote: > >> This should be related to the alignment problem reported before >> (http://lists.mpich.org/pipermail/discuss/2016-May/004764.html) >> >> We plan to include a fix in the 3.2.x bug fix release series. Meanwhile, >> please try the repo version (git.mpich.org/mpich.git) which should not >> suffer from this problem. >> >> --Halim >> www.mcs.anl.gov/~aamer >> >> On 8/11/16 8:48 AM, Mark Davis wrote: >> >>> Hello, I'm running into a segfault when I run some relatively simple >>> MPI programs. In this particular case, I'm running a small program in >>> a loop that does MPI_Bcast, once per loop, within MPI_COMM_WORLD. The >>> buffer consists of just 7 doubles. I'm running with 6 procs on a >>> machine with 8 cores on OSX (Darwin - 15.6.0 Darwin Kernel Version >>> 15.6.0: Thu Jun 23 18:25:34 PDT 2016; >>> root:xnu-3248.60.10~1/RELEASE_X86_64 x86_64). When I run the same >>> program with a smaller number of procs, the error usually doesn't show >>> up. My compiler (both for compiling the MPICH source as well as my >>> application) is clang 3.8.1. >>> >>> When I run the same program on linux, also with MPICH-3.2 (I believe >>> the same exact source), compiled with gcc 5.3, I do not get this >>> error. This seems to be something I get only with >>> >>> gdb shows the following stack trace. I have a feeling that this has >>> something to do with my toolchain and/or libraries on my system given >>> that I never get this error on my other system (linux). However, it's >>> possible that there's an application bug as well. >>> >>> I'm running the MPICH-3.2 stable release; I haven't tried anything >>> from the repository yet. >>> >>> Does anyone have any ideas about what's going on here? I'm happy to >>> provide more details. >>> >>> Thank you, >>> Mark >>> >>> >>> Program received signal SIGSEGV, Segmentation fault. >>> MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 >>> 101 req->dev.ext_hdr_ptr = NULL; >>> (gdb) bt full >>> #0 MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 >>> No locals. >>> #1 0x00000001003ac4c9 in MPIDI_CH3U_Recvq_FDP_or_AEU >>> (match=<optimized out>, foundp=0x7fff5fbfe2bc) at >>> src/mpid/ch3/src/ch3u_recvq.c:830 >>> proc_failure_bit_masked = <error reading variable >>> proc_failure_bit_masked (Cannot access memory at address 0x1)> >>> error_bit_masked = <error reading variable error_bit_masked >>> (Cannot access memory at address 0x1)> >>> prev_rreq = <optimized out> >>> channel_matched = <optimized out> >>> rreq = <optimized out> >>> #2 0x00000001003d1ffe in MPIDI_CH3_PktHandler_EagerSend >>> (vc=<optimized out>, pkt=0x1004b3fd8 <MPIU_DBG_MaxLevel>, >>> buflen=0x7fff5fbfe440, rreqp=0x7fff5fbfe438) at >>> src/mpid/ch3/src/ch3u_eager.c:629 >>> mpi_errno = <error reading variable mpi_errno (Cannot access >>> memory at address 0x0)> >>> found = <error reading variable found (Cannot access memory at >>> address 0xefefefefefefefef)> >>> rreq = <optimized out> >>> data_len = <optimized out> >>> complete = <optimized out> >>> #3 0x00000001003f6045 in MPID_nem_handle_pkt (vc=<optimized out>, >>> buf=0x102ad07e0 "", buflen=<optimized out>) at >>> src/mpid/ch3/channels/nemesis/src/ch3_progress.c:760 >>> len = 140734799800192 >>> mpi_errno = <optimized out> >>> complete = <error reading variable complete (Cannot access >>> memory at address 0x1)> >>> rreq = <optimized out> >>> #4 0x00000001003f4e41 in MPIDI_CH3I_Progress >>> (progress_state=0x7fff5fbfe750, is_blocking=1) at >>> src/mpid/ch3/channels/nemesis/src/ch3_progress.c:570 >>> payload_len = 4299898840 >>> cell_buf = <optimized out> >>> rreq = <optimized out> >>> vc = 0x102ad07e8 >>> made_progress = <error reading variable made_progress (Cannot >>> access memory at address 0x0)> >>> mpi_errno = <optimized out> >>> #5 0x000000010035386d in MPIC_Wait (request_ptr=<optimized out>, >>> errflag=<optimized out>) at src/mpi/coll/helper_fns.c:225 >>> progress_state = {ch = {completion_count = -1409286143}} >>> mpi_errno = <error reading variable mpi_errno (Cannot access >>> memory at address 0x0)> >>> #6 0x0000000100353b10 in MPIC_Send (buf=0x100917c30, >>> count=4299945096, datatype=-1581855963, dest=<optimized out>, >>> tag=4975608, comm_ptr=0x1004b3fd8 <MPIU_DBG_MaxLevel>, >>> errflag=<optimized out>) at src/mpi/coll/helper_fns.c:302 >>> mpi_errno = <optimized out> >>> request_ptr = 0x1004bf7e0 <MPID_Request_direct+1760> >>> #7 0x0000000100246031 in MPIR_Bcast_binomial (buffer=<optimized out>, >>> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >>> comm_ptr=<optimized out>, errflag=<optimized out>) at >>> src/mpi/coll/bcast.c:280 >>> nbytes = <optimized out> >>> mpi_errno_ret = <optimized out> >>> mpi_errno = 0 >>> comm_size = <optimized out> >>> rank = 2 >>> type_size = <optimized out> >>> tmp_buf = 0x0 >>> position = <optimized out> >>> relative_rank = <optimized out> >>> mask = <optimized out> >>> src = <optimized out> >>> status = <optimized out> >>> recvd_size = <optimized out> >>> dst = <optimized out> >>> #8 0x00000001002455a3 in MPIR_SMP_Bcast (buffer=<optimized out>, >>> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >>> comm_ptr=<optimized out>, errflag=<optimized out>) at >>> src/mpi/coll/bcast.c:1087 >>> mpi_errno_ = <error reading variable mpi_errno_ (Cannot access >>> memory at address 0x0)> >>> mpi_errno = <optimized out> >>> mpi_errno_ret = <optimized out> >>> nbytes = <optimized out> >>> type_size = <optimized out> >>> status = <optimized out> >>> recvd_size = <optimized out> >>> #9 MPIR_Bcast_intra (buffer=0x100917c30, count=<optimized out>, >>> datatype=<optimized out>, root=1, comm_ptr=<optimized out>, >>> errflag=<optimized out>) at src/mpi/coll/bcast.c:1245 >>> nbytes = <optimized out> >>> mpi_errno_ret = <error reading variable mpi_errno_ret (Cannot >>> access memory at address 0x0)> >>> mpi_errno = <optimized out> >>> type_size = <optimized out> >>> comm_size = <optimized out> >>> #10 0x000000010024751e in MPIR_Bcast (buffer=<optimized out>, >>> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >>> comm_ptr=0x0, errflag=<optimized out>) at src/mpi/coll/bcast.c:1475 >>> mpi_errno = <optimized out> >>> #11 MPIR_Bcast_impl (buffer=0x1004bf7e0 <MPID_Request_direct+1760>, >>> count=-269488145, datatype=-16, root=0, comm_ptr=0x0, >>> errflag=0x1004bf100 <MPID_Request_direct>) at >>> src/mpi/coll/bcast.c:1451 >>> mpi_errno = <optimized out> >>> #12 0x00000001000f3c24 in MPI_Bcast (buffer=<optimized out>, count=7, >>> datatype=1275069445, root=1, comm=<optimized out>) at >>> src/mpi/coll/bcast.c:1585 >>> errflag = 2885681152 >>> mpi_errno = <optimized out> >>> comm_ptr = <optimized out> >>> #13 0x0000000100001df7 in run_test<int> (my_rank=2, >>> num_ranks=<optimized out>, count=<optimized out>, root_rank=1, >>> datatype=@0x7fff5fbfeaec: 1275069445, iterations=<optimized out>) at >>> bcast_test.cpp:83 >>> No locals. >>> #14 0x00000001000019cd in main (argc=<optimized out>, argv=<optimized >>> out>) at bcast_test.cpp:137 >>> root_rank = <optimized out> >>> count = <optimized out> >>> iterations = <optimized out> >>> my_rank = 4978656 >>> num_errors = <optimized out> >>> runtime_ns = <optimized out> >>> stats = {<std::__1::__basic_string_common<true>> = {<No data >>> fields>}, __r_ = >>> {<std::__1::__libcpp_compressed_pair_imp<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >::__rep, >>> std::__1::allocator<char>, 2>> = {<std::__1::allocator<char>> = {<No >>> data fields>}, __first_ = {{__l = {__cap_ = 17289301308300324847, >>> __size_ = 17289301308300324847, __data_ = 0xefefefefefefefef <error: >>> Cannot access memory at address 0xefefefefefefefef>} >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH-3.2: SIGSEGV in MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101
by Kenneth Raffenetti 11 Aug '16

11 Aug '16

Or a snapshot tarball: http://www.mpich.org/static/downloads/nightly/master/mpich/ On 08/11/2016 04:21 PM, Halim Amer wrote: > This should be related to the alignment problem reported before > (http://lists.mpich.org/pipermail/discuss/2016-May/004764.html) > > We plan to include a fix in the 3.2.x bug fix release series. Meanwhile, > please try the repo version (git.mpich.org/mpich.git) which should not > suffer from this problem. > > --Halim > www.mcs.anl.gov/~aamer > > On 8/11/16 8:48 AM, Mark Davis wrote: >> Hello, I'm running into a segfault when I run some relatively simple >> MPI programs. In this particular case, I'm running a small program in >> a loop that does MPI_Bcast, once per loop, within MPI_COMM_WORLD. The >> buffer consists of just 7 doubles. I'm running with 6 procs on a >> machine with 8 cores on OSX (Darwin - 15.6.0 Darwin Kernel Version >> 15.6.0: Thu Jun 23 18:25:34 PDT 2016; >> root:xnu-3248.60.10~1/RELEASE_X86_64 x86_64). When I run the same >> program with a smaller number of procs, the error usually doesn't show >> up. My compiler (both for compiling the MPICH source as well as my >> application) is clang 3.8.1. >> >> When I run the same program on linux, also with MPICH-3.2 (I believe >> the same exact source), compiled with gcc 5.3, I do not get this >> error. This seems to be something I get only with >> >> gdb shows the following stack trace. I have a feeling that this has >> something to do with my toolchain and/or libraries on my system given >> that I never get this error on my other system (linux). However, it's >> possible that there's an application bug as well. >> >> I'm running the MPICH-3.2 stable release; I haven't tried anything >> from the repository yet. >> >> Does anyone have any ideas about what's going on here? I'm happy to >> provide more details. >> >> Thank you, >> Mark >> >> >> Program received signal SIGSEGV, Segmentation fault. >> MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 >> 101 req->dev.ext_hdr_ptr = NULL; >> (gdb) bt full >> #0 MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 >> No locals. >> #1 0x00000001003ac4c9 in MPIDI_CH3U_Recvq_FDP_or_AEU >> (match=<optimized out>, foundp=0x7fff5fbfe2bc) at >> src/mpid/ch3/src/ch3u_recvq.c:830 >> proc_failure_bit_masked = <error reading variable >> proc_failure_bit_masked (Cannot access memory at address 0x1)> >> error_bit_masked = <error reading variable error_bit_masked >> (Cannot access memory at address 0x1)> >> prev_rreq = <optimized out> >> channel_matched = <optimized out> >> rreq = <optimized out> >> #2 0x00000001003d1ffe in MPIDI_CH3_PktHandler_EagerSend >> (vc=<optimized out>, pkt=0x1004b3fd8 <MPIU_DBG_MaxLevel>, >> buflen=0x7fff5fbfe440, rreqp=0x7fff5fbfe438) at >> src/mpid/ch3/src/ch3u_eager.c:629 >> mpi_errno = <error reading variable mpi_errno (Cannot access >> memory at address 0x0)> >> found = <error reading variable found (Cannot access memory at >> address 0xefefefefefefefef)> >> rreq = <optimized out> >> data_len = <optimized out> >> complete = <optimized out> >> #3 0x00000001003f6045 in MPID_nem_handle_pkt (vc=<optimized out>, >> buf=0x102ad07e0 "", buflen=<optimized out>) at >> src/mpid/ch3/channels/nemesis/src/ch3_progress.c:760 >> len = 140734799800192 >> mpi_errno = <optimized out> >> complete = <error reading variable complete (Cannot access >> memory at address 0x1)> >> rreq = <optimized out> >> #4 0x00000001003f4e41 in MPIDI_CH3I_Progress >> (progress_state=0x7fff5fbfe750, is_blocking=1) at >> src/mpid/ch3/channels/nemesis/src/ch3_progress.c:570 >> payload_len = 4299898840 >> cell_buf = <optimized out> >> rreq = <optimized out> >> vc = 0x102ad07e8 >> made_progress = <error reading variable made_progress (Cannot >> access memory at address 0x0)> >> mpi_errno = <optimized out> >> #5 0x000000010035386d in MPIC_Wait (request_ptr=<optimized out>, >> errflag=<optimized out>) at src/mpi/coll/helper_fns.c:225 >> progress_state = {ch = {completion_count = -1409286143}} >> mpi_errno = <error reading variable mpi_errno (Cannot access >> memory at address 0x0)> >> #6 0x0000000100353b10 in MPIC_Send (buf=0x100917c30, >> count=4299945096, datatype=-1581855963, dest=<optimized out>, >> tag=4975608, comm_ptr=0x1004b3fd8 <MPIU_DBG_MaxLevel>, >> errflag=<optimized out>) at src/mpi/coll/helper_fns.c:302 >> mpi_errno = <optimized out> >> request_ptr = 0x1004bf7e0 <MPID_Request_direct+1760> >> #7 0x0000000100246031 in MPIR_Bcast_binomial (buffer=<optimized out>, >> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >> comm_ptr=<optimized out>, errflag=<optimized out>) at >> src/mpi/coll/bcast.c:280 >> nbytes = <optimized out> >> mpi_errno_ret = <optimized out> >> mpi_errno = 0 >> comm_size = <optimized out> >> rank = 2 >> type_size = <optimized out> >> tmp_buf = 0x0 >> position = <optimized out> >> relative_rank = <optimized out> >> mask = <optimized out> >> src = <optimized out> >> status = <optimized out> >> recvd_size = <optimized out> >> dst = <optimized out> >> #8 0x00000001002455a3 in MPIR_SMP_Bcast (buffer=<optimized out>, >> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >> comm_ptr=<optimized out>, errflag=<optimized out>) at >> src/mpi/coll/bcast.c:1087 >> mpi_errno_ = <error reading variable mpi_errno_ (Cannot access >> memory at address 0x0)> >> mpi_errno = <optimized out> >> mpi_errno_ret = <optimized out> >> nbytes = <optimized out> >> type_size = <optimized out> >> status = <optimized out> >> recvd_size = <optimized out> >> #9 MPIR_Bcast_intra (buffer=0x100917c30, count=<optimized out>, >> datatype=<optimized out>, root=1, comm_ptr=<optimized out>, >> errflag=<optimized out>) at src/mpi/coll/bcast.c:1245 >> nbytes = <optimized out> >> mpi_errno_ret = <error reading variable mpi_errno_ret (Cannot >> access memory at address 0x0)> >> mpi_errno = <optimized out> >> type_size = <optimized out> >> comm_size = <optimized out> >> #10 0x000000010024751e in MPIR_Bcast (buffer=<optimized out>, >> count=<optimized out>, datatype=<optimized out>, root=<optimized out>, >> comm_ptr=0x0, errflag=<optimized out>) at src/mpi/coll/bcast.c:1475 >> mpi_errno = <optimized out> >> #11 MPIR_Bcast_impl (buffer=0x1004bf7e0 <MPID_Request_direct+1760>, >> count=-269488145, datatype=-16, root=0, comm_ptr=0x0, >> errflag=0x1004bf100 <MPID_Request_direct>) at >> src/mpi/coll/bcast.c:1451 >> mpi_errno = <optimized out> >> #12 0x00000001000f3c24 in MPI_Bcast (buffer=<optimized out>, count=7, >> datatype=1275069445, root=1, comm=<optimized out>) at >> src/mpi/coll/bcast.c:1585 >> errflag = 2885681152 >> mpi_errno = <optimized out> >> comm_ptr = <optimized out> >> #13 0x0000000100001df7 in run_test<int> (my_rank=2, >> num_ranks=<optimized out>, count=<optimized out>, root_rank=1, >> datatype=@0x7fff5fbfeaec: 1275069445, iterations=<optimized out>) at >> bcast_test.cpp:83 >> No locals. >> #14 0x00000001000019cd in main (argc=<optimized out>, argv=<optimized >> out>) at bcast_test.cpp:137 >> root_rank = <optimized out> >> count = <optimized out> >> iterations = <optimized out> >> my_rank = 4978656 >> num_errors = <optimized out> >> runtime_ns = <optimized out> >> stats = {<std::__1::__basic_string_common<true>> = {<No data >> fields>}, __r_ = >> {<std::__1::__libcpp_compressed_pair_imp<std::__1::basic_string<char, >> std::__1::char_traits<char>, std::__1::allocator<char> >::__rep, >> std::__1::allocator<char>, 2>> = {<std::__1::allocator<char>> = {<No >> data fields>}, __first_ = {{__l = {__cap_ = 17289301308300324847, >> __size_ = 17289301308300324847, __data_ = 0xefefefefefefefef <error: >> Cannot access memory at address 0xefefefefefefefef>} >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH-3.2: SIGSEGV in MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101
by Halim Amer 11 Aug '16

11 Aug '16

This should be related to the alignment problem reported before (http://lists.mpich.org/pipermail/discuss/2016-May/004764.html) We plan to include a fix in the 3.2.x bug fix release series. Meanwhile, please try the repo version (git.mpich.org/mpich.git) which should not suffer from this problem. --Halim www.mcs.anl.gov/~aamer On 8/11/16 8:48 AM, Mark Davis wrote: > Hello, I'm running into a segfault when I run some relatively simple > MPI programs. In this particular case, I'm running a small program in > a loop that does MPI_Bcast, once per loop, within MPI_COMM_WORLD. The > buffer consists of just 7 doubles. I'm running with 6 procs on a > machine with 8 cores on OSX (Darwin - 15.6.0 Darwin Kernel Version > 15.6.0: Thu Jun 23 18:25:34 PDT 2016; > root:xnu-3248.60.10~1/RELEASE_X86_64 x86_64). When I run the same > program with a smaller number of procs, the error usually doesn't show > up. My compiler (both for compiling the MPICH source as well as my > application) is clang 3.8.1. > > When I run the same program on linux, also with MPICH-3.2 (I believe > the same exact source), compiled with gcc 5.3, I do not get this > error. This seems to be something I get only with > > gdb shows the following stack trace. I have a feeling that this has > something to do with my toolchain and/or libraries on my system given > that I never get this error on my other system (linux). However, it's > possible that there's an application bug as well. > > I'm running the MPICH-3.2 stable release; I haven't tried anything > from the repository yet. > > Does anyone have any ideas about what's going on here? I'm happy to > provide more details. > > Thank you, > Mark > > > Program received signal SIGSEGV, Segmentation fault. > MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 > 101 req->dev.ext_hdr_ptr = NULL; > (gdb) bt full > #0 MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 > No locals. > #1 0x00000001003ac4c9 in MPIDI_CH3U_Recvq_FDP_or_AEU > (match=<optimized out>, foundp=0x7fff5fbfe2bc) at > src/mpid/ch3/src/ch3u_recvq.c:830 > proc_failure_bit_masked = <error reading variable > proc_failure_bit_masked (Cannot access memory at address 0x1)> > error_bit_masked = <error reading variable error_bit_masked > (Cannot access memory at address 0x1)> > prev_rreq = <optimized out> > channel_matched = <optimized out> > rreq = <optimized out> > #2 0x00000001003d1ffe in MPIDI_CH3_PktHandler_EagerSend > (vc=<optimized out>, pkt=0x1004b3fd8 <MPIU_DBG_MaxLevel>, > buflen=0x7fff5fbfe440, rreqp=0x7fff5fbfe438) at > src/mpid/ch3/src/ch3u_eager.c:629 > mpi_errno = <error reading variable mpi_errno (Cannot access > memory at address 0x0)> > found = <error reading variable found (Cannot access memory at > address 0xefefefefefefefef)> > rreq = <optimized out> > data_len = <optimized out> > complete = <optimized out> > #3 0x00000001003f6045 in MPID_nem_handle_pkt (vc=<optimized out>, > buf=0x102ad07e0 "", buflen=<optimized out>) at > src/mpid/ch3/channels/nemesis/src/ch3_progress.c:760 > len = 140734799800192 > mpi_errno = <optimized out> > complete = <error reading variable complete (Cannot access > memory at address 0x1)> > rreq = <optimized out> > #4 0x00000001003f4e41 in MPIDI_CH3I_Progress > (progress_state=0x7fff5fbfe750, is_blocking=1) at > src/mpid/ch3/channels/nemesis/src/ch3_progress.c:570 > payload_len = 4299898840 > cell_buf = <optimized out> > rreq = <optimized out> > vc = 0x102ad07e8 > made_progress = <error reading variable made_progress (Cannot > access memory at address 0x0)> > mpi_errno = <optimized out> > #5 0x000000010035386d in MPIC_Wait (request_ptr=<optimized out>, > errflag=<optimized out>) at src/mpi/coll/helper_fns.c:225 > progress_state = {ch = {completion_count = -1409286143}} > mpi_errno = <error reading variable mpi_errno (Cannot access > memory at address 0x0)> > #6 0x0000000100353b10 in MPIC_Send (buf=0x100917c30, > count=4299945096, datatype=-1581855963, dest=<optimized out>, > tag=4975608, comm_ptr=0x1004b3fd8 <MPIU_DBG_MaxLevel>, > errflag=<optimized out>) at src/mpi/coll/helper_fns.c:302 > mpi_errno = <optimized out> > request_ptr = 0x1004bf7e0 <MPID_Request_direct+1760> > #7 0x0000000100246031 in MPIR_Bcast_binomial (buffer=<optimized out>, > count=<optimized out>, datatype=<optimized out>, root=<optimized out>, > comm_ptr=<optimized out>, errflag=<optimized out>) at > src/mpi/coll/bcast.c:280 > nbytes = <optimized out> > mpi_errno_ret = <optimized out> > mpi_errno = 0 > comm_size = <optimized out> > rank = 2 > type_size = <optimized out> > tmp_buf = 0x0 > position = <optimized out> > relative_rank = <optimized out> > mask = <optimized out> > src = <optimized out> > status = <optimized out> > recvd_size = <optimized out> > dst = <optimized out> > #8 0x00000001002455a3 in MPIR_SMP_Bcast (buffer=<optimized out>, > count=<optimized out>, datatype=<optimized out>, root=<optimized out>, > comm_ptr=<optimized out>, errflag=<optimized out>) at > src/mpi/coll/bcast.c:1087 > mpi_errno_ = <error reading variable mpi_errno_ (Cannot access > memory at address 0x0)> > mpi_errno = <optimized out> > mpi_errno_ret = <optimized out> > nbytes = <optimized out> > type_size = <optimized out> > status = <optimized out> > recvd_size = <optimized out> > #9 MPIR_Bcast_intra (buffer=0x100917c30, count=<optimized out>, > datatype=<optimized out>, root=1, comm_ptr=<optimized out>, > errflag=<optimized out>) at src/mpi/coll/bcast.c:1245 > nbytes = <optimized out> > mpi_errno_ret = <error reading variable mpi_errno_ret (Cannot > access memory at address 0x0)> > mpi_errno = <optimized out> > type_size = <optimized out> > comm_size = <optimized out> > #10 0x000000010024751e in MPIR_Bcast (buffer=<optimized out>, > count=<optimized out>, datatype=<optimized out>, root=<optimized out>, > comm_ptr=0x0, errflag=<optimized out>) at src/mpi/coll/bcast.c:1475 > mpi_errno = <optimized out> > #11 MPIR_Bcast_impl (buffer=0x1004bf7e0 <MPID_Request_direct+1760>, > count=-269488145, datatype=-16, root=0, comm_ptr=0x0, > errflag=0x1004bf100 <MPID_Request_direct>) at > src/mpi/coll/bcast.c:1451 > mpi_errno = <optimized out> > #12 0x00000001000f3c24 in MPI_Bcast (buffer=<optimized out>, count=7, > datatype=1275069445, root=1, comm=<optimized out>) at > src/mpi/coll/bcast.c:1585 > errflag = 2885681152 > mpi_errno = <optimized out> > comm_ptr = <optimized out> > #13 0x0000000100001df7 in run_test<int> (my_rank=2, > num_ranks=<optimized out>, count=<optimized out>, root_rank=1, > datatype=@0x7fff5fbfeaec: 1275069445, iterations=<optimized out>) at > bcast_test.cpp:83 > No locals. > #14 0x00000001000019cd in main (argc=<optimized out>, argv=<optimized > out>) at bcast_test.cpp:137 > root_rank = <optimized out> > count = <optimized out> > iterations = <optimized out> > my_rank = 4978656 > num_errors = <optimized out> > runtime_ns = <optimized out> > stats = {<std::__1::__basic_string_common<true>> = {<No data > fields>}, __r_ = > {<std::__1::__libcpp_compressed_pair_imp<std::__1::basic_string<char, > std::__1::char_traits<char>, std::__1::allocator<char> >::__rep, > std::__1::allocator<char>, 2>> = {<std::__1::allocator<char>> = {<No > data fields>}, __first_ = {{__l = {__cap_ = 17289301308300324847, > __size_ = 17289301308300324847, __data_ = 0xefefefefefefefef <error: > Cannot access memory at address 0xefefefefefefefef>} > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Per Thread CS
by Halim Amer 11 Aug '16

11 Aug '16

--enable-thread-cs=per-object is only experimental (currently not functional) and users are not supposed to use it. If you have access to a Blue Gene system, you can try MPICH 3.1 with per-object locking, which, as far as I remember, is functional. --Halim www.mcs.anl.gov/~aamer On 8/9/16 4:51 PM, mdosanjh(a)cs.unm.edu wrote: > Hello, > > I'm trying to do a research improvements to fine grained threading models > in MPI. I was trying to compile MPICH 3.2 with --enable-threads=multiple > --enable-thread-cs=per-object and got the following error: > > src/mpid/ch3/src/ch3u_handle_connection.c:225: error: 'MPIDI_VC_t' has no > member named 'pobj_mutex' > > Is there a way to get this to compile or another version I can try? > > Thanks, > Matthew > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPICH-3.2: SIGSEGV in MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101
by Mark Davis 11 Aug '16

11 Aug '16

Hello, I'm running into a segfault when I run some relatively simple MPI programs. In this particular case, I'm running a small program in a loop that does MPI_Bcast, once per loop, within MPI_COMM_WORLD. The buffer consists of just 7 doubles. I'm running with 6 procs on a machine with 8 cores on OSX (Darwin - 15.6.0 Darwin Kernel Version 15.6.0: Thu Jun 23 18:25:34 PDT 2016; root:xnu-3248.60.10~1/RELEASE_X86_64 x86_64). When I run the same program with a smaller number of procs, the error usually doesn't show up. My compiler (both for compiling the MPICH source as well as my application) is clang 3.8.1. When I run the same program on linux, also with MPICH-3.2 (I believe the same exact source), compiled with gcc 5.3, I do not get this error. This seems to be something I get only with gdb shows the following stack trace. I have a feeling that this has something to do with my toolchain and/or libraries on my system given that I never get this error on my other system (linux). However, it's possible that there's an application bug as well. I'm running the MPICH-3.2 stable release; I haven't tried anything from the repository yet. Does anyone have any ideas about what's going on here? I'm happy to provide more details. Thank you, Mark Program received signal SIGSEGV, Segmentation fault. MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 101 req->dev.ext_hdr_ptr = NULL; (gdb) bt full #0 MPID_Request_create () at src/mpid/ch3/src/ch3u_request.c:101 No locals. #1 0x00000001003ac4c9 in MPIDI_CH3U_Recvq_FDP_or_AEU (match=<optimized out>, foundp=0x7fff5fbfe2bc) at src/mpid/ch3/src/ch3u_recvq.c:830 proc_failure_bit_masked = <error reading variable proc_failure_bit_masked (Cannot access memory at address 0x1)> error_bit_masked = <error reading variable error_bit_masked (Cannot access memory at address 0x1)> prev_rreq = <optimized out> channel_matched = <optimized out> rreq = <optimized out> #2 0x00000001003d1ffe in MPIDI_CH3_PktHandler_EagerSend (vc=<optimized out>, pkt=0x1004b3fd8 <MPIU_DBG_MaxLevel>, buflen=0x7fff5fbfe440, rreqp=0x7fff5fbfe438) at src/mpid/ch3/src/ch3u_eager.c:629 mpi_errno = <error reading variable mpi_errno (Cannot access memory at address 0x0)> found = <error reading variable found (Cannot access memory at address 0xefefefefefefefef)> rreq = <optimized out> data_len = <optimized out> complete = <optimized out> #3 0x00000001003f6045 in MPID_nem_handle_pkt (vc=<optimized out>, buf=0x102ad07e0 "", buflen=<optimized out>) at src/mpid/ch3/channels/nemesis/src/ch3_progress.c:760 len = 140734799800192 mpi_errno = <optimized out> complete = <error reading variable complete (Cannot access memory at address 0x1)> rreq = <optimized out> #4 0x00000001003f4e41 in MPIDI_CH3I_Progress (progress_state=0x7fff5fbfe750, is_blocking=1) at src/mpid/ch3/channels/nemesis/src/ch3_progress.c:570 payload_len = 4299898840 cell_buf = <optimized out> rreq = <optimized out> vc = 0x102ad07e8 made_progress = <error reading variable made_progress (Cannot access memory at address 0x0)> mpi_errno = <optimized out> #5 0x000000010035386d in MPIC_Wait (request_ptr=<optimized out>, errflag=<optimized out>) at src/mpi/coll/helper_fns.c:225 progress_state = {ch = {completion_count = -1409286143}} mpi_errno = <error reading variable mpi_errno (Cannot access memory at address 0x0)> #6 0x0000000100353b10 in MPIC_Send (buf=0x100917c30, count=4299945096, datatype=-1581855963, dest=<optimized out>, tag=4975608, comm_ptr=0x1004b3fd8 <MPIU_DBG_MaxLevel>, errflag=<optimized out>) at src/mpi/coll/helper_fns.c:302 mpi_errno = <optimized out> request_ptr = 0x1004bf7e0 <MPID_Request_direct+1760> #7 0x0000000100246031 in MPIR_Bcast_binomial (buffer=<optimized out>, count=<optimized out>, datatype=<optimized out>, root=<optimized out>, comm_ptr=<optimized out>, errflag=<optimized out>) at src/mpi/coll/bcast.c:280 nbytes = <optimized out> mpi_errno_ret = <optimized out> mpi_errno = 0 comm_size = <optimized out> rank = 2 type_size = <optimized out> tmp_buf = 0x0 position = <optimized out> relative_rank = <optimized out> mask = <optimized out> src = <optimized out> status = <optimized out> recvd_size = <optimized out> dst = <optimized out> #8 0x00000001002455a3 in MPIR_SMP_Bcast (buffer=<optimized out>, count=<optimized out>, datatype=<optimized out>, root=<optimized out>, comm_ptr=<optimized out>, errflag=<optimized out>) at src/mpi/coll/bcast.c:1087 mpi_errno_ = <error reading variable mpi_errno_ (Cannot access memory at address 0x0)> mpi_errno = <optimized out> mpi_errno_ret = <optimized out> nbytes = <optimized out> type_size = <optimized out> status = <optimized out> recvd_size = <optimized out> #9 MPIR_Bcast_intra (buffer=0x100917c30, count=<optimized out>, datatype=<optimized out>, root=1, comm_ptr=<optimized out>, errflag=<optimized out>) at src/mpi/coll/bcast.c:1245 nbytes = <optimized out> mpi_errno_ret = <error reading variable mpi_errno_ret (Cannot access memory at address 0x0)> mpi_errno = <optimized out> type_size = <optimized out> comm_size = <optimized out> #10 0x000000010024751e in MPIR_Bcast (buffer=<optimized out>, count=<optimized out>, datatype=<optimized out>, root=<optimized out>, comm_ptr=0x0, errflag=<optimized out>) at src/mpi/coll/bcast.c:1475 mpi_errno = <optimized out> #11 MPIR_Bcast_impl (buffer=0x1004bf7e0 <MPID_Request_direct+1760>, count=-269488145, datatype=-16, root=0, comm_ptr=0x0, errflag=0x1004bf100 <MPID_Request_direct>) at src/mpi/coll/bcast.c:1451 mpi_errno = <optimized out> #12 0x00000001000f3c24 in MPI_Bcast (buffer=<optimized out>, count=7, datatype=1275069445, root=1, comm=<optimized out>) at src/mpi/coll/bcast.c:1585 errflag = 2885681152 mpi_errno = <optimized out> comm_ptr = <optimized out> #13 0x0000000100001df7 in run_test<int> (my_rank=2, num_ranks=<optimized out>, count=<optimized out>, root_rank=1, datatype=@0x7fff5fbfeaec: 1275069445, iterations=<optimized out>) at bcast_test.cpp:83 No locals. #14 0x00000001000019cd in main (argc=<optimized out>, argv=<optimized out>) at bcast_test.cpp:137 root_rank = <optimized out> count = <optimized out> iterations = <optimized out> my_rank = 4978656 num_errors = <optimized out> runtime_ns = <optimized out> stats = {<std::__1::__basic_string_common<true>> = {<No data fields>}, __r_ = {<std::__1::__libcpp_compressed_pair_imp<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__rep, std::__1::allocator<char>, 2>> = {<std::__1::allocator<char>> = {<No data fields>}, __first_ = {{__l = {__cap_ = 17289301308300324847, __size_ = 17289301308300324847, __data_ = 0xefefefefefefefef <error: Cannot access memory at address 0xefefefefefefefef>} _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPIDI_CH3U_Receive_data_unexpected: memory leak ?
by VandeVondele Joost 11 Aug '16

11 Aug '16

Hi, using mpich 3.2 with our code (CP2K) we see the following memory leak with lsan: Direct leak of 12544 byte(s) in 1 object(s) allocated from: #0 0x7f64851477e6 in __interceptor_malloc /data/vjoost/toolchain-r17215/build/gcc-5.4.0/libsanitizer/lsan/lsan_interceptors.cc:51 #1 0x7f6484dd556f in MPIDI_CH3U_Receive_data_unexpected src/mpid/ch3/src/ch3u_handle_recv_pkt.c:251 #2 0x7f6484e0dbb3 in MPIDI_CH3_PktHandler_EagerSend src/mpid/ch3/src/ch3u_eager.c:661 #3 0x7f6484e218bf in MPIDI_CH3I_Progress src/mpid/ch3/channels/nemesis/src/ch3_progress.c:541 #4 0x7f6484d76dde in PMPI_Recv src/mpi/pt2pt/recv.c:157 #5 0x264730a in BI_Srecv /data/vjoost/toolchain-r17215/build/scalapack-2.0.2/BLACS/SRC/BI_Srecv.c:8 #6 0x264b6c2 in Cdgerv2d /data/vjoost/toolchain-r17215/build/scalapack-2.0.2/BLACS/SRC/dgerv2d_.c:79 #7 0x26cfd4d in PB_CpaxpbyDN /data/vjoost/toolchain-r17215/build/scalapack-2.0.2/PBLAS/SRC/PTOOLS/PB_CpaxpbyDN.c:661 #8 0x2651ba2 in PB_Cpaxpby /data/vjoost/toolchain-r17215/build/scalapack-2.0.2/PBLAS/SRC/PTOOLS/PB_Cpaxpby.c:754 #9 0x2663060 in PB_Cptran /data/vjoost/toolchain-r17215/build/scalapack-2.0.2/PBLAS/SRC/PTOOLS/PB_Cptran.c:414 #10 0x2617973 in pdtran_ /data/vjoost/toolchain-r17215/build/scalapack-2.0.2/PBLAS/SRC/pdtran_.c:240 #11 0x1de4388 in __cp_fm_basic_linalg_MOD_cp_fm_transpose /data/vjoost/clean/cp2k/cp2k/src/fm/cp_fm_basic_linalg.F:925 [...] In the code at the line indicate there is indeed a FIXME, but I'm wondering if this location could indeed leak memory unbounded, which would be problematic. Best regards, Joost VandeVondele ________________________________ Prof. Dr. Joost VandeVondele Nanoscale Simulations ETH Zurich HIT G 41.4 Wolfgang-Pauli-Strasse 27 8093 Zurich, Switzerland Joost.VandeVondele(a)mat.ethz.ch http://www.nanosim.mat.ethz.ch/ Tel.: +41 (0)44 6338451 ________________________________ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] undefined symbol: MPI_UNWEIGHTED
by Kenneth Raffenetti 10 Aug '16

10 Aug '16

Great! Yes, it looks like your binary was picking up a libmpi.so.12 from the tecplot directory that may not have been compatible with the Fortran library. Ken On 08/10/2016 01:13 PM, Robert Baurle wrote: > I have figured out the problem. There was some sort of conflict with > other libraries that were in my library path. In particular, I had > Tecplot360EX in my LD_LIBRARY_PATH, and this package includes some > libraries that take advantage of MPI-IO. Here is the result of ldd when > this package was in my path: > > hyp22 rbaurle 2007> ldd hello.out > linux-vdso.so.1 => (0x00007ffe001df000) > libmpifort.so.12 => /opt/mpich/lib/libmpifort.so.12 > (0x00002b0e09bf5000) > libmpi.so.12 => /opt/tecplotex/bin/libmpi.so.12 (0x00002b0e09e2d000) > libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00002b0e0a126000) > libm.so.6 => /lib64/libm.so.6 (0x00000038bcc00000) > libgcc_s.so.1 => /opt/tecplotex/bin/libgcc_s.so.1 (0x00002b0e0a419000) > libc.so.6 => /lib64/libc.so.6 (0x00000038bc800000) > librt.so.1 => /lib64/librt.so.1 (0x00000038bd800000) > libpthread.so.0 => /lib64/libpthread.so.0 (0x00000038bd400000) > libopen-rte.so.12 => /opt/tecplotex/bin/libopen-rte.so.12 > (0x00002b0e0a630000) > libopen-pal.so.13 => /opt/tecplotex/bin/libopen-pal.so.13 > (0x00002b0e0a8ab000) > libpciaccess.so.0 => /usr/lib64/libpciaccess.so.0 (0x00000038bec00000) > libdl.so.2 => /lib64/libdl.so.2 (0x00000038bd000000) > libutil.so.1 => /lib64/libutil.so.1 (0x00000038ce800000) > /lib64/ld-linux-x86-64.so.2 (0x00000038bc400000) > > When I removed this package from my LD_LIBRARY_PATH, the hello > executable runs just fine and this is the result of of ldd: > > hyp22 rbaurle 2020> ldd hello.out > linux-vdso.so.1 => (0x00007ffda41fd000) > libmpifort.so.12 => /opt/mpich/lib/libmpifort.so.12 > (0x00002ae95cdf7000) > libmpi.so.12 => /opt/mpich/lib/libmpi.so.12 (0x00002ae95d02f000) > libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00002ae95d4ce000) > libm.so.6 => /lib64/libm.so.6 (0x00000038bcc00000) > libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000038c3c00000) > libc.so.6 => /lib64/libc.so.6 (0x00000038bc800000) > librt.so.1 => /lib64/librt.so.1 (0x00000038bd800000) > libpthread.so.0 => /lib64/libpthread.so.0 (0x00000038bd400000) > /lib64/ld-linux-x86-64.so.2 (0x00000038bc400000) > > > Thanks for the ldd suggestion, > > Rob > > On 08/10/2016 12:55 PM, Kenneth Raffenetti wrote: >> Hi Robert, >> >> Apologies for the delayed response. I'm unable to reproduce this on a >> CentOS 6 machine so far. Can you run ldd against your hello world >> binaries to double check the linkage? The MPI_UNWEIGHTED symbol should >> be resolved in the libmpi.so library by the inter-library dependency. >> >> Ken >> >> On 08/02/2016 02:00 PM, Robert Baurle wrote: >>> Sorry, I should have also stated that /opt/mpich is a symbolic link to >>> /opt/mpich_gfc. I define /opt/mpich to be a symbolic link to either: >>> >>> /opt/mpich_gfc >>> >>> or >>> >>> /opt/mpich_lahey >>> >>> depending on which Fortran90 compiler I want to use. I currently have >>> /opt/mpich symbolically linked to /opt/mpich_gfc, and /opt/mpich is >>> defined in my paths as shown below >>> >>> hyp22 rbaurle > echo $LD_LIBRARY_PATH >>> /usr/lib64/atlas:/opt/tecplotex/bin:/opt/mpich/lib:/opt/lahey-64/lib64:/usr/local/lib >>> >>> >>> >>> hyp22 rbaurle > echo $PATH >>> /home/rbaurle/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/sbin:/opt/lahey-64/bin:/opt/tecplot/bin:/opt/gridpro/bin:/opt/mpich/bin:.:/home/rbaurle/Vulcan/vulcan.bzr/Vulcan/Scripts >>> >>> >>> >>> >>> Rob >>> >>> On 08/02/2016 02:48 PM, Halim Amer wrote: >>>> you are probably linking to the wrong MPICH. You installed MPICH to >>>> /opt/mpich_gfc but the error occurs for MPICH installed at /opt/mpich. >>>> >>>> Do you have /opt/mpich in LD_LIBRARY_PATH? If that's the case, then >>>> you should remove it or place /opt/mpich_gfc before it. >>>> >>>> --Halim >>>> >>>> www.mcs.anl.gov/~aamer >>>> >>>> On 8/2/16 11:33 AM, Robert Baurle wrote: >>>>> Hello, >>>>> >>>>> I have installed MPICH 3.2 on a CENTOS 6 machine with the following >>>>> configuration options: >>>>> >>>>> setenv CC /usr/bin/gcc >>>>> setenv FC /usr/bin/gfortran >>>>> setenv CXX /usr/bin/c++ >>>>> setenv INS_DIR /opt/mpich_gfc >>>>> >>>>> ../mpich-3.2/configure --prefix=$INS_DIR --with-device=ch3:nemesis |& >>>>> tee configure.log >>>>> >>>>> make --jobs 4 |& tee make.log >>>>> >>>>> make install >>>>> >>>>> and the installation appeared to have completed without error (see the >>>>> attached configure.log and make.log files). However, when I attempt to >>>>> run a simple hello_world FORTRAN 90 program, I get the following >>>>> error: >>>>> >>>>> hyp22 rbaurle 2293> mpirun -n 3 a.out < hello.inp >>>>> a.out: symbol lookup error: /opt/mpich/lib/libmpifort.so.12: undefined >>>>> symbol: MPI_UNWEIGHTED >>>>> >>>>> If I repeat the installation, but change the Fortran90 compiler from >>>>> GFORTRAN to the Lahey 8.1 compiler, i.e.: >>>>> >>>>> setenv FC /opt/lahey-64/bin/lf95 >>>>> >>>>> then the hello_world program runs just fine. Do you have any idea what >>>>> might be going on here? I should also mention that the same error >>>>> appears if I install MPICH 3.1.3. >>>>> >>>>> Thanks, >>>>> >>>>> Rob >>>>> >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] undefined symbol: MPI_UNWEIGHTED
by Robert Baurle 10 Aug '16

10 Aug '16

I have figured out the problem. There was some sort of conflict with other libraries that were in my library path. In particular, I had Tecplot360EX in my LD_LIBRARY_PATH, and this package includes some libraries that take advantage of MPI-IO. Here is the result of ldd when this package was in my path: hyp22 rbaurle 2007> ldd hello.out linux-vdso.so.1 => (0x00007ffe001df000) libmpifort.so.12 => /opt/mpich/lib/libmpifort.so.12 (0x00002b0e09bf5000) libmpi.so.12 => /opt/tecplotex/bin/libmpi.so.12 (0x00002b0e09e2d000) libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00002b0e0a126000) libm.so.6 => /lib64/libm.so.6 (0x00000038bcc00000) libgcc_s.so.1 => /opt/tecplotex/bin/libgcc_s.so.1 (0x00002b0e0a419000) libc.so.6 => /lib64/libc.so.6 (0x00000038bc800000) librt.so.1 => /lib64/librt.so.1 (0x00000038bd800000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00000038bd400000) libopen-rte.so.12 => /opt/tecplotex/bin/libopen-rte.so.12 (0x00002b0e0a630000) libopen-pal.so.13 => /opt/tecplotex/bin/libopen-pal.so.13 (0x00002b0e0a8ab000) libpciaccess.so.0 => /usr/lib64/libpciaccess.so.0 (0x00000038bec00000) libdl.so.2 => /lib64/libdl.so.2 (0x00000038bd000000) libutil.so.1 => /lib64/libutil.so.1 (0x00000038ce800000) /lib64/ld-linux-x86-64.so.2 (0x00000038bc400000) When I removed this package from my LD_LIBRARY_PATH, the hello executable runs just fine and this is the result of of ldd: hyp22 rbaurle 2020> ldd hello.out linux-vdso.so.1 => (0x00007ffda41fd000) libmpifort.so.12 => /opt/mpich/lib/libmpifort.so.12 (0x00002ae95cdf7000) libmpi.so.12 => /opt/mpich/lib/libmpi.so.12 (0x00002ae95d02f000) libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00002ae95d4ce000) libm.so.6 => /lib64/libm.so.6 (0x00000038bcc00000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000038c3c00000) libc.so.6 => /lib64/libc.so.6 (0x00000038bc800000) librt.so.1 => /lib64/librt.so.1 (0x00000038bd800000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00000038bd400000) /lib64/ld-linux-x86-64.so.2 (0x00000038bc400000) Thanks for the ldd suggestion, Rob On 08/10/2016 12:55 PM, Kenneth Raffenetti wrote: > Hi Robert, > > Apologies for the delayed response. I'm unable to reproduce this on a > CentOS 6 machine so far. Can you run ldd against your hello world > binaries to double check the linkage? The MPI_UNWEIGHTED symbol should > be resolved in the libmpi.so library by the inter-library dependency. > > Ken > > On 08/02/2016 02:00 PM, Robert Baurle wrote: >> Sorry, I should have also stated that /opt/mpich is a symbolic link to >> /opt/mpich_gfc. I define /opt/mpich to be a symbolic link to either: >> >> /opt/mpich_gfc >> >> or >> >> /opt/mpich_lahey >> >> depending on which Fortran90 compiler I want to use. I currently have >> /opt/mpich symbolically linked to /opt/mpich_gfc, and /opt/mpich is >> defined in my paths as shown below >> >> hyp22 rbaurle > echo $LD_LIBRARY_PATH >> /usr/lib64/atlas:/opt/tecplotex/bin:/opt/mpich/lib:/opt/lahey-64/lib64:/usr/local/lib >> >> >> >> hyp22 rbaurle > echo $PATH >> /home/rbaurle/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/sbin:/opt/lahey-64/bin:/opt/tecplot/bin:/opt/gridpro/bin:/opt/mpich/bin:.:/home/rbaurle/Vulcan/vulcan.bzr/Vulcan/Scripts >> >> >> >> >> Rob >> >> On 08/02/2016 02:48 PM, Halim Amer wrote: >>> you are probably linking to the wrong MPICH. You installed MPICH to >>> /opt/mpich_gfc but the error occurs for MPICH installed at /opt/mpich. >>> >>> Do you have /opt/mpich in LD_LIBRARY_PATH? If that's the case, then >>> you should remove it or place /opt/mpich_gfc before it. >>> >>> --Halim >>> >>> www.mcs.anl.gov/~aamer >>> >>> On 8/2/16 11:33 AM, Robert Baurle wrote: >>>> Hello, >>>> >>>> I have installed MPICH 3.2 on a CENTOS 6 machine with the following >>>> configuration options: >>>> >>>> setenv CC /usr/bin/gcc >>>> setenv FC /usr/bin/gfortran >>>> setenv CXX /usr/bin/c++ >>>> setenv INS_DIR /opt/mpich_gfc >>>> >>>> ../mpich-3.2/configure --prefix=$INS_DIR --with-device=ch3:nemesis |& >>>> tee configure.log >>>> >>>> make --jobs 4 |& tee make.log >>>> >>>> make install >>>> >>>> and the installation appeared to have completed without error (see the >>>> attached configure.log and make.log files). However, when I attempt to >>>> run a simple hello_world FORTRAN 90 program, I get the following >>>> error: >>>> >>>> hyp22 rbaurle 2293> mpirun -n 3 a.out < hello.inp >>>> a.out: symbol lookup error: /opt/mpich/lib/libmpifort.so.12: undefined >>>> symbol: MPI_UNWEIGHTED >>>> >>>> If I repeat the installation, but change the Fortran90 compiler from >>>> GFORTRAN to the Lahey 8.1 compiler, i.e.: >>>> >>>> setenv FC /opt/lahey-64/bin/lf95 >>>> >>>> then the hello_world program runs just fine. Do you have any idea what >>>> might be going on here? I should also mention that the same error >>>> appears if I install MPICH 3.1.3. >>>> >>>> Thanks, >>>> >>>> Rob >>>> >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] undefined symbol: MPI_UNWEIGHTED
by Kenneth Raffenetti 10 Aug '16

10 Aug '16

Hi Robert, Apologies for the delayed response. I'm unable to reproduce this on a CentOS 6 machine so far. Can you run ldd against your hello world binaries to double check the linkage? The MPI_UNWEIGHTED symbol should be resolved in the libmpi.so library by the inter-library dependency. Ken On 08/02/2016 02:00 PM, Robert Baurle wrote: > Sorry, I should have also stated that /opt/mpich is a symbolic link to > /opt/mpich_gfc. I define /opt/mpich to be a symbolic link to either: > > /opt/mpich_gfc > > or > > /opt/mpich_lahey > > depending on which Fortran90 compiler I want to use. I currently have > /opt/mpich symbolically linked to /opt/mpich_gfc, and /opt/mpich is > defined in my paths as shown below > > hyp22 rbaurle > echo $LD_LIBRARY_PATH > /usr/lib64/atlas:/opt/tecplotex/bin:/opt/mpich/lib:/opt/lahey-64/lib64:/usr/local/lib > > > hyp22 rbaurle > echo $PATH > /home/rbaurle/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/sbin:/opt/lahey-64/bin:/opt/tecplot/bin:/opt/gridpro/bin:/opt/mpich/bin:.:/home/rbaurle/Vulcan/vulcan.bzr/Vulcan/Scripts > > > > Rob > > On 08/02/2016 02:48 PM, Halim Amer wrote: >> you are probably linking to the wrong MPICH. You installed MPICH to >> /opt/mpich_gfc but the error occurs for MPICH installed at /opt/mpich. >> >> Do you have /opt/mpich in LD_LIBRARY_PATH? If that's the case, then >> you should remove it or place /opt/mpich_gfc before it. >> >> --Halim >> >> www.mcs.anl.gov/~aamer >> >> On 8/2/16 11:33 AM, Robert Baurle wrote: >>> Hello, >>> >>> I have installed MPICH 3.2 on a CENTOS 6 machine with the following >>> configuration options: >>> >>> setenv CC /usr/bin/gcc >>> setenv FC /usr/bin/gfortran >>> setenv CXX /usr/bin/c++ >>> setenv INS_DIR /opt/mpich_gfc >>> >>> ../mpich-3.2/configure --prefix=$INS_DIR --with-device=ch3:nemesis |& >>> tee configure.log >>> >>> make --jobs 4 |& tee make.log >>> >>> make install >>> >>> and the installation appeared to have completed without error (see the >>> attached configure.log and make.log files). However, when I attempt to >>> run a simple hello_world FORTRAN 90 program, I get the following error: >>> >>> hyp22 rbaurle 2293> mpirun -n 3 a.out < hello.inp >>> a.out: symbol lookup error: /opt/mpich/lib/libmpifort.so.12: undefined >>> symbol: MPI_UNWEIGHTED >>> >>> If I repeat the installation, but change the Fortran90 compiler from >>> GFORTRAN to the Lahey 8.1 compiler, i.e.: >>> >>> setenv FC /opt/lahey-64/bin/lf95 >>> >>> then the hello_world program runs just fine. Do you have any idea what >>> might be going on here? I should also mention that the same error >>> appears if I install MPICH 3.1.3. >>> >>> Thanks, >>> >>> Rob >>> >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Per Thread CS
by mdosanjh＠cs.unm.edu 09 Aug '16

09 Aug '16

Hello, I'm trying to do a research improvements to fine grained threading models in MPI. I was trying to compile MPICH 3.2 with --enable-threads=multiple --enable-thread-cs=per-object and got the following error: src/mpid/ch3/src/ch3u_handle_connection.c:225: error: 'MPIDI_VC_t' has no member named 'pobj_mutex' Is there a way to get this to compile or another version I can try? Thanks, Matthew _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0