March 2021 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] How to set port range used under LSF?
by Zhou, Hui 09 Mar '21

09 Mar '21

Hi Sendu, Could you try `MPIEXEC_PORT_RANGE` in stead? I know it is confusing and the documentation probably need update/correction, but `MPICH_PORT_RANGE` is for `MPICH` rather than `MPIEXEC`. -- Hui Zhou From: Sendu Bala via discuss <discuss(a)mpich.org> Date: Tuesday, March 9, 2021 at 7:16 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Sendu Bala <sb10(a)sanger.ac.uk> Subject: [mpich-discuss] How to set port range used under LSF? Via a bsub, I’m doing: MPICH_PORT_RANGE="46107:46140” mpiexec mpich/examples/cpi When I ssh to the controlling node, I see it has spawned a set of blaunch processes with `--control-port node-12-3-2:46107` as expected, but: ss -l -p -n | grep blaunch tcp LISTEN 0 128 0.0.0.0:34361 0.0.0.0:* users:(("blaunch",pid=2823,fd=7)) tcp LISTEN 0 128 0.0.0.0:46107 0.0.0.0:* users:(("cpi",pid=2839,fd=5),("blaunch",pid=2837,fd=5),("blaunch",pid=2836,fd=5),("blaunch",pid=2835,fd=5),("blaunch",pid=2834,fd=5),("blaunch",pid=2833,fd=5),("blaunch",pid=2832,fd=5),("blaunch",pid=2831,fd=5),("blaunch",pid=2830,fd=5),("blaunch",pid=2829,fd=5),("blaunch",pid=2828,fd=5),("blaunch",pid=2827,fd=5),("blaunch",pid=2826,fd=5),("blaunch",pid=2825,fd=5),("blaunch",pid=2824,fd=5),("blaunch",pid=2823,fd=5),("hydra_pmi_proxy",pid=2822,fd=5),("mpiexec",pid=2821,fd=5)) tcp LISTEN 0 128 0.0.0.0:43741 0.0.0.0:* users:(("blaunch",pid=2825,fd=12)) tcp LISTEN 0 128 0.0.0.0:41983 0.0.0.0:* users:(("blaunch",pid=2830,fd=22)) tcp LISTEN 0 128 0.0.0.0:41215 0.0.0.0:* users:(("blaunch",pid=2832,fd=26)) tcp LISTEN 0 128 0.0.0.0:34433 0.0.0.0:* users:(("blaunch",pid=2831,fd=24)) tcp LISTEN 0 128 0.0.0.0:33219 0.0.0.0:* users:(("blaunch",pid=2827,fd=16)) tcp LISTEN 0 128 0.0.0.0:34405 0.0.0.0:* users:(("blaunch",pid=2837,fd=36)) tcp LISTEN 0 128 0.0.0.0:43465 0.0.0.0:* users:(("blaunch",pid=2836,fd=34)) tcp LISTEN 0 128 0.0.0.0:39755 0.0.0.0:* users:(("blaunch",pid=2833,fd=28)) tcp LISTEN 0 128 0.0.0.0:38095 0.0.0.0:* users:(("blaunch",pid=2829,fd=20)) tcp LISTEN 0 128 0.0.0.0:44625 0.0.0.0:* users:(("blaunch",pid=2834,fd=30)) tcp LISTEN 0 128 0.0.0.0:35345 0.0.0.0:* users:(("blaunch",pid=2835,fd=32)) tcp LISTEN 0 128 0.0.0.0:43827 0.0.0.0:* users:(("blaunch",pid=2826,fd=14)) tcp LISTEN 0 128 0.0.0.0:40915 0.0.0.0:* users:(("blaunch",pid=2828,fd=18)) tcp LISTEN 0 128 0.0.0.0:42549 0.0.0.0:* users:(("blaunch",pid=2824,fd=9)) Why are these all listening on ports outside my range? I’ve also tried setting MPIEXEC_PORT_RANGE and MPIR_CVAR_CH3_PORT_RANGE and still have the problem. Is there any way to fully control the ports used? Cheers, Sendu. -- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

How to set port range used under LSF?
by Sendu Bala 09 Mar '21

09 Mar '21

Via a bsub, I’m doing: MPICH_PORT_RANGE="46107:46140” mpiexec mpich/examples/cpi When I ssh to the controlling node, I see it has spawned a set of blaunch processes with `--control-port node-12-3-2:46107` as expected, but: ss -l -p -n | grep blaunch tcp LISTEN 0 128 0.0.0.0:34361 0.0.0.0:* users:(("blaunch",pid=2823,fd=7)) tcp LISTEN 0 128 0.0.0.0:46107 0.0.0.0:* users:(("cpi",pid=2839,fd=5),("blaunch",pid=2837,fd=5),("blaunch",pid=2836,fd=5),("blaunch",pid=2835,fd=5),("blaunch",pid=2834,fd=5),("blaunch",pid=2833,fd=5),("blaunch",pid=2832,fd=5),("blaunch",pid=2831,fd=5),("blaunch",pid=2830,fd=5),("blaunch",pid=2829,fd=5),("blaunch",pid=2828,fd=5),("blaunch",pid=2827,fd=5),("blaunch",pid=2826,fd=5),("blaunch",pid=2825,fd=5),("blaunch",pid=2824,fd=5),("blaunch",pid=2823,fd=5),("hydra_pmi_proxy",pid=2822,fd=5),("mpiexec",pid=2821,fd=5)) tcp LISTEN 0 128 0.0.0.0:43741 0.0.0.0:* users:(("blaunch",pid=2825,fd=12)) tcp LISTEN 0 128 0.0.0.0:41983 0.0.0.0:* users:(("blaunch",pid=2830,fd=22)) tcp LISTEN 0 128 0.0.0.0:41215 0.0.0.0:* users:(("blaunch",pid=2832,fd=26)) tcp LISTEN 0 128 0.0.0.0:34433 0.0.0.0:* users:(("blaunch",pid=2831,fd=24)) tcp LISTEN 0 128 0.0.0.0:33219 0.0.0.0:* users:(("blaunch",pid=2827,fd=16)) tcp LISTEN 0 128 0.0.0.0:34405 0.0.0.0:* users:(("blaunch",pid=2837,fd=36)) tcp LISTEN 0 128 0.0.0.0:43465 0.0.0.0:* users:(("blaunch",pid=2836,fd=34)) tcp LISTEN 0 128 0.0.0.0:39755 0.0.0.0:* users:(("blaunch",pid=2833,fd=28)) tcp LISTEN 0 128 0.0.0.0:38095 0.0.0.0:* users:(("blaunch",pid=2829,fd=20)) tcp LISTEN 0 128 0.0.0.0:44625 0.0.0.0:* users:(("blaunch",pid=2834,fd=30)) tcp LISTEN 0 128 0.0.0.0:35345 0.0.0.0:* users:(("blaunch",pid=2835,fd=32)) tcp LISTEN 0 128 0.0.0.0:43827 0.0.0.0:* users:(("blaunch",pid=2826,fd=14)) tcp LISTEN 0 128 0.0.0.0:40915 0.0.0.0:* users:(("blaunch",pid=2828,fd=18)) tcp LISTEN 0 128 0.0.0.0:42549 0.0.0.0:* users:(("blaunch",pid=2824,fd=9)) Why are these all listening on ports outside my range? I’ve also tried setting MPIEXEC_PORT_RANGE and MPIR_CVAR_CH3_PORT_RANGE and still have the problem. Is there any way to fully control the ports used? Cheers, Sendu. -- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

1 0

Make check failure: macOS Big Sur intel gcc 10.2
by Blaise A Bourdin 08 Mar '21

08 Mar '21

Hi, The test_json_pointer test in make check fails with mpich-3.4.1 MacSO Big Sur intel and gcc-10.2 (home-brew compilers): MacBookGray:tests $ ./test_json_pointer PASSED - GET - LOADED TEST JSON { "foo": [ "bar", "baz" ], "": 0, "a\/b": 1, "c%d": 2, "e^f": 3, "g|h": 4, "i\\j": 5, "k\"l": 6, " ": 7, "m~n": 8 } PASSED - GET - ENTIRE OBJECT WORKED PASSED - GET - /foo == ['bar', 'baz'] PASSED - GET - /foo/0 == 'bar' PASSED - GET - / == 0 PASSED - GET - /a~1b == 1 PASSED - GET - /c%d == 2 PASSED - GET - /e^f == 3 PASSED - GET - /g|h == 4 PASSED - GET - /i\j == 5 PASSED - GET - /k"l == 6 PASSED - GET - / == 7 PASSED - GET - /m~0n == 8 { "arr": [ { "obj": [ { }, { }, { "obj1": 0, "obj2": "1" } ] } ], "obj": { "obj": { "obj": [ { "obj1": 0, "obj2": "1" } ] } } } Assertion failed: (0 == json_pointer_getf(jo1, &jo2, "/%s/%d/%s/%d/%s", "arr", 0, "obj", 2, "obj2")), function test_recursion_get, file test_json_pointer.c, line 136. Abort trap: 6 Strangely enough, the same test works fine under arm64. Any suggestion? Regards, Blaise -- A.K. & Shirley Barton Professor of Mathematics Adjunct Professor of Mechanical Engineering Adjunct of the Center for Computation & Technology Louisiana State University, Lockett Hall Room 344, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 Web http://www.math.lsu.edu/~bourdin

1 0

Re: [mpich-discuss] Build failure on macOS Big Sur arm64
by Zhou, Hui 07 Mar '21

07 Mar '21

Yes, this is possible, and it probably should be done this way. But currently there are a few steps before this can happen. It’s in my todo list. Sorry that I can’t have a better answer at the moment. -- Hui Zhou From: Blaise A Bourdin <bourdin(a)lsu.edu> Date: Sunday, March 7, 2021 at 4:11 PM To: Zhou, Hui <zhouh(a)anl.gov> Cc: discuss(a)mpich.org <discuss(a)mpich.org> Subject: Re: Build failure on macOS Big Sur arm64 As far as I understand, the FORTRAN bindings are auto generated by a python script. Would there be a way to get the result of the configure test for float128 to trigger or not the generation of the real128 and complex128 bindings? Or is there a preprocessor variables that stores the result of this test?

1 0

Re: [mpich-discuss] Build failure on macOS Big Sur arm64
by Blaise A Bourdin 07 Mar '21

07 Mar '21

Hi, As far as I understand, the FORTRAN bindings are auto generated by a python script. Would there be a way to get the result of the configure test for float128 to trigger or not the generation of the real128 and complex128 bindings? Or is there a preprocessor variables that stores the result of this test? Blaise Sent from a mobile device On Mar 7, 2021, at 3:58 PM, Zhou, Hui <zhouh(a)anl.gov> wrote: Unfortunately there is no option to disable real128 or complex128 at the moment. You can remove them by editing src/binding/fortran/use_mpi_f08/mpi_f08_types.f90, search for “real128” and “complex128”, remove the lines declaring them as well as the two subroutines below. -- Hui Zhou From: Blaise A Bourdin via discuss <discuss(a)mpich.org> Date: Sunday, March 7, 2021 at 7:15 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Blaise A Bourdin <bourdin(a)lsu.edu> Subject: [mpich-discuss] Build failure on macOS Big Sur arm64 So it looks like in this case, the f08 bindings for real128 and complex128 are created even though they shouldn’t. I was not able to figure out how to disable their creation during the configure stage. Can somebody help? My config.log is attached.

1 0

Re: [mpich-discuss] Build failure on macOS Big Sur arm64
by Zhou, Hui 07 Mar '21

07 Mar '21

Unfortunately there is no option to disable real128 or complex128 at the moment. You can remove them by editing src/binding/fortran/use_mpi_f08/mpi_f08_types.f90, search for “real128” and “complex128”, remove the lines declaring them as well as the two subroutines below. -- Hui Zhou From: Blaise A Bourdin via discuss <discuss(a)mpich.org> Date: Sunday, March 7, 2021 at 7:15 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Blaise A Bourdin <bourdin(a)lsu.edu> Subject: [mpich-discuss] Build failure on macOS Big Sur arm64 So it looks like in this case, the f08 bindings for real128 and complex128 are created even though they shouldn’t. I was not able to figure out how to disable their creation during the configure stage. Can somebody help? My config.log is attached.

1 0

Build failure on macOS Big Sur arm64
by Blaise A Bourdin 07 Mar '21

07 Mar '21

Hi, As far as I understand, the Big Sur arm64 port of gcc/gfortran 10.2 doesn’t know about float128 hence real128 and complex128 yet. This leads to the following failure when trying to build mpich-3.4.1 compiled with ./configure --prefix=$MPICH_HOME \ --enable-fast=all,O3 \ --enable-g=dbg \ --enable-romio \ --enable-shared \ --enable-sharedlibs=gcc-osx \ --with-pm=hydra \ CC=gcc-10 CXX=g++-10 FC=gfortran-10 F77=gfortran-10 \ CFLAGS="-fgnu89-inline -Wno-deprecated" \ CXXFLAGS="-Wno-deprecated" \ FFLAGS=-fallow-argument-mismatch SiMini:mpich-3.4.1 $ make if test ! -h ./src/include/mpio.h ; then \ rm -f ./src/include/mpio.h ; \ ( cd ./src/include && \ ln -s ../mpi/romio/include/mpio.h ) ; \ fi MOD src/binding/fortran/use_mpi_f08/mpi_c_interface_types.stamp MOD src/binding/fortran/use_mpi_f08/mpi_f08_types.stamp src/binding/fortran/use_mpi_f08/mpi_f08_types.f90:355:17: 355 | real(real128),dimension(..) :: x | 1 Error: Kind -1 not supported for type REAL at (1) src/binding/fortran/use_mpi_f08/mpi_f08_types.f90:385:20: 385 | complex(real128),dimension(..) :: x | 1 Error: Kind -1 not supported for type COMPLEX at (1) src/binding/fortran/use_mpi_f08/mpi_f08_types.f90:383:36: 383 | subroutine MPI_Sizeof_xcomplex128 (x, size, ierror) | 1 Error: Symbol 'x' at (1) has no IMPLICIT type src/binding/fortran/use_mpi_f08/mpi_f08_types.f90:353:33: 353 | subroutine MPI_Sizeof_xreal128 (x, size, ierror) | 1 Error: Symbol 'x' at (1) has no IMPLICIT type src/binding/fortran/use_mpi_f08/mpi_f08_types.f90:353:30: 353 | subroutine MPI_Sizeof_xreal128 (x, size, ierror) | 1 ...... 383 | subroutine MPI_Sizeof_xcomplex128 (x, size, ierror) | 2 Error: Ambiguous interfaces in generic interface 'mpi_sizeof' for 'mpi_sizeof_xreal128' at (1) and 'mpi_sizeof_xcomplex128' at (2) make: *** [src/binding/fortran/use_mpi_f08/mpi_f08_types.stamp] Error 1 After communicating with him, I got the following reply rom FX Coudert who was involved in the port of gcc-10 to arm64: 1. The Fortran 2008 standard states: ---- 13.8.2.21 REAL32, REAL64, and REAL128 The values of these default integer scalar named constants shall be those of the kind type parameters that specify a REAL type whose storage size expressed in bits is 32, 64, and 128 respectively. If, for any of these constants, the processor supports more than one kind of that size, it is processor dependent which kind value is provided. If the processor supports no kind of a particular size, that constant shall be equal to -2 if the processor supports kinds of a larger size and -1 otherwise. ---- From your error message it is clear that REAL128 is correctly defined as -1. 2. Your configure output shows "checking size of __float128: 0” and "checking whether real*16 is supported: no”, so it seems the build has all the information it needs to avoid using REAL128 unconditionally. So it looks like in this case, the f08 bindings for real128 and complex128 are created even though they shouldn’t. I was not able to figure out how to disable their creation during the configure stage. Can somebody help? My config.log is attached. Regards, Blaise -- A.K. & Shirley Barton Professor of Mathematics Adjunct Professor of Mechanical Engineering Adjunct of the Center for Computation & Technology Louisiana State University, Lockett Hall Room 344, Baton Rouge, LA 70803, USA<x-apple-data-detectors://0/1> Tel. +1 (225) 578 1612<tel:+1%20(225)%20578%201612>, Fax +1 (225) 578 4276<tel:+1%20(225)%20578%204276> Web http://www.math.lsu.edu/~bourdin

1 0

Re: [mpich-discuss] Error with mpich-3.3.2 and ucx-1.8.0
by Raffenetti, Kenneth J. 04 Mar '21

04 Mar '21

Hi Junchao, We do not have any plans to provide another MPICH release in the 3.3.x series. It is good to know this bug is fixed in the latest version of MPICH. Ken On 3/2/21, 8:17 PM, "Junchao Zhang via discuss" <discuss(a)mpich.org> wrote: Hello, We met an error with mpich-3.3.2 and ucx-1.8.0. See the attached example, which uses a user defined data type in MPI_Startall. MPICH was configured with --with-device=ch4:ucx --with-ucx=/path/to/ucx-1.8.0. The error stack is $ mpirun -n 2 ./dtype Assertion failed in file src/mpi/datatype/type_free.c at line 38: (((datatype_ptr)))->ref_count >= 0 /home/jczhang/soft/lib/libmpi.so.12(+0x48fa1f) [0x7f4019b6ba1f] /home/jczhang/soft/lib/libmpi.so.12(MPL_backtrace_show+0x18) [0x7f4019b6baff] /home/jczhang/soft/lib/libmpi.so.12(+0x441e27) [0x7f4019b1de27] /home/jczhang/soft/lib/libmpi.so.12(+0xfebc2) [0x7f40197dabc2] /home/jczhang/soft/lib/libmpi.so.12(MPI_Type_free+0x5fd) [0x7f40197db239] It does not happen with mpich-3.4.1. --Junchao Zhang

1 0

Re: [mpich-discuss] MPI_Info key host error in SLURM
by Raffenetti, Kenneth J. 04 Mar '21

04 Mar '21

I notice in the output that the sbatch version uses --launcher slurm while the interactive version uses --launcher ssh. Can you try adding --launcher ssh to the mpiexec command of you sbatch script and see if it makes a difference? Ken On 3/1/21, 1:38 PM, "Iker Martín Álvarez" <martini(a)uji.es> wrote: Hi Ken, Thanks for your reply. Here are attached two files with the arg "-v". BatchOutput.txt has the output which is giving the error message when the code is executed with the command sbatch, while the file InteractiveOutput.txt is the one executed interactively in the node and work as expected. Both of them has been compiled and executed with mpich 3.4.1. Thanks, Iker El vie, 26 feb 2021 a las 16:15, Raffenetti, Kenneth J. (<raffenet(a)mcs.anl.gov>) escribió: Hi, Could you add "-v" to your mpiexec command and provide the output? The "host" info key is handled by the process manager when executing the spawn command. Ken On 2/24/21, 11:55 AM, "Iker Martín Álvarez via discuss" <discuss(a)mpich.org> wrote: Hi, I have been working around the MPI_Comm_spawn function with the key "host" along a value in a system with the SLURM resource manager (slurm-wlm 17.11.2). The function works as expected, but when I send to execute the code with the sbatch command, an error arises. This does not happen when I execute directly the code in the same machine SLURM decided to execute it when it was sended with sbatch. In both cases with the key "host", as when I do not use the key, it works just fine. The same code has been tested with MPICH 3.3.2 and 3.4.1, which gives different errors. Also, I tried it with other implementations (OpenMPI and Intel MPI), which works as expected creating the processes in the indicated host. I would like to create processes by MPI_Comm_spawn in an assigned host, so if there are other key values for the Info argument, I could try them, but I have not found any looking through MPICH documentation. Here is the code I have been using: int main(int argc, char ** argv) { int myId, numP; MPI_Info info; MPI_Comm comm; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &myId); MPI_Comm_size(MPI_COMM_WORLD, &numP); MPI_Comm comm_par; MPI_Comm_get_parent(&comm_par); if(comm_par != MPI_COMM_NULL ) { if(myId == ROOT) { printf("SONS\n"); fflush(stdout); } } else { if(myId == ROOT) { printf("FATHERS\n"); fflush(stdout); } MPI_Info_create(&info); MPI_Info_set(info, "host", "n00"); MPI_Comm_spawn(argv[0], MPI_ARGV_NULL, numP, info, ROOT, MPI_COMM_WORLD, &comm, MPI_ERRCODES_IGNORE); } MPI_Finalize(); return 0; } For MPICH 3.4.1 there is sometimes no error and the code stops working at MPI_Comm_spawn function, other times this error is shown: Abort(1615120) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init: Internal MPI error!, error stack: MPIR_Init_thread(152)...: MPID_Init(562)..........: MPIDU_Init_shm_init(195): Init_shm_barrier(94)....: Internal MPI error! barrier not initialized Also, the error code for MPICH 3.3.2: Assertion failed in file src/mpid/ch3/channels/nemesis/src/mpid_nem_init.c at line 683: our_pg_rank < pg->size Assertion failed in file src/mpid/ch3/channels/nemesis/src/mpid_nem_init.c at line 683: our_pg_rank < pg->size Assertion failed in file src/mpid/ch3/channels/nemesis/src/mpid_nem_init.c at line 683: our_pg_rank < pg->size libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable libbacktrace: no debug info in ELF executable internal ABORT - process 0 Thanks, Iker

1 0

Re: [mpich-discuss] [EXTERNAL] Re: Spawned process hanging in MPI_Finalize
by Joachim Protze 04 Mar '21

04 Mar '21

Hi Kurt, as a mental model, we can see MPI_Comm_spawn and MPI_Intercomm_create as collective constructors of the inter-comm object. All collectively created MPI objects also need a collective destructor call for full cleanup. MPI_Comm_disconnect is this collective destructor. MPI_Finalize implies MPI_Comm_disconnect for all undestroyed communicators of the process. From that perspective, it might even be sufficient to call MPI_Comm_disconnect on the parent process and let MPI_Finalize in the child process do the job. If MPI_Intercomm_create creates an inter-comm in a process you want to stop early, all involved processes will need to call MPI_Comm_disconnect before the process can terminate. Best Joachim Am 04.03.21 um 01:55 schrieb Mccall, Kurt E. (MSFC-EV41): > Joachim, > > Thanks, that helped! Is it necessary to call MPI_Comm_disconnect on inter-communicators that are created by MPI_Intercomm_create? > > Kurt > > -----Original Message----- > From: Joachim Protze <protze(a)itc.rwth-aachen.de> > Sent: Wednesday, March 3, 2021 5:52 AM > To: discuss(a)mpich.org > Cc: Mccall, Kurt E. (MSFC-EV41) <kurt.e.mccall(a)nasa.gov> > Subject: [EXTERNAL] Re: [mpich-discuss] Spawned process hanging in MPI_Finalize > > Hi Kurt, > > did you call MPI_Comm_disconnect on all processes connected with the > inter-communicator? Also the parent process needs to disconnect from the > inter-comm before the MPI_Comm_disconnect can return. > > - Joachim > > Am 03.03.21 um 01:54 schrieb Mccall, Kurt E. (MSFC-EV41) via discuss: >> I have a parent process that creates a child via MPI_Comm_spawn(). When the child decides it has to exit, it is hanging in MPI_Finalize(). It does the same if it calls MPI_Comm_disconnect() before MPI_Finalize. >> >> Here is the stack trace in the child: >> >> (gdb) where >> #0 0x00007fc6f2fedde0 in __poll_nocancel () from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libc.so.6 >> #1 0x00007fc6f4dc840e in MPID_nem_tcp_connpoll () at src/mpid/ch3/channels/nemesis/netmod/tcp/socksm.c:1819 >> #2 0x00007fc6f4db857e in MPID_nem_network_poll () at src/mpid/ch3/channels/nemesis/src/mpid_nem_network_poll.c:16 >> #3 0x00007fc6f4dafc43 in MPIDI_CH3I_Progress () at src/mpid/ch3/channels/nemesis/src/ch3_progress.c:1019 >> #4 0x00007fc6f4d5094d in MPIDI_CH3U_VC_WaitForClose () at src/mpid/ch3/src/ch3u_handle_connection.c:383 >> #5 0x00007fc6f4d94efa in MPID_Finalize () at src/mpid/ch3/src/mpid_finalize.c:110 >> #6 0x00007fc6f4c432ca in PMPI_Finalize () at src/mpi/init/finalize.c:260 >> #7 0x0000000000408a85 in needles::MpiWorker::finalize () at src/MpiWorker.cpp:470 >> >> Maybe I have a communication that hasn't completed, or the child is waiting for the parent to call MPI_Finalize. I believe that you (Ken, Hui) told me that it shouldn't do the latter. >> >> Is there a way for the child to cleanly exit without hanging in MPI_Finalize? I tried calling MPI_Cancel() in the child on the only possible communication request that I knew of, but it didn't help. >> It just occurred to me that I haven't tried calling MPI_Cancel on the requests in the parent... >> >> Thanks, >> Kurt >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > -- Dipl.-Inf. Joachim Protze IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0