I don't think there's any uncompleted communication. The example I show below is one of the very simple examples shipping with MPICH itself (sorry if that wasn't completely clear). Cheers, Edric. ________________________________ From: Joachim Jenke via discuss <[email protected]> Sent: 13 December 2023 17:16 To: [email protected] <[email protected]>; Zhou, Hui <[email protected]> Cc: Joachim Jenke <[email protected]> Subject: Re: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 If your code hangs in MPI_Finalize for certain communication implementation, this sounds like an uncompleted communication. Are you sure that you have no MPI communication ongoing when calling MPI_Finalize? - Joachim Am 13.12.23 um 17:45 schrieb Edric Ellis via discuss:

Ok, that's good to know, I'll stick with simply "ofi:tcp" for now.

Thanks, Edric. ------------------------------------------------------------------------ *From:* Zhou, Hui <[email protected]> *Sent:* 13 December 2023 15:39 *To:* [email protected] <[email protected]> *Cc:* Edric Ellis <[email protected]> *Subject:* Re: Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 Hi Edric,

I am not sure which part is hanging, but you don't need to enable |ofi:shm|​ (libfabric shm provider). The ch4 device comes with its own shared memory functionality.

-- Hui ------------------------------------------------------------------------ *From:* Edric Ellis via discuss <[email protected]> *Sent:* Wednesday, December 13, 2023 7:05 AM *To:* [email protected] <[email protected]> *Cc:* Edric Ellis <[email protected]> *Subject:* [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 I'm working on getting a build of mpich-4.1.2 ready to replace our old build of mpich-3.3.2. With older MPICH releases, we used the "nemesis" channel via ch3 to provide support for shared-memory configurations as well as TCP/IP. In ch4, I thought the nearest equivalent would be:

--with-device=ch4:ofi:tcp,shm

The "tcp" portion of this seems to work just fine, but "shm" hangs during (I think) MPI_Finalize, requiring a CTRL-C to kill it. For example, in the build area,

$ ./src/pm/hydra/mpiexec.hydra -n 2 ./examples/cpi Process 0 of 2 is on uk-eellis-l Process 1 of 2 is on uk-eellis-l pi is approximately 3.1415926544231318, Error is 0.0000000008333387 wall clock time = 0.000019 ^C[mpiexec@uk-eellis-l] Sending Ctrl-C to processes as requested [mpiexec@uk-eellis-l] Press Ctrl-C again to force abort

=================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 829015 RUNNING AT uk-eellis-l = EXIT CODE: 2 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Interrupt (signal 2) This typically refers to a problem with your application. Please see the FAQ page for debugging suggestions

Things work fine if I force FI_PROVIDER=tcp. Am I missing something?

Here's the configure line I'm using:

$ ./configure --prefix <prefix> --with-device=ch4:ofi:tcp,shm --enable-shared --with-libfabric=embedded --enable-fortran --enable-efa=no

This is running on a Debian 11 system, gcc 10.3.0.

Cheers, Edric.

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>

-- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 [email protected] www.itc.rwth-aachen.de<http://www.itc.rwth-aachen.de>