- discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Install errors on MAC OS Sonoma: "error: expected declaration specifiers or ‘...’ before string constant"
by Zhou, Hui 01 Mar '24

01 Mar '24

I don't know what is causing the error, but here are a few work arounds: 1. You may try configure with --with-device=ch3. It uses an older device that is in maintenance mode, but likely it will work for you. 2. Try install libfabric separately and configure mpich with --with-libfabric=prefix 3. You may try brew install libfabric or brew install mpich. -- Hui ________________________________ From: Douglas G Dommermuth via discuss <discuss(a)mpich.org> Sent: Friday, March 1, 2024 6:09 PM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Douglas G Dommermuth <dgd(a)mit.edu> Subject: [mpich-discuss] Install errors on MAC OS Sonoma: "error: expected declaration specifiers or ‘...’ before string constant" I am compiling mpich 4. 2. 0 on a MacBook Air Apple M1 running macOS Sonoma 14. 3. 1. I am using gcc and gfortran versions 13. 2. 0. I get the following errors when I build MPICH: /Applications/Xcode. app/Contents/Developer/usr/bin/make all-am depbase=`echo ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd I am compiling mpich 4.2.0 on a MacBook Air Apple M1 running macOS Sonoma 14.3.1. I am using gcc and gfortran versions 13.2.0. I get the following errors when I build MPICH: /Applications/Xcode.app/Contents/Developer/usr/bin/make all-am depbase=`echo util/info.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\ gcc -DHAVE_CONFIG_H -I. -I./include -D_GNU_SOURCE -D__USE_XOPEN2K8 -DSYSCONFDIR=\"/Users/douglasdommermuth/mpich2/mpich2-install/etc\" -DRDMADIR=\"@rdmadir@\" -DPROVDLDIR=\"/Users/douglasdommermuth/mpich2/mpich2-install/lib/libfabric\" -I./prov/sockets/include -I./prov/sockets -Wall -O2 -DNDEBUG -MT util/info.o -MD -MP -MF $depbase.Tpo -c -o util/info.o util/info.c &&\ mv -f $depbase.Tpo $depbase.Po In file included from ./include/osx/osd.h:182, from ./include/ofi_osd.h:60, from util/info.c:38: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/lock.h:114:33: error: expected declaration specifiers or ‘...’ before string constant 114 | OS_SWIFT_UNAVAILABLE_FROM_ASYNC("Use OSAllocatedUnfairLock.performWhileLocked() for async-safe scoped locking") | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/lock.h:140:33: error: expected declaration specifiers or ‘...’ before string constant 140 | OS_SWIFT_UNAVAILABLE_FROM_ASYNC("Use OSAllocatedUnfairLock.tryPerformWhileLocked() for async-safe scoped locking") | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/lock.h:154:33: error: expected declaration specifiers or ‘...’ before string constant 154 | OS_SWIFT_UNAVAILABLE_FROM_ASYNC("Use OSAllocatedUnfairLock.performWhileLocked() for async-safe scoped locking") | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./include/osx/osd.h: In function ‘pthread_spin_lock’: ./include/osx/osd.h:194:9: warning: implicit declaration of function ‘os_unfair_lock_lock’ [-Wimplicit-function-declaration] 194 | os_unfair_lock_lock(lock); | ^~~~~~~~~~~~~~~~~~~ ./include/osx/osd.h: In function ‘pthread_spin_unlock’: ./include/osx/osd.h:200:9: warning: implicit declaration of function ‘os_unfair_lock_unlock’ [-Wimplicit-function-declaration] 200 | os_unfair_lock_unlock(lock); | ^~~~~~~~~~~~~~~~~~~~~ ./include/osx/osd.h: In function ‘pthread_spin_trylock’: ./include/osx/osd.h:206:16: warning: implicit declaration of function ‘os_unfair_lock_trylock’ [-Wimplicit-function-declaration] 206 | return os_unfair_lock_trylock(lock) ? 0 : EBUSY; | ^~~~~~~~~~~~~~~~~~~~~~ make[3]: *** [util/info.o] Error 1 make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 It looks like errors in macOS’s command line tools. I don’t know how to circumvent the problem. Various text and log files are attached in mpich2.tar.gz. A copy of the lock.h code is also included in the zip file. Thank you, Doug.

1 0

Install errors on MAC OS Sonoma: "error: expected declaration specifiers or ‘...’ before string constant"
by Douglas G Dommermuth 01 Mar '24

01 Mar '24

I am compiling mpich 4.2.0 on a MacBook Air Apple M1 running macOS Sonoma 14.3.1. I am using gcc and gfortran versions 13.2.0. I get the following errors when I build MPICH: /Applications/Xcode.app/Contents/Developer/usr/bin/make all-am depbase=`echo util/info.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\ gcc -DHAVE_CONFIG_H -I. -I./include -D_GNU_SOURCE -D__USE_XOPEN2K8 -DSYSCONFDIR=\"/Users/douglasdommermuth/mpich2/mpich2-install/etc\" -DRDMADIR=\"@rdmadir@\" -DPROVDLDIR=\"/Users/douglasdommermuth/mpich2/mpich2-install/lib/libfabric\" -I./prov/sockets/include -I./prov/sockets -Wall -O2 -DNDEBUG -MT util/info.o -MD -MP -MF $depbase.Tpo -c -o util/info.o util/info.c &&\ mv -f $depbase.Tpo $depbase.Po In file included from ./include/osx/osd.h:182, from ./include/ofi_osd.h:60, from util/info.c:38: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/lock.h:114:33: error: expected declaration specifiers or ‘...’ before string constant 114 | OS_SWIFT_UNAVAILABLE_FROM_ASYNC("Use OSAllocatedUnfairLock.performWhileLocked() for async-safe scoped locking") | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/lock.h:140:33: error: expected declaration specifiers or ‘...’ before string constant 140 | OS_SWIFT_UNAVAILABLE_FROM_ASYNC("Use OSAllocatedUnfairLock.tryPerformWhileLocked() for async-safe scoped locking") | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/os/lock.h:154:33: error: expected declaration specifiers or ‘...’ before string constant 154 | OS_SWIFT_UNAVAILABLE_FROM_ASYNC("Use OSAllocatedUnfairLock.performWhileLocked() for async-safe scoped locking") | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./include/osx/osd.h: In function ‘pthread_spin_lock’: ./include/osx/osd.h:194:9: warning: implicit declaration of function ‘os_unfair_lock_lock’ [-Wimplicit-function-declaration] 194 | os_unfair_lock_lock(lock); | ^~~~~~~~~~~~~~~~~~~ ./include/osx/osd.h: In function ‘pthread_spin_unlock’: ./include/osx/osd.h:200:9: warning: implicit declaration of function ‘os_unfair_lock_unlock’ [-Wimplicit-function-declaration] 200 | os_unfair_lock_unlock(lock); | ^~~~~~~~~~~~~~~~~~~~~ ./include/osx/osd.h: In function ‘pthread_spin_trylock’: ./include/osx/osd.h:206:16: warning: implicit declaration of function ‘os_unfair_lock_trylock’ [-Wimplicit-function-declaration] 206 | return os_unfair_lock_trylock(lock) ? 0 : EBUSY; | ^~~~~~~~~~~~~~~~~~~~~~ make[3]: *** [util/info.o] Error 1 make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 It looks like errors in macOS’s command line tools. I don’t know how to circumvent the problem. Various text and log files are attached in mpich2.tar.gz. A copy of the lock.h code is also included in the zip file. Thank you, Doug.

1 0

Re: [mpich-discuss] Implicitly remove "mpif.h" support?
by Zhou, Hui 23 Feb '24

23 Feb '24

Just to follow up. It turns out that we just need add -Wl,-flat_namespace in linking libmpifort.so. So all good for now. -- Hui ________________________________ From: Zhou, Hui via discuss <discuss(a)mpich.org> Sent: Thursday, February 22, 2024 1:42 PM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Zhou, Hui <zhouh(a)anl.gov> Subject: [mpich-discuss] Implicitly remove "mpif.h" support? First, are there folks who still maintain or need to work with legacy Fortran code that relies on include 'mpif.h interface? Our CI testing machine for Macos recently upgraded and left us (MPICH development) with some regression failures. The failures are from not recognizing special constants such as MPI_BOTTOM and MPI_IN_PLACE. These constants are implemented using COMMON BLOCK. For those who are not familiar, COMMON BLOCK allows different FORTRAN objects to declare global variables and to be merged at link time. The upgraded dyld on Mac refuse to do that between user code and libmpifort.dylib, resulting in constants such as MPI_IN_PLACE not being recognized. The better way of implementing global variables in Fortran is via the module interface. But what about those legacy codes that can't be updated? Here are the options: 1. Figure out a way to make COMMON BLOCKS on modern Mac work – we need help! 2. Leave it as is - everything will work except legacy Fortran on MacOS. We add a note in the documentation and move on. 3. Suggest users to use mpi instead. But if that suggestion will always work, maybe we can put "use mpi" inside "mpif.h" – assuming that all Fortran compilers are able to support the module interface today even when users don't realize that. The option 3 is what the subject line refers to. Feedback? -- Hui Zhou

1 0

Implicitly remove "mpif.h" support?
by Zhou, Hui 22 Feb '24

22 Feb '24

First, are there folks who still maintain or need to work with legacy Fortran code that relies on include 'mpif.h interface? Our CI testing machine for Macos recently upgraded and left us (MPICH development) with some regression failures. The failures are from not recognizing special constants such as MPI_BOTTOM and MPI_IN_PLACE. These constants are implemented using COMMON BLOCK. For those who are not familiar, COMMON BLOCK allows different FORTRAN objects to declare global variables and to be merged at link time. The upgraded dyld on Mac refuse to do that between user code and libmpifort.dylib, resulting in constants such as MPI_IN_PLACE not being recognized. The better way of implementing global variables in Fortran is via the module interface. But what about those legacy codes that can't be updated? Here are the options: 1. Figure out a way to make COMMON BLOCKS on modern Mac work – we need help! 2. Leave it as is - everything will work except legacy Fortran on MacOS. We add a note in the documentation and move on. 3. Suggest users to use mpi instead. But if that suggestion will always work, maybe we can put "use mpi" inside "mpif.h" – assuming that all Fortran compilers are able to support the module interface today even when users don't realize that. The option 3 is what the subject line refers to. Feedback? -- Hui Zhou

1 0

Re: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2
by Edric Ellis 14 Dec '23

14 Dec '23

I don't think there's any uncompleted communication. The example I show below is one of the very simple examples shipping with MPICH itself (sorry if that wasn't completely clear). Cheers, Edric. ________________________________ From: Joachim Jenke via discuss <discuss(a)mpich.org> Sent: 13 December 2023 17:16 To: discuss(a)mpich.org <discuss(a)mpich.org>; Zhou, Hui <zhouh(a)anl.gov> Cc: Joachim Jenke <jenke(a)itc.rwth-aachen.de> Subject: Re: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 If your code hangs in MPI_Finalize for certain communication implementation, this sounds like an uncompleted communication. Are you sure that you have no MPI communication ongoing when calling MPI_Finalize? - Joachim Am 13.12.23 um 17:45 schrieb Edric Ellis via discuss: > Ok, that's good to know, I'll stick with simply "ofi:tcp" for now. > > Thanks, > Edric. > ------------------------------------------------------------------------ > *From:* Zhou, Hui <zhouh(a)anl.gov> > *Sent:* 13 December 2023 15:39 > *To:* discuss(a)mpich.org <discuss(a)mpich.org> > *Cc:* Edric Ellis <eellis(a)mathworks.com> > *Subject:* Re: Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 > Hi Edric, > > I am not sure which part is hanging, but you don't need to enable > |ofi:shm| (libfabric shm provider). The ch4 device comes with its own > shared memory functionality. > > -- > Hui > ------------------------------------------------------------------------ > *From:* Edric Ellis via discuss <discuss(a)mpich.org> > *Sent:* Wednesday, December 13, 2023 7:05 AM > *To:* discuss(a)mpich.org <discuss(a)mpich.org> > *Cc:* Edric Ellis <eellis(a)mathworks.com> > *Subject:* [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in > mpich-4.1.2 > I'm working on getting a build of mpich-4.1.2 ready to replace our old > build of mpich-3.3.2. With older MPICH releases, we used the "nemesis" > channel via ch3 to provide support for shared-memory configurations as > well as TCP/IP. In ch4, I thought the nearest equivalent would be: > > --with-device=ch4:ofi:tcp,shm > > The "tcp" portion of this seems to work just fine, but "shm" hangs > during (I think) MPI_Finalize, requiring a CTRL-C to kill it. For > example, in the build area, > > $ ./src/pm/hydra/mpiexec.hydra -n 2 ./examples/cpi > Process 0 of 2 is on uk-eellis-l > Process 1 of 2 is on uk-eellis-l > pi is approximately 3.1415926544231318, Error is 0.0000000008333387 > wall clock time = 0.000019 > ^C[mpiexec@uk-eellis-l] Sending Ctrl-C to processes as requested > [mpiexec@uk-eellis-l] Press Ctrl-C again to force abort > > =================================================================================== > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = PID 829015 RUNNING AT uk-eellis-l > = EXIT CODE: 2 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Interrupt (signal 2) > This typically refers to a problem with your application. > Please see the FAQ page for debugging suggestions > > Things work fine if I force FI_PROVIDER=tcp. Am I missing something? > > Here's the configure line I'm using: > > $ ./configure --prefix <prefix> --with-device=ch4:ofi:tcp,shm > --enable-shared --with-libfabric=embedded --enable-fortran --enable-efa=no > > This is running on a Debian 11 system, gcc 10.3.0. > > Cheers, > Edric. > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss> -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de www.itc.rwth-aachen.de<http://www.itc.rwth-aachen.de>

1 0

Re: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2
by Joachim Jenke 13 Dec '23

13 Dec '23

If your code hangs in MPI_Finalize for certain communication implementation, this sounds like an uncompleted communication. Are you sure that you have no MPI communication ongoing when calling MPI_Finalize? - Joachim Am 13.12.23 um 17:45 schrieb Edric Ellis via discuss: > Ok, that's good to know, I'll stick with simply "ofi:tcp" for now. > > Thanks, > Edric. > ------------------------------------------------------------------------ > *From:* Zhou, Hui <zhouh(a)anl.gov> > *Sent:* 13 December 2023 15:39 > *To:* discuss(a)mpich.org <discuss(a)mpich.org> > *Cc:* Edric Ellis <eellis(a)mathworks.com> > *Subject:* Re: Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 > Hi Edric, > > I am not sure which part is hanging, but you don't need to enable > |ofi:shm| (libfabric shm provider). The ch4 device comes with its own > shared memory functionality. > > -- > Hui > ------------------------------------------------------------------------ > *From:* Edric Ellis via discuss <discuss(a)mpich.org> > *Sent:* Wednesday, December 13, 2023 7:05 AM > *To:* discuss(a)mpich.org <discuss(a)mpich.org> > *Cc:* Edric Ellis <eellis(a)mathworks.com> > *Subject:* [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in > mpich-4.1.2 > I'm working on getting a build of mpich-4.1.2 ready to replace our old > build of mpich-3.3.2. With older MPICH releases, we used the "nemesis" > channel via ch3 to provide support for shared-memory configurations as > well as TCP/IP. In ch4, I thought the nearest equivalent would be: > > --with-device=ch4:ofi:tcp,shm > > The "tcp" portion of this seems to work just fine, but "shm" hangs > during (I think) MPI_Finalize, requiring a CTRL-C to kill it. For > example, in the build area, > > $ ./src/pm/hydra/mpiexec.hydra -n 2 ./examples/cpi > Process 0 of 2 is on uk-eellis-l > Process 1 of 2 is on uk-eellis-l > pi is approximately 3.1415926544231318, Error is 0.0000000008333387 > wall clock time = 0.000019 > ^C[mpiexec@uk-eellis-l] Sending Ctrl-C to processes as requested > [mpiexec@uk-eellis-l] Press Ctrl-C again to force abort > > =================================================================================== > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES > = PID 829015 RUNNING AT uk-eellis-l > = EXIT CODE: 2 > = CLEANING UP REMAINING PROCESSES > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES > =================================================================================== > YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Interrupt (signal 2) > This typically refers to a problem with your application. > Please see the FAQ page for debugging suggestions > > Things work fine if I force FI_PROVIDER=tcp. Am I missing something? > > Here's the configure line I'm using: > > $ ./configure --prefix <prefix> --with-device=ch4:ofi:tcp,shm > --enable-shared --with-libfabric=embedded --enable-fortran --enable-efa=no > > This is running on a Debian 11 system, gcc 10.3.0. > > Cheers, > Edric. > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0

Re: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2
by Edric Ellis 13 Dec '23

13 Dec '23

Ok, that's good to know, I'll stick with simply "ofi:tcp" for now. Thanks, Edric. ________________________________ From: Zhou, Hui <zhouh(a)anl.gov> Sent: 13 December 2023 15:39 To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Edric Ellis <eellis(a)mathworks.com> Subject: Re: Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 Hi Edric, I am not sure which part is hanging, but you don't need to enable ofi:shm (libfabric shm provider). The ch4 device comes with its own shared memory functionality. -- Hui ________________________________ From: Edric Ellis via discuss <discuss(a)mpich.org> Sent: Wednesday, December 13, 2023 7:05 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Edric Ellis <eellis(a)mathworks.com> Subject: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 I'm working on getting a build of mpich-4.1.2 ready to replace our old build of mpich-3.3.2. With older MPICH releases, we used the "nemesis" channel via ch3 to provide support for shared-memory configurations as well as TCP/IP. In ch4, I thought the nearest equivalent would be: --with-device=ch4:ofi:tcp,shm The "tcp" portion of this seems to work just fine, but "shm" hangs during (I think) MPI_Finalize, requiring a CTRL-C to kill it. For example, in the build area, $ ./src/pm/hydra/mpiexec.hydra -n 2 ./examples/cpi Process 0 of 2 is on uk-eellis-l Process 1 of 2 is on uk-eellis-l pi is approximately 3.1415926544231318, Error is 0.0000000008333387 wall clock time = 0.000019 ^C[mpiexec@uk-eellis-l] Sending Ctrl-C to processes as requested [mpiexec@uk-eellis-l] Press Ctrl-C again to force abort =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 829015 RUNNING AT uk-eellis-l = EXIT CODE: 2 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Interrupt (signal 2) This typically refers to a problem with your application. Please see the FAQ page for debugging suggestions Things work fine if I force FI_PROVIDER=tcp. Am I missing something? Here's the configure line I'm using: $ ./configure --prefix <prefix> --with-device=ch4:ofi:tcp,shm --enable-shared --with-libfabric=embedded --enable-fortran --enable-efa=no This is running on a Debian 11 system, gcc 10.3.0. Cheers, Edric.

1 0

Re: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2
by Zhou, Hui 13 Dec '23

13 Dec '23

Hi Edric, I am not sure which part is hanging, but you don't need to enable ofi:shm (libfabric shm provider). The ch4 device comes with its own shared memory functionality. -- Hui ________________________________ From: Edric Ellis via discuss <discuss(a)mpich.org> Sent: Wednesday, December 13, 2023 7:05 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Edric Ellis <eellis(a)mathworks.com> Subject: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2 I'm working on getting a build of mpich-4.1.2 ready to replace our old build of mpich-3.3.2. With older MPICH releases, we used the "nemesis" channel via ch3 to provide support for shared-memory configurations as well as TCP/IP. In ch4, I thought the nearest equivalent would be: --with-device=ch4:ofi:tcp,shm The "tcp" portion of this seems to work just fine, but "shm" hangs during (I think) MPI_Finalize, requiring a CTRL-C to kill it. For example, in the build area, $ ./src/pm/hydra/mpiexec.hydra -n 2 ./examples/cpi Process 0 of 2 is on uk-eellis-l Process 1 of 2 is on uk-eellis-l pi is approximately 3.1415926544231318, Error is 0.0000000008333387 wall clock time = 0.000019 ^C[mpiexec@uk-eellis-l] Sending Ctrl-C to processes as requested [mpiexec@uk-eellis-l] Press Ctrl-C again to force abort =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 829015 RUNNING AT uk-eellis-l = EXIT CODE: 2 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Interrupt (signal 2) This typically refers to a problem with your application. Please see the FAQ page for debugging suggestions Things work fine if I force FI_PROVIDER=tcp. Am I missing something? Here's the configure line I'm using: $ ./configure --prefix <prefix> --with-device=ch4:ofi:tcp,shm --enable-shared --with-libfabric=embedded --enable-fortran --enable-efa=no This is running on a Debian 11 system, gcc 10.3.0. Cheers, Edric.

1 0

Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2
by Edric Ellis 13 Dec '23

13 Dec '23

I'm working on getting a build of mpich-4.1.2 ready to replace our old build of mpich-3.3.2. With older MPICH releases, we used the "nemesis" channel via ch3 to provide support for shared-memory configurations as well as TCP/IP. In ch4, I thought the nearest equivalent would be: --with-device=ch4:ofi:tcp,shm The "tcp" portion of this seems to work just fine, but "shm" hangs during (I think) MPI_Finalize, requiring a CTRL-C to kill it. For example, in the build area, $ ./src/pm/hydra/mpiexec.hydra -n 2 ./examples/cpi Process 0 of 2 is on uk-eellis-l Process 1 of 2 is on uk-eellis-l pi is approximately 3.1415926544231318, Error is 0.0000000008333387 wall clock time = 0.000019 ^C[mpiexec@uk-eellis-l] Sending Ctrl-C to processes as requested [mpiexec@uk-eellis-l] Press Ctrl-C again to force abort =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 829015 RUNNING AT uk-eellis-l = EXIT CODE: 2 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Interrupt (signal 2) This typically refers to a problem with your application. Please see the FAQ page for debugging suggestions Things work fine if I force FI_PROVIDER=tcp. Am I missing something? Here's the configure line I'm using: $ ./configure --prefix <prefix> --with-device=ch4:ofi:tcp,shm --enable-shared --with-libfabric=embedded --enable-fortran --enable-efa=no This is running on a Debian 11 system, gcc 10.3.0. Cheers, Edric.

1 0

Re: [mpich-discuss] MPI user-defined datatype size
by Zhou, Hui 04 Dec '23

04 Dec '23

Hi Kurt, MPI_Type_size returns the size of a datatype excluding the gaps (hole). You should use MPI_Type_get_extent to query the extent of the datatype, which should come out as 568. Alternatively, if you never going to break the struct apart during communication, you may try simply treating the struct as an opaque blob, i.e. use MPI_Type_contiguous(sizeof(struct MsgTupleSeq), MPI_BYTE, &newtype). It is likely to perform better by avoiding treating it as a non-contiguous datatype. -- Hui ________________________________ From: Mccall, Kurt E. (MSFC-EV41) via discuss <discuss(a)mpich.org> Sent: Sunday, December 3, 2023 2:55 PM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Mccall, Kurt E. (MSFC-EV41) <kurt.e.mccall(a)nasa.gov> Subject: [mpich-discuss] MPI user-defined datatype size Hi, I’m running MPICH 4.1.2. Should a MPI user-defined datatype created by MPI_Type_create_struct have the same size as the C++ object on which it is based? Comparing sizeof(cpp_obj) and the output From MPI_Type_size(mpi_data_type_, &size1), the C++ object is four bytes larger than the MPI datatype. I don’t know if this is because I have to do something special to take padding into account when I create the MPI user datatype. Here is my C++ message class: struct MsgTupleSeq { static const int N_INDICES_MAX_ = 4; // culprit? int n_vars_; int n_ndxs_; short start_[N_INDICES_MAX_]; short end_[N_INDICES_MAX_]; short current_[N_INDICES_MAX_]; long long n_tuples_; int unique_id_; int subordinate_id_; int opcode_; char error_msg_[512]; } That is 564 bytes for the non-static members, but such an object is 568 bytes long with the 4 bytes of padding that the compiler inserts. I define the MPI user datatype with the code below, using the MsgTupleSeq object to compute the offsets. MPI_Type_size returns 564 bytes for the user datatype, so it seems like it isn’t seeing the padding. MsgTupleSeq obj; MPI_Datatype mpi_data_type; int struct_len = 10, i = 0; int block_len[struct_len]; MPI_Datatype types[struct_len]; MPI_Aint displacements[struct_len]; // the integer "n_vars_" member block_len[i] = 1; types[i] = MPI_INT; displacements[i] = (size_t) &obj.n_vars_ - (size_t) &obj; // the integer "n_ndxs_" member ++i; block_len[i] = 1; types[i] = MPI_INT; displacements[i] = (size_t) &obj.n_ndxs_ - (size_t) &obj; // the short integer "start_" array member ++i; block_len[i] = N_INDICES_MAX_; types[i] = MPI_SHORT; displacements[i] = (size_t) &obj.start_ - (size_t) &obj; // the short integer "end_" array member ++i; block_len[i] = N_INDICES_MAX_; types[i] = MPI_SHORT; displacements[i] = (size_t) &obj.end_ - (size_t) &obj; // the short integer "current_" array member ++i; block_len[i] = N_INDICES_MAX_; types[i] = MPI_SHORT; displacements[i] = (size_t) &obj.current_ - (size_t) &obj; // the long long integer "n_tuples_" member ++i; block_len[i] = 1; types[i] = MPI_LONG_LONG; displacements[i] = (size_t) &obj.n_tuples_ - (size_t) &obj; // the integer "unique_id_" member ++i; block_len[i] = 1; types[i] = MPI_INT; displacements[i] = (size_t) &obj.unique_id_ - (size_t) &obj; // the integer "subordinate_id_" member ++i; block_len[i] = 1; types[i] = MPI_INT; displacements[i] = (size_t) &obj.subordinate_id_ - (size_t) &obj; // the integer "opcode_" member ++i; block_len[i] = 1; types[i] = MPI_INT; displacements[i] = (size_t) &obj.opcode_ - (size_t) &obj; // the char array "error_msg_" member ++i; block_len[i] = 512; types[i] = MPI_CHAR; displacements[i] = (size_t) &obj.error_msg_[0] - (size_t) &obj; MPI_Type_create_struct(struct_len, block_len, displacements, types, &mpi_data_type); MPI_Type_commit(&mpi_data_type);

1 0