January 2022 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Deprecated C++ bindings
by Raffenetti, Ken 24 Jan '22

24 Jan '22

Checking return codes will work, but another option that you might want to try is to install a custom error handler function. https://www.mpich.org/static/docs/v3.4.x/www3/MPI_Comm_create_errhandler.ht… Your error handler callback function will then be invoked by MPI at the detection of an error, before returning back to the application. Ken On 1/24/22, 1:56 PM, "Mccall, Kurt E. (MSFC-EV41) via discuss" <discuss(a)mpich.org> wrote: Hi, Up to now I’ve been using this in my C++ code: MPI_Comm_set_errhandler(intercom, MPI::ERRORS_THROW_EXCEPTIONS); and hoping the MPI function calls would throw MPI::Exception when there is a problem. Is the recommended approach now just to test the return values from the functions? Using 4.0rc3. Thanks, Kurt

1 0

Deprecated C++ bindings
by Mccall, Kurt E. (MSFC-EV41) 24 Jan '22

24 Jan '22

Hi, Up to now I've been using this in my C++ code: MPI_Comm_set_errhandler(intercom, MPI::ERRORS_THROW_EXCEPTIONS); and hoping the MPI function calls would throw MPI::Exception when there is a problem. Is the recommended approach now just to test the return values from the functions? Using 4.0rc3. Thanks, Kurt

1 0

ROMIO and asynchronous MPI-IO
by Ahmad Tarraf 24 Jan '22

24 Jan '22

Dear ROMIO developers, It is hard to find any literature or documentation about the asynchronous implementation ROMIO (or ADIO) use. I have few questions and It would be great if you can answer them: 1. How does ROMIO implement asynchronous I/O? I mean do you use POSIX-AIO, native Linux kernel AIO (io_submit,..), io_uring, or anything else? If I debug my code at our Clusters with gdb, I can see that POSIX-AIO is used. But is this on all systems so? 2. When and how does asynchronous I/O progress occur? Do you span multiple I/O threads for a rank? Is this related to variable MPICH_ASYNC_PROGRESS? Again with gdb, I can see that several asynchronous write requests by the same MPI rank are handled successively by a single thread. 3. Is there any related recent publication is can use to cite? I didn't find anything related to the implementation of asynchronous MPI-IO, except this: https://web.cels.anl.gov/~thakur/papers/grequest-redesign.pdf. But is it not outdated? Best regards, Ahmad

1 0

Re: [mpich-discuss] MPI_Cancel + MPI_Wait stalls when using ch4, not when using ch3
by Zhou, Hui 19 Jan '22

19 Jan '22

Thanks for reporting. We'll fix this. Tracking issue: https://github.com/pmodels/mpich/issues/5775 [https://opengraph.githubassets.com/abeb43664f034115c44e99f3e273e49dbfdeb9eb…]<https://github.com/pmodels/mpich/issues/5775> ch4: MPI_Cancel stalls MPI_Wait · Issue #5775 · pmodels/mpich<https://github.com/pmodels/mpich/issues/5775> Reported on discuss mailinglist: ---- quote ---- Hi, Running one of our test programs using MPICH 3.4.3 and ch4:ofi, I notice that MPI_Wait on an MPI_Request that has been MPI_Cancelled never compl... github.com ________________________________ From: Edric Ellis via discuss <discuss(a)mpich.org> Sent: Wednesday, January 19, 2022 11:21 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Edric Ellis <eellis(a)mathworks.com> Subject: [mpich-discuss] MPI_Cancel + MPI_Wait stalls when using ch4, not when using ch3 Hi, Running one of our test programs using MPICH 3.4.3 and ch4:ofi, I notice that MPI_Wait on an MPI_Request that has been MPI_Cancelled never completes (it does when using ch3). (The documentation for MPI_Cancel states "If a communication is marked for cancellation, then a MPI_WAIT call for that communication is guaranteed to return, irrespective of the activities of other processes (i.e., MPI_WAIT behaves as a local function)") Here's a simple example: #include <mpi.h> #include <stdio.h> #include <stdlib.h> void check(int const value) { if (value != MPI_SUCCESS) { fprintf(stderr, "Failed.\n"); exit(1); } } int main(int argc, char** argv) { MPI_Request r1; int payload = 42; int result; check(MPI_Init(0,0)); check(MPI_Issend(&payload, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, &r1)); check(MPI_Test(&r1, &result, MPI_STATUS_IGNORE)); fprintf(stdout, "MPI_Test result: %d\n", result); check(MPI_Cancel(&r1)); check(MPI_Wait(&r1, MPI_STATUS_IGNORE)); MPI_Finalize(); return 0; } This stalls in MPI_Wait when executed using "mpiexec -n 1 ./a.out". Cheers, Edric. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPI_Cancel + MPI_Wait stalls when using ch4, not when using ch3
by Edric Ellis 19 Jan '22

19 Jan '22

Hi, Running one of our test programs using MPICH 3.4.3 and ch4:ofi, I notice that MPI_Wait on an MPI_Request that has been MPI_Cancelled never completes (it does when using ch3). (The documentation for MPI_Cancel states "If a communication is marked for cancellation, then a MPI_WAIT call for that communication is guaranteed to return, irrespective of the activities of other processes (i.e., MPI_WAIT behaves as a local function)") Here's a simple example: #include <mpi.h> #include <stdio.h> #include <stdlib.h> void check(int const value) { if (value != MPI_SUCCESS) { fprintf(stderr, "Failed.\n"); exit(1); } } int main(int argc, char** argv) { MPI_Request r1; int payload = 42; int result; check(MPI_Init(0,0)); check(MPI_Issend(&payload, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, &r1)); check(MPI_Test(&r1, &result, MPI_STATUS_IGNORE)); fprintf(stdout, "MPI_Test result: %d\n", result); check(MPI_Cancel(&r1)); check(MPI_Wait(&r1, MPI_STATUS_IGNORE)); MPI_Finalize(); return 0; } This stalls in MPI_Wait when executed using "mpiexec -n 1 ./a.out". Cheers, Edric.

1 0

Re: [mpich-discuss] [EXTERNAL] Re: Help building MPICH as shared library with NAG Fortran and Clang
by Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] 18 Jan '22

18 Jan '22

Hui, Yeah. That is my current thought. Now, I do have libtool installed via Brew and maybe that's the issue, but try as I may by moving around that file/symlink, I can't seem to avoid this...or maybe autotools is just *really* good at finding it no matter what I do?[1] I had hoped maybe the Spack folks had a workaround, but the only nag+mpich patches they have: https://github.com/spack/spack/tree/develop/var/spack/repos/builtin/package… seem to either be in mpich 4.0rc3 or not related to this! Matt [1] I thought about `brew uninstall libtool` but that uninstalls a *lot* of other packages. And a brew-less environment sort of breaks my fancy terminal environment! :) -- Matt Thompson, SSAI, Ld Scientific Programmer/Analyst NASA GSFC, Global Modeling and Assimilation Office Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 Phone: 301-614-6712 Fax: 301-614-6246 http://science.gsfc.nasa.gov/sed/bio/matthew.thompson From: "Zhou, Hui" <zhouh(a)anl.gov> Date: Tuesday, January 18, 2022 at 2:16 PM To: "discuss(a)mpich.org" <discuss(a)mpich.org> Cc: "Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]" <matthew.thompson(a)nasa.gov> Subject: [EXTERNAL] Re: Help building MPICH as shared library with NAG Fortran and Clang Hi Matt, I suspect this is an issue in libtool. We'll take a look at it. We routinely test NAG compiler, but I guess we never tried clang+NAG. Once we figure out the issue, we typically will maintain custom patches to libtool until libtool fixes it upstream. -- Hui Zhou ________________________________ From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Sent: Tuesday, January 18, 2022 12:49 PM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: [mpich-discuss] Help building MPICH as shared library with NAG Fortran and Clang All, As the subject says, I'm having issues building MPICH as a shared library with NAG Fortran and Clang on my MacBook. I configured mpich as: ../configure FCFLAGS="-mismatch_all -fpp" FFLAGS="-mismatch_all -fpp" --prefix=$HOME/installed/Compiler/nag-7.0_7066/mpich/4.0rc3 |& tee configure.nag-7.0_7066.log where CC=clang, CXX=clang++, and FC=nagfor (set in environment). When I do this, the configure step is fine, but upon build: make[2]: Entering directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' GEN lib/libmpifort.la NAG Fortran Compiler Release 7.0(Yurakucho) Build 7066 Option error: Unrecognised option -dynamiclib make[2]: *** [Makefile:7606: lib/libmpifort.la] Error 2 make[2]: Leaving directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' make[1]: *** [Makefile:30237: all-recursive] Error 1 make[1]: Leaving directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' make: *** [Makefile:7300: all] Error 2 The issue seems to be that MPICH (or libtool?) assumes that flags that clang accepts will be accepted by the Fortran compiler as well. But NAG doesn't accept '-dynamiclib' as an option. (I am asking NAG about this as well.) I can kind of see places where this is happening. For example in config.status: 749:archive_cmds_FC='$CC -dynamiclib $allow_undefined_flag -o $lib $libobjs $deplibs $compiler_flags -install_name $rpath/$soname $verstring $single_module' Now, the interesting thing is this *exact same* issue happened to me with Open MPI: https://www.mail-archive.com/[email protected]//msg34635.html<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-…> On that thread, I eventually figured out how to build Open MPI as static, but now I ran into this: https://discourse.cmake.org/t/cmake-and-mpi-fortran-cmake-doesnt-see-it/4869 My fear is that if I try and build MPICH as a static library, the same thing will happen. Does anyone at MPICH have any thoughts on this? Thanks, Matt -- Matt Thompson, SSAI, Ld Scientific Programmer/Analyst NASA GSFC, Global Modeling and Assimilation Office Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 Phone: 301-614-6712 Fax: 301-614-6246 http://science.gsfc.nasa.gov/sed/bio/matthew.thompson

1 0

Re: [mpich-discuss] Help building MPICH as shared library with NAG Fortran and Clang
by Zhou, Hui 18 Jan '22

18 Jan '22

Hi Matt, I suspect this is an issue in libtool. We'll take a look at it. We routinely test NAG compiler, but I guess we never tried clang+NAG. Once we figure out the issue, we typically will maintain custom patches to libtool until libtool fixes it upstream. -- Hui Zhou ________________________________ From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Sent: Tuesday, January 18, 2022 12:49 PM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: [mpich-discuss] Help building MPICH as shared library with NAG Fortran and Clang All, As the subject says, I'm having issues building MPICH as a shared library with NAG Fortran and Clang on my MacBook. I configured mpich as: ../configure FCFLAGS="-mismatch_all -fpp" FFLAGS="-mismatch_all -fpp" --prefix=$HOME/installed/Compiler/nag-7.0_7066/mpich/4.0rc3 |& tee configure.nag-7.0_7066.log where CC=clang, CXX=clang++, and FC=nagfor (set in environment). When I do this, the configure step is fine, but upon build: make[2]: Entering directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' GEN lib/libmpifort.la NAG Fortran Compiler Release 7.0(Yurakucho) Build 7066 Option error: Unrecognised option -dynamiclib make[2]: *** [Makefile:7606: lib/libmpifort.la] Error 2 make[2]: Leaving directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' make[1]: *** [Makefile:30237: all-recursive] Error 1 make[1]: Leaving directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' make: *** [Makefile:7300: all] Error 2 The issue seems to be that MPICH (or libtool?) assumes that flags that clang accepts will be accepted by the Fortran compiler as well. But NAG doesn't accept '-dynamiclib' as an option. (I am asking NAG about this as well.) I can kind of see places where this is happening. For example in config.status: 749:archive_cmds_FC='$CC -dynamiclib $allow_undefined_flag -o $lib $libobjs $deplibs $compiler_flags -install_name $rpath/$soname $verstring $single_module' Now, the interesting thing is this *exact same* issue happened to me with Open MPI: https://www.mail-archive.com/[email protected]//msg34635.html<https://www.mail-archive.com/[email protected]/msg34635.html> On that thread, I eventually figured out how to build Open MPI as static, but now I ran into this: https://discourse.cmake.org/t/cmake-and-mpi-fortran-cmake-doesnt-see-it/4869 My fear is that if I try and build MPICH as a static library, the same thing will happen. Does anyone at MPICH have any thoughts on this? Thanks, Matt -- Matt Thompson, SSAI, Ld Scientific Programmer/Analyst NASA GSFC, Global Modeling and Assimilation Office Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 Phone: 301-614-6712 Fax: 301-614-6246 http://science.gsfc.nasa.gov/sed/bio/matthew.thompson

1 0

Help building MPICH as shared library with NAG Fortran and Clang
by Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] 18 Jan '22

18 Jan '22

All, As the subject says, I'm having issues building MPICH as a shared library with NAG Fortran and Clang on my MacBook. I configured mpich as: ../configure FCFLAGS="-mismatch_all -fpp" FFLAGS="-mismatch_all -fpp" --prefix=$HOME/installed/Compiler/nag-7.0_7066/mpich/4.0rc3 |& tee configure.nag-7.0_7066.log where CC=clang, CXX=clang++, and FC=nagfor (set in environment). When I do this, the configure step is fine, but upon build: make[2]: Entering directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' GEN lib/libmpifort.la NAG Fortran Compiler Release 7.0(Yurakucho) Build 7066 Option error: Unrecognised option -dynamiclib make[2]: *** [Makefile:7606: lib/libmpifort.la] Error 2 make[2]: Leaving directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' make[1]: *** [Makefile:30237: all-recursive] Error 1 make[1]: Leaving directory '/Users/mathomp4/src/MPI/mpich-4.0rc3/build-nag-7.0_7066' make: *** [Makefile:7300: all] Error 2 The issue seems to be that MPICH (or libtool?) assumes that flags that clang accepts will be accepted by the Fortran compiler as well. But NAG doesn't accept '-dynamiclib' as an option. (I am asking NAG about this as well.) I can kind of see places where this is happening. For example in config.status: 749:archive_cmds_FC='$CC -dynamiclib $allow_undefined_flag -o $lib $libobjs $deplibs $compiler_flags -install_name $rpath/$soname $verstring $single_module' Now, the interesting thing is this *exact same* issue happened to me with Open MPI: https://www.mail-archive.com/[email protected]//msg34635.html<https://www.mail-archive.com/[email protected]/msg34635.html> On that thread, I eventually figured out how to build Open MPI as static, but now I ran into this: https://discourse.cmake.org/t/cmake-and-mpi-fortran-cmake-doesnt-see-it/4869 My fear is that if I try and build MPICH as a static library, the same thing will happen. Does anyone at MPICH have any thoughts on this? Thanks, Matt -- Matt Thompson, SSAI, Ld Scientific Programmer/Analyst NASA GSFC, Global Modeling and Assimilation Office Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 Phone: 301-614-6712 Fax: 301-614-6246 http://science.gsfc.nasa.gov/sed/bio/matthew.thompson

1 0

Re: [mpich-discuss] MPI_Comm_spawn and strace
by Zhou, Hui 18 Jan '22

18 Jan '22

Hi Kurt, In this case, "strace" should be your command, and "<my_executable>" should be the first in argv. -- Hui ________________________________ From: Mccall, Kurt E. (MSFC-EV41) via discuss <discuss(a)mpich.org> Sent: Tuesday, January 18, 2022 11:28 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Mccall, Kurt E. (MSFC-EV41) <kurt.e.mccall(a)nasa.gov> Subject: [mpich-discuss] MPI_Comm_spawn and strace Hi, I need to run each of my processes created using MPI_Comm_spawn under strace, so that I can see all system calls that they make. Is this possible? I’ve made a few naïve attempts, like passing “strace <my_executable>” to MPI_Comm_spawn, but those attempts didn’t work. Thanks, Kurt

1 0

MPI_Comm_spawn and strace
by Mccall, Kurt E. (MSFC-EV41) 18 Jan '22

18 Jan '22

Hi, I need to run each of my processes created using MPI_Comm_spawn under strace, so that I can see all system calls that they make. Is this possible? I've made a few naïve attempts, like passing "strace <my_executable>" to MPI_Comm_spawn, but those attempts didn't work. Thanks, Kurt

1 0