- discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Assertion Failure
by Zhou, Hui 19 Jun '25

19 Jun '25

Hi Chris, Can you try to dump the value of *comm at the assertion? I suspect the value is corrupted and it results in a wrong comm handle passed to MPI_Comm_free. Hui ________________________________ From: Chris Hewson via discuss <discuss(a)mpich.org> Sent: Wednesday, June 18, 2025 1:41 PM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Chris Hewson <chris(a)resfrac.com> Subject: [mpich-discuss] Assertion Failure Hi All, I am having a consistent issue when using mpich with petsc. When calling another external library (mkl's pardiso), I will get the following assertion failure: Assertion failed in file src/binding/c/c_binding. c at line 29448: ((*comm)&(0x03ffffff)) ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Hi All, I am having a consistent issue when using mpich with petsc. When calling another external library (mkl's pardiso), I will get the following assertion failure: Assertion failed in file src/binding/c/c_binding.c at line 29448: ((*comm)&(0x03ffffff)) < MPIR_COMM_N_BUILTIN I'm not really too sure why, if I turn assertions off in mpich, the program will seg fault later on. Stack trace for this looks like this: Assertion failed in file src/binding/c/c_binding.c at line 29448: ((*comm)&(0x03ffffff)) < MPIR_COMM_N_BUILTIN resfrac73bfaea(+0x1e992e0) [0x55b9357d52e0] resfrac73bfaea(+0x1e07ca8) [0x55b935743ca8] resfrac73bfaea(PMPI_Comm_free+0x580) [0x55b935642570] resfrac73bfaea(MKLMPI_Comm_free+0x38) [0x55b9385830f8] resfrac73bfaea(mkl_pds_lp64_reduce_rhs_real+0x4aa) [0x55b938edc2aa] resfrac73bfaea(mkl_pds_lp64_slv_omp_real+0x1c46) [0x55b93869a4c6] resfrac73bfaea(mkl_pds_lp64_solve_slave+0xd84) [0x55b935ba8514] resfrac73bfaea(mkl_pds_lp64_cluster_sparse_solver+0xec4) [0x55b935ba4b74] resfrac73bfaea(+0xdef3de) [0x55b93472b3de] resfrac73bfaea(MatSolve+0x521) [0x55b9349f17ef] resfrac73bfaea(+0x1a29db7) [0x55b935365db7] resfrac73bfaea(PCApply+0x73c) [0x55b934c53065] resfrac73bfaea(+0x19ee12d) [0x55b93532a12d] resfrac73bfaea(+0x115a74c) [0x55b934a9674c] resfrac73bfaea(KSPSolve+0x1b) [0x55b934a98d04] resfrac73bfaea(_ZN11nextstim_ns18PetscSolveParallelERNS_17SystemOfEquationsE+0x28b7) [0x55b934046767] resfrac73bfaea(_ZN11nextstim_ns21SlavePoolWaitAndSolveEv+0x5c7) [0x55b9341d5037] resfrac73bfaea(main+0x5db) [0x55b933eb292b] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f14c99d5d90] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f14c99d5e40] resfrac73bfaea(_start+0x25) [0x55b933ecb0d5] The mpich configuration and version that we're using: MPICH Version: 4.3.0 MPICH Release date: Mon Feb 3 09:09:47 AM CST 2025 MPICH ABI: 17:0:5 MPICH Device: ch3:sock MPICH configure: --prefix=/opt/anl/petsc-3.23.2 MAKE=/usr/bin/make --libdir=/opt/anl/petsc-3.23.2/lib CC=gcc CFLAGS=-fPIC -Wno-lto-type-mismatch -Wno-stringop-overflow -O0 AR=/usr/bin/ar ARFLAGS=cr CXX=g++ CXXFLAGS=-Wno-lto-type-mismatch -Wno-psabi -O0 -std=gnu++20 FFLAGS=-ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -O0 -fallow-argument-mismatch FC=gfortran F77=gfortran FCFLAGS=-ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -O0 -fallow-argument-mismatch --disable-shared --with-pm=hydra --disable-java --with-hwloc=embedded --enable-fast=no --enable-error-messages=all --with-device=ch3:sock --enable-g=meminit,dbg PYTHON=/opt/intel/oneapi/intelpython/python3.12/bin/python3 --disable-maintainer-mode --disable-dependency-tracking MPICH CC: gcc -fPIC -Wno-lto-type-mismatch -Wno-stringop-overflow -O0 -g -O0 MPICH CXX: g++ -Wno-lto-type-mismatch -Wno-psabi -O0 -std=gnu++20 -g -O0 MPICH F77: gfortran -ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -O0 -fallow-argument-mismatch -g -O0 MPICH FC: gfortran -ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -O0 -fallow-argument-mismatch -g -O0 Any information or help on this would be greatly appreciated. Chris Hewson Senior Reservoir Simulation Engineer ResFrac +1.587.575.9792

1 0

Assertion Failure
by Chris Hewson 18 Jun '25

18 Jun '25

Hi All, I am having a consistent issue when using mpich with petsc. When calling another external library (mkl's pardiso), I will get the following assertion failure: Assertion failed in file src/binding/c/c_binding.c at line 29448: ((*comm)&(0x03ffffff)) < MPIR_COMM_N_BUILTIN I'm not really too sure why, if I turn assertions off in mpich, the program will seg fault later on. Stack trace for this looks like this: Assertion failed in file src/binding/c/c_binding.c at line 29448: ((*comm)&(0x03ffffff)) < MPIR_COMM_N_BUILTIN resfrac73bfaea(+0x1e992e0) [0x55b9357d52e0] resfrac73bfaea(+0x1e07ca8) [0x55b935743ca8] resfrac73bfaea(PMPI_Comm_free+0x580) [0x55b935642570] resfrac73bfaea(MKLMPI_Comm_free+0x38) [0x55b9385830f8] resfrac73bfaea(mkl_pds_lp64_reduce_rhs_real+0x4aa) [0x55b938edc2aa] resfrac73bfaea(mkl_pds_lp64_slv_omp_real+0x1c46) [0x55b93869a4c6] resfrac73bfaea(mkl_pds_lp64_solve_slave+0xd84) [0x55b935ba8514] resfrac73bfaea(mkl_pds_lp64_cluster_sparse_solver+0xec4) [0x55b935ba4b74] resfrac73bfaea(+0xdef3de) [0x55b93472b3de] resfrac73bfaea(MatSolve+0x521) [0x55b9349f17ef] resfrac73bfaea(+0x1a29db7) [0x55b935365db7] resfrac73bfaea(PCApply+0x73c) [0x55b934c53065] resfrac73bfaea(+0x19ee12d) [0x55b93532a12d] resfrac73bfaea(+0x115a74c) [0x55b934a9674c] resfrac73bfaea(KSPSolve+0x1b) [0x55b934a98d04] resfrac73bfaea(_ZN11nextstim_ns18PetscSolveParallelERNS_17SystemOfEquationsE+0x28b7) [0x55b934046767] resfrac73bfaea(_ZN11nextstim_ns21SlavePoolWaitAndSolveEv+0x5c7) [0x55b9341d5037] resfrac73bfaea(main+0x5db) [0x55b933eb292b] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f14c99d5d90] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f14c99d5e40] resfrac73bfaea(_start+0x25) [0x55b933ecb0d5] The mpich configuration and version that we're using: MPICH Version: 4.3.0 MPICH Release date: Mon Feb 3 09:09:47 AM CST 2025 MPICH ABI: 17:0:5 MPICH Device: ch3:sock MPICH configure: --prefix=/opt/anl/petsc-3.23.2 MAKE=/usr/bin/make --libdir=/opt/anl/petsc-3.23.2/lib CC=gcc CFLAGS=-fPIC -Wno-lto-type-mismatch -Wno-stringop-overflow -O0 AR=/usr/bin/ar ARFLAGS=cr CXX=g++ CXXFLAGS=-Wno-lto-type-mismatch -Wno-psabi -O0 -std=gnu++20 FFLAGS=-ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -O0 -fallow-argument-mismatch FC=gfortran F77=gfortran FCFLAGS=-ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -O0 -fallow-argument-mismatch --disable-shared --with-pm=hydra --disable-java --with-hwloc=embedded --enable-fast=no --enable-error-messages=all --with-device=ch3:sock --enable-g=meminit,dbg PYTHON=/opt/intel/oneapi/intelpython/python3.12/bin/python3 --disable-maintainer-mode --disable-dependency-tracking MPICH CC: gcc -fPIC -Wno-lto-type-mismatch -Wno-stringop-overflow -O0 -g -O0 MPICH CXX: g++ -Wno-lto-type-mismatch -Wno-psabi -O0 -std=gnu++20 -g -O0 MPICH F77: gfortran -ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -O0 -fallow-argument-mismatch -g -O0 MPICH FC: gfortran -ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -O0 -fallow-argument-mismatch -g -O0 Any information or help on this would be greatly appreciated. *Chris Hewson* Senior Reservoir Simulation Engineer ResFrac +1.587.575.9792

1 0

Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests?
by Latham, Robert J. 13 Jun '25

13 Jun '25

Ok, this fix turned out to be easier once I tried things for myself: the prefix should be "quobyte:" (not "quobytefs:"). ==rob ________________________________________ From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Sent: Wednesday, June 11, 2025 10:25 To: discuss(a)mpich.org Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] Subject: Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests? An addendum, it's possible my "bugfix" as detailed in https: //github. com/pmodels/mpich/issues/7452 might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd An addendum, it's possible my "bugfix" as detailed in https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… <https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…> might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: export ROMIO_PRINT_HINTS=1 and then in my code I added: call MPI_INFO_SET(info,"romio_filesystem_type","QUOBYTEFS:", status) but, well: key = romio_filesystem_type value = UFS: Generic ROMIO driver for all UNIX-like file systems I also tried it in lowercase as well. No joy. I've also tried: export ROMIO_FSTYPE_FORCE="QUOBYTEFS:" and: export ROMIO_FSTYPE_FORCE="quobytefs:" but in each case, it's still UFS. I do know my Infos are being seen by MPICH because if I toggled: call MPI_INFO_SET(info,"romio_cb_write","enable", status) from enable to disable, the ROMIO_PRINT_HINTS show the change. Matt [signature_3627681122]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!aqjYcygb…> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]> From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Date: Wednesday, June 11, 2025 at 9:37 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: [EXTERNAL] [BULK] [mpich-discuss] How to make/run romio tests? All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https: //www. quobyte. com/blog/quobyte-mpi-io-support-with-kernel-bypass-now-available/ ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp… <https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp…> and, yep, quobyte seems to be in MPICH! I grabbed MPICH 4.3.0 and built it (with one bug fix[1]) and well, it seems to run Hello World. So, following that Quobyte page, I get to Step 10 and tried the romio tests and...I am apparently dumb. I went to my build/src/mpi/romio/test directory and I see 'runtests' there, but: > ./runtests make: *** No rule to make target 'default'. Stop. I also tried with -fname and: > ./runtests -fname=quobyte:test_mpi_file ./runtests: line 90: [quobyte:test_mpi_file: command not found make: *** No rule to make target 'default'. Stop. > ./runtests -fname=quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file ./runtests: line 90: [quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file: No such file or directory make: *** No rule to make target 'default'. Stop. Yeah. I'm stumped. I'm guessing I've missed some simple step along the way. NOTE: I have no idea even how to use romio really, but I was hoping the tests could, well, help me figure that out. Thanks for any help, Matt [1] See https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… <https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…> [signature_3314915376]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!foSdMay5…> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]>

1 0

Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests?
by Latham, Robert J. 11 Jun '25

11 Jun '25

OK well yeah that looks right.... I guess you'll have to send me (just me, not the list) the whole config.log. ==rob ________________________________________ From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Sent: Wednesday, June 11, 2025 13:43 To: Latham, Robert J.; discuss(a)mpich.org Subject: Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests? Rob, Here is the head of that file: > head src/mpi/romio/config. log This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by ROMIO configure 4. 3. 0, which ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Rob, Here is the head of that file: > head src/mpi/romio/config.log This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by ROMIO configure 4.3.0, which was generated by GNU Autoconf 2.69. Invocation command line was $ ../../../../src/mpi/romio/configure --disable-option-checking --prefix=/home/mathomp4/MPI/mpich/4.3.0/ifort-2021.13 CC=icx CXX=icpx FC=ifort --with-file-system=ufs+nfs+quobytefs --cache-file=/dev/null --srcdir=../../../../src/mpi/romio ## --------- ## ## Platform. ## [signature_4121572227]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!f15j764o…> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]> From: Latham, Robert J. <robl(a)mcs.anl.gov> Date: Wednesday, June 11, 2025 at 2:26 PM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests? We could do a better job documenting the "read-only" vs "read-write" nature of these hints... you cannot override the file system with the "romio_filesystem_type" hint, only confirm the file system is what you thought it was. you should be able to with the ROMIO_FSTYPE_FORCE environment variable. These symptoms suggest you requested quobyte support but ROMIO's configure could not find the development headers.... except ROMIO doesn't check for any development headers. Can you share with me the first few lines of ROMIO's config.log? From your build directory, that would be ` head mpich/src/mpi/romio/config.log` (the whole config.log is too much... for now) ==rob ________________________________________ From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Sent: Wednesday, June 11, 2025 10:25 To: discuss(a)mpich.org Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] Subject: Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests? An addendum, it's possible my "bugfix" as detailed in https: //github. com/pmodels/mpich/issues/7452<https://urldefense.us/v3/__https://**B/github.**Bcom/pmodels/mpich/issues/7…> might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd An addendum, it's possible my "bugfix" as detailed in https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… <https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…><https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…> might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: export ROMIO_PRINT_HINTS=1 and then in my code I added: call MPI_INFO_SET(info,"romio_filesystem_type","QUOBYTEFS:", status) but, well: key = romio_filesystem_type value = UFS: Generic ROMIO driver for all UNIX-like file systems I also tried it in lowercase as well. No joy. I've also tried: export ROMIO_FSTYPE_FORCE="QUOBYTEFS:" and: export ROMIO_FSTYPE_FORCE="quobytefs:" but in each case, it's still UFS. I do know my Infos are being seen by MPICH because if I toggled: call MPI_INFO_SET(info,"romio_cb_write","enable", status) from enable to disable, the ROMIO_PRINT_HINTS show the change. Matt [signature_3627681122]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!aqjYcygb…<https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?…>> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]> From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Date: Wednesday, June 11, 2025 at 9:37 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: [EXTERNAL] [BULK] [mpich-discuss] How to make/run romio tests? All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https: //www. quobyte. com/blog/quobyte-mpi-io-support-with-kernel-bypass-now-available/<https://urldefense.us/v3/__https://**B/www.**Bquobyte.**Bcom/blog/quobyte-m…> ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp… <https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp…><https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp…> and, yep, quobyte seems to be in MPICH! I grabbed MPICH 4.3.0 and built it (with one bug fix[1]) and well, it seems to run Hello World. So, following that Quobyte page, I get to Step 10 and tried the romio tests and...I am apparently dumb. I went to my build/src/mpi/romio/test directory and I see 'runtests' there, but: > ./runtests make: *** No rule to make target 'default'. Stop. I also tried with -fname and: > ./runtests -fname=quobyte:test_mpi_file ./runtests: line 90: [quobyte:test_mpi_file: command not found make: *** No rule to make target 'default'. Stop. > ./runtests -fname=quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file ./runtests: line 90: [quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file: No such file or directory make: *** No rule to make target 'default'. Stop. Yeah. I'm stumped. I'm guessing I've missed some simple step along the way. NOTE: I have no idea even how to use romio really, but I was hoping the tests could, well, help me figure that out. Thanks for any help, Matt [1] See https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… <https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…><https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…> [signature_3314915376]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!foSdMay5…<https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?…>> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]>

1 0

Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests?
by Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] 11 Jun '25

11 Jun '25

Rob, Here is the head of that file: > head src/mpi/romio/config.log This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by ROMIO configure 4.3.0, which was generated by GNU Autoconf 2.69. Invocation command line was $ ../../../../src/mpi/romio/configure --disable-option-checking --prefix=/home/mathomp4/MPI/mpich/4.3.0/ifort-2021.13 CC=icx CXX=icpx FC=ifort --with-file-system=ufs+nfs+quobytefs --cache-file=/dev/null --srcdir=../../../../src/mpi/romio ## --------- ## ## Platform. ## [signature_4121572227]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!fMuLGLjh… > Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]> From: Latham, Robert J. <robl(a)mcs.anl.gov> Date: Wednesday, June 11, 2025 at 2:26 PM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests? We could do a better job documenting the "read-only" vs "read-write" nature of these hints... you cannot override the file system with the "romio_filesystem_type" hint, only confirm the file system is what you thought it was. you should be able to with the ROMIO_FSTYPE_FORCE environment variable. These symptoms suggest you requested quobyte support but ROMIO's configure could not find the development headers.... except ROMIO doesn't check for any development headers. Can you share with me the first few lines of ROMIO's config.log? From your build directory, that would be ` head mpich/src/mpi/romio/config.log` (the whole config.log is too much... for now) ==rob ________________________________________ From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Sent: Wednesday, June 11, 2025 10:25 To: discuss(a)mpich.org Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] Subject: Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests? An addendum, it's possible my "bugfix" as detailed in https: //github. com/pmodels/mpich/issues/7452<https:// /github. com/pmodels/mpich/issues/7452> might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd An addendum, it's possible my "bugfix" as detailed in https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… <https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… ><https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… > might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: export ROMIO_PRINT_HINTS=1 and then in my code I added: call MPI_INFO_SET(info,"romio_filesystem_type","QUOBYTEFS:", status) but, well: key = romio_filesystem_type value = UFS: Generic ROMIO driver for all UNIX-like file systems I also tried it in lowercase as well. No joy. I've also tried: export ROMIO_FSTYPE_FORCE="QUOBYTEFS:" and: export ROMIO_FSTYPE_FORCE="quobytefs:" but in each case, it's still UFS. I do know my Infos are being seen by MPICH because if I toggled: call MPI_INFO_SET(info,"romio_cb_write","enable", status) from enable to disable, the ROMIO_PRINT_HINTS show the change. Matt [signature_3627681122]<https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… > Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]> From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Date: Wednesday, June 11, 2025 at 9:37 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: [EXTERNAL] [BULK] [mpich-discuss] How to make/run romio tests? All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https: //www. quobyte. com/blog/quobyte-mpi-io-support-with-kernel-bypass-now-available/<https:// /www. quobyte. com/blog/quobyte-mpi-io-support-with-kernel-bypass-now-available/> ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… <https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… ><https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp… > and, yep, quobyte seems to be in MPICH! I grabbed MPICH 4.3.0 and built it (with one bug fix[1]) and well, it seems to run Hello World. So, following that Quobyte page, I get to Step 10 and tried the romio tests and...I am apparently dumb. I went to my build/src/mpi/romio/test directory and I see 'runtests' there, but: > ./runtests make: *** No rule to make target 'default'. Stop. I also tried with -fname and: > ./runtests -fname=quobyte:test_mpi_file ./runtests: line 90: [quobyte:test_mpi_file: command not found make: *** No rule to make target 'default'. Stop. > ./runtests -fname=quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file ./runtests: line 90: [quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file: No such file or directory make: *** No rule to make target 'default'. Stop. Yeah. I'm stumped. I'm guessing I've missed some simple step along the way. NOTE: I have no idea even how to use romio really, but I was hoping the tests could, well, help me figure that out. Thanks for any help, Matt [1] See https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… <https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… ><https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… > [signature_3314915376]<https://urldefense.us/v3/__https://gcc02.safelinks.protection.outlook.com/?… > Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]>

1 0

Re: [mpich-discuss] How to make/run romio tests?
by Latham, Robert J. 11 Jun '25

11 Jun '25

Over time, our ROMIO tests have slowly migrated to "mpich tests", and now these legacy tests live on as reference. Thanks for the bug report -- 'runtests' really is in bad shape these days and I should either fix it or delete it. ==rob ________________________________________ From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Sent: Wednesday, June 11, 2025 08:33 To: discuss(a)mpich.org Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] Subject: [mpich-discuss] How to make/run romio tests? All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https: //www. quobyte. com/blog/quobyte-mpi-io-support-with-kernel-bypass-now-available/ ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp… <https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp…> and, yep, quobyte seems to be in MPICH! I grabbed MPICH 4.3.0 and built it (with one bug fix[1]) and well, it seems to run Hello World. So, following that Quobyte page, I get to Step 10 and tried the romio tests and...I am apparently dumb. I went to my build/src/mpi/romio/test directory and I see 'runtests' there, but: > ./runtests make: *** No rule to make target 'default'. Stop. I also tried with -fname and: > ./runtests -fname=quobyte:test_mpi_file ./runtests: line 90: [quobyte:test_mpi_file: command not found make: *** No rule to make target 'default'. Stop. > ./runtests -fname=quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file ./runtests: line 90: [quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file: No such file or directory make: *** No rule to make target 'default'. Stop. Yeah. I'm stumped. I'm guessing I've missed some simple step along the way. NOTE: I have no idea even how to use romio really, but I was hoping the tests could, well, help me figure that out. Thanks for any help, Matt [1] See https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… <https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…> [signature_3314915376]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!foSdMay5…> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]>

1 0

Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests?
by Latham, Robert J. 11 Jun '25

11 Jun '25

We could do a better job documenting the "read-only" vs "read-write" nature of these hints... you cannot override the file system with the "romio_filesystem_type" hint, only confirm the file system is what you thought it was. you should be able to with the ROMIO_FSTYPE_FORCE environment variable. These symptoms suggest you requested quobyte support but ROMIO's configure could not find the development headers.... except ROMIO doesn't check for any development headers. Can you share with me the first few lines of ROMIO's config.log? From your build directory, that would be ` head mpich/src/mpi/romio/config.log` (the whole config.log is too much... for now) ==rob ________________________________________ From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Sent: Wednesday, June 11, 2025 10:25 To: discuss(a)mpich.org Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] Subject: Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests? An addendum, it's possible my "bugfix" as detailed in https: //github. com/pmodels/mpich/issues/7452 might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd An addendum, it's possible my "bugfix" as detailed in https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… <https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…> might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: export ROMIO_PRINT_HINTS=1 and then in my code I added: call MPI_INFO_SET(info,"romio_filesystem_type","QUOBYTEFS:", status) but, well: key = romio_filesystem_type value = UFS: Generic ROMIO driver for all UNIX-like file systems I also tried it in lowercase as well. No joy. I've also tried: export ROMIO_FSTYPE_FORCE="QUOBYTEFS:" and: export ROMIO_FSTYPE_FORCE="quobytefs:" but in each case, it's still UFS. I do know my Infos are being seen by MPICH because if I toggled: call MPI_INFO_SET(info,"romio_cb_write","enable", status) from enable to disable, the ROMIO_PRINT_HINTS show the change. Matt [signature_3627681122]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!aqjYcygb…> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]> From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Date: Wednesday, June 11, 2025 at 9:37 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: [EXTERNAL] [BULK] [mpich-discuss] How to make/run romio tests? All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https: //www. quobyte. com/blog/quobyte-mpi-io-support-with-kernel-bypass-now-available/ ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp… <https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp…> and, yep, quobyte seems to be in MPICH! I grabbed MPICH 4.3.0 and built it (with one bug fix[1]) and well, it seems to run Hello World. So, following that Quobyte page, I get to Step 10 and tried the romio tests and...I am apparently dumb. I went to my build/src/mpi/romio/test directory and I see 'runtests' there, but: > ./runtests make: *** No rule to make target 'default'. Stop. I also tried with -fname and: > ./runtests -fname=quobyte:test_mpi_file ./runtests: line 90: [quobyte:test_mpi_file: command not found make: *** No rule to make target 'default'. Stop. > ./runtests -fname=quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file ./runtests: line 90: [quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file: No such file or directory make: *** No rule to make target 'default'. Stop. Yeah. I'm stumped. I'm guessing I've missed some simple step along the way. NOTE: I have no idea even how to use romio really, but I was hoping the tests could, well, help me figure that out. Thanks for any help, Matt [1] See https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… <https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…> [signature_3314915376]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!foSdMay5…> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]>

1 0

Re: [mpich-discuss] [EXTERNAL] [BULK] How to make/run romio tests?
by Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] 11 Jun '25

11 Jun '25

An addendum, it's possible my "bugfix" as detailed in https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… might not be enough. I have a code that uses `MPI_Info_set` to, well, set infos and I can't seem to affect the fs type. First I set: export ROMIO_PRINT_HINTS=1 and then in my code I added: call MPI_INFO_SET(info,"romio_filesystem_type","QUOBYTEFS:", status) but, well: key = romio_filesystem_type value = UFS: Generic ROMIO driver for all UNIX-like file systems I also tried it in lowercase as well. No joy. I've also tried: export ROMIO_FSTYPE_FORCE="QUOBYTEFS:" and: export ROMIO_FSTYPE_FORCE="quobytefs:" but in each case, it's still UFS. I do know my Infos are being seen by MPICH because if I toggled: call MPI_INFO_SET(info,"romio_cb_write","enable", status) from enable to disable, the ROMIO_PRINT_HINTS show the change. Matt [signature_3627681122]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!aqjYcygb… > Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]> From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss <discuss(a)mpich.org> Date: Wednesday, June 11, 2025 at 9:37 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson(a)nasa.gov> Subject: [EXTERNAL] [BULK] [mpich-discuss] How to make/run romio tests? All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https: //www. quobyte. com/blog/quobyte-mpi-io-support-with-kernel-bypass-now-available/ ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp… <https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp…> and, yep, quobyte seems to be in MPICH! I grabbed MPICH 4.3.0 and built it (with one bug fix[1]) and well, it seems to run Hello World. So, following that Quobyte page, I get to Step 10 and tried the romio tests and...I am apparently dumb. I went to my build/src/mpi/romio/test directory and I see 'runtests' there, but: > ./runtests make: *** No rule to make target 'default'. Stop. I also tried with -fname and: > ./runtests -fname=quobyte:test_mpi_file ./runtests: line 90: [quobyte:test_mpi_file: command not found make: *** No rule to make target 'default'. Stop. > ./runtests -fname=quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file ./runtests: line 90: [quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file: No such file or directory make: *** No rule to make target 'default'. Stop. Yeah. I'm stumped. I'm guessing I've missed some simple step along the way. NOTE: I have no idea even how to use romio really, but I was hoping the tests could, well, help me figure that out. Thanks for any help, Matt [1] See https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… <https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!…> [signature_3314915376]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!foSdMay5…> Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]>

1 0

How to make/run romio tests?
by Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] 11 Jun '25

11 Jun '25

All, I'm trying to do some tests on a Quobyte system and we've seen confusing file write timings with Intel MPI and Open MPI. So, a bit of googling seeing if some MPI stack has a mention of Quobyte turns up: https://urldefense.us/v3/__https://www.quobyte.com/blog/quobyte-mpi-io-supp… and, yep, quobyte seems to be in MPICH! I grabbed MPICH 4.3.0 and built it (with one bug fix[1]) and well, it seems to run Hello World. So, following that Quobyte page, I get to Step 10 and tried the romio tests and...I am apparently dumb. I went to my build/src/mpi/romio/test directory and I see 'runtests' there, but: > ./runtests make: *** No rule to make target 'default'. Stop. I also tried with -fname and: > ./runtests -fname=quobyte:test_mpi_file ./runtests: line 90: [quobyte:test_mpi_file: command not found make: *** No rule to make target 'default'. Stop. > ./runtests -fname=quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file ./runtests: line 90: [quobyte:/home/mathomp4/MPI/src/mpich-4.3.0/build-ifort-2021.13/src/mpi/romio/test/test_mpi_file: No such file or directory make: *** No rule to make target 'default'. Stop. Yeah. I'm stumped. I'm guessing I've missed some simple step along the way. NOTE: I have no idea even how to use romio really, but I was hoping the tests could, well, help me figure that out. Thanks for any help, Matt [1] See https://urldefense.us/v3/__https://github.com/pmodels/mpich/issues/7452__;!… [signature_3314915376]<https://urldefense.us/v3/__http://www.ssaihq.com/__;!!G_uCfscf7eWS!foSdMay5… > Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson(a)nasa.gov<mailto:[email protected]>

1 0

Re: [mpich-discuss] mpich 4.3.0 compilation problem when using --with-hcoll=/opt/mellanox/hcoll
by Audet, Martin 16 Apr '25

16 Apr '25

Hello Hui, I tried the patch and it works (it compiles at least). Thanks for your very quick response ! Martin ________________________________ From: Zhou, Hui <zhouh(a)anl.gov> Sent: April 16, 2025 12:17 PM To: discuss(a)mpich.org Cc: Audet, Martin Subject: EXT: Re: [mpich-discuss] mpich 4.3.0 compilation problem when using --with-hcoll=/opt/mellanox/hcoll ***Attention*** This email originated from outside of the NRC. ***Attention*** Ce courriel provient de l'extérieur du CNRC. Hi Martin, Could you try the patch in https://urldefense.us/v3/__https://github.com/pmodels/mpich/pull/7047?__;!!… -- Hui ________________________________ From: Audet, Martin via discuss <discuss(a)mpich.org> Sent: Wednesday, April 16, 2025 11:13 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Audet, Martin <Martin.Audet(a)cnrc-nrc.gc.ca> Subject: [mpich-discuss] mpich 4.3.0 compilation problem when using --with-hcoll=/opt/mellanox/hcoll Hello mpich community, When I try to compile mpich 4. 3. 0 configured with --with-hcoll=/opt/mellanox/hcoll option, I get a compilation error because the hcoll_do_progress() function is defined with two arguments in hcoll_init. c but it is called ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Hello mpich community, When I try to compile mpich 4.3.0 configured with --with-hcoll=/opt/mellanox/hcoll option, I get a compilation error because the hcoll_do_progress() function is defined with two arguments in hcoll_init.c but it is called only with one in hcoll_rte.c ! Here is the error message I get: src/mpid/common/hcoll/hcoll_rte.c: In function 'progress': src/mpid/common/hcoll/hcoll_rte.c:58:33: warning: passing argument 1 of 'hcoll_do_progress' makes integer from pointer without a cast [-Wint-conversion] 58 | ret = hcoll_do_progress(&made_progress); | ^~~~~~~~~~~~~~ | | | int * In file included from ./src/mpid/ch4/netmod/include/../ucx/ucx_coll.h:11, from ./src/mpid/ch4/netmod/include/../ucx/netmod_inline.h:15, from ./src/mpid/ch4/netmod/include/netmod_impl.h:1589, from ./src/mpid/ch4/include/mpidch4.h:448, from ./src/mpid/ch4/include/mpidpost.h:10, from ./src/include/mpiimpl.h:232, from src/mpid/common/hcoll/hcoll_rte.c:6: ./src/mpid/ch4/netmod/include/../ucx/../../../common/hcoll/hcoll.h:42:27: note: expected 'int' but argument is of type 'int *' 42 | int hcoll_do_progress(int vci, int *made_progress); | ~~~~^~~ src/mpid/common/hcoll/hcoll_rte.c:58:15: error: too few arguments to function 'hcoll_do_progress' 58 | ret = hcoll_do_progress(&made_progress); | ^~~~~~~~~~~~~~~~~ In file included from ./src/mpid/ch4/netmod/include/../ucx/ucx_coll.h:11, from ./src/mpid/ch4/netmod/include/../ucx/netmod_inline.h:15, from ./src/mpid/ch4/netmod/include/netmod_impl.h:1589, from ./src/mpid/ch4/include/mpidch4.h:448, from ./src/mpid/ch4/include/mpidpost.h:10, from ./src/include/mpiimpl.h:232, from src/mpid/common/hcoll/hcoll_rte.c:6: ./src/mpid/ch4/netmod/include/../ucx/../../../common/hcoll/hcoll.h:42:5: note: declared here 42 | int hcoll_do_progress(int vci, int *made_progress); | ^~~~~~~~~~~~~~~~~ I use to compile mpich versions 3.4,x, 4.1.x, and 4.2.x configured with this option (--with-hcoll=) in the past without any problems. It looks like some recent changes in the related files introduced a problem that slip into 4.3.0 and makes compilation impossible. Could it be fixed ? Or could the --with-hcoll option be removed if it is no longer relevant (I guess that if we use ch4:ucx, ucx may itself use hcoll internally to optimize collective operations when running on hierarchical environment) ? Here are some details: arch: x86_64 OS: RHEL 9.5 (up to date except kernel) MOFED: 24.10-2.1.8.0-LTS hcoll: 4.8.3230-1.2410068 ucx: 1.18.0-1.2410068 The complete configuration line: ./configure --with-device=ch4:ucx --with-hcoll=/opt/mellanox/hcoll --prefix=/work/software/x86_64/mpich/mpich-ch4_ucx-4.3.0 --with-xpmem --enable-g=none --enable-fast=all --enable-romio --with-file-system=ufs+nfs+lustre --enable-shared --enable-sharedlibs=gcc Thanks, Martin Audet

1 0