- discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Installation error: size of array element is not a multiple of its alignment
by Raffenetti, Ken 03 Jan '22

03 Jan '22

Thanks for reporting. I have created https://github.com/pmodels/mpich/issues/5744 to track the issue. If you have a chance, could you confirm the issue also exists with the latest MPICH release candidate? http://www.mpich.org/static/downloads/4.0rc1/mpich-4.0rc1.tar.gz Ken On 12/29/21, 2:51 PM, "Douglas G Dommermuth via discuss" <discuss(a)mpich.org> wrote: Dear mpich.org <http://mpich.org>, I was able to compile with gcc 10.2.0. Thank you, Doug. On Dec 29, 2021, at 9:40 AM, Douglas G Dommermuth <dgd(a)mit.edu> wrote: Dear mpich.org <http://mpich.org/>, I apologize. I sent this email out with the wrong subject matter. Here is the correct subject. Thank you. I am getting an error installing mpich-3.4.3 on a MacBook Air (M1,2020) with macOS Monterey 12.1, gcc 11.2.0, g++ 11.2.0, and gfortran 11.2.0. I used the following commands: ./configure FFLAGS=-fallow-argument-mismatch --prefix=/Users/douglasdommermuth/mpich2/mpich2-install | & tee c.txt make clean make V=1 | & tee m.txt I get the following error: CC src/mpi/attr/lib_libpmpi_la-attr_delete.lo In file included from ./src/mpid/ch4/netmod/include/../ofi/ofi_impl.h:10, from ./src/mpid/ch4/netmod/include/../ofi/ofi_am.h:8, from ./src/mpid/ch4/netmod/include/../ofi/netmod_inline.h:9, from ./src/mpid/ch4/netmod/include/netmod_impl.h:1644, from ./src/mpid/ch4/include/mpidch4.h:322, from ./src/mpid/ch4/include/mpidpost.h:10, from ./src/include/mpiimpl.h:241, from src/mpi/attr/attr_delete.c:6: ./src/mpid/ch4/netmod/include/../ofi/ofi_types.h:326:5: error: size of array element is not a multiple of its alignment 326 | MPIDI_OFI_cacheline_mutex_t mutexes[MAX_OFI_MUTEXES]; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ make[2]: *** [src/mpi/attr/lib_libpmpi_la-attr_delete.lo] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 An online search indicates that others are experiencing this error with gcc 11. For example, here https://github.com/JayDDee/cpuminer-opt/issues/313, https://gitlab.com/xonotic/darkplaces/-/issues/363, and https://github.com/relic-toolkit/relic/issues/202. I have not tried an earlier version of gcc. Compressed versions of c.txt, m.txt, and config log files are attached. Thank you, Doug. <m.txt.gz><c.txt.gz><config.log.gz><src_pm_hydra_config.log.gz><src_mpl_config.log.gz>

1 0

Re: [mpich-discuss] Slurm and MPICH configuration
by Raffenetti, Ken 03 Jan '22

03 Jan '22

Kurt, No, using sbatch alone should not require special configuration. Only if your job scripts use srun, and not mpiexec/mpirun, should you need to use the Slurm PMI library. Ken On 1/3/22, 5:16 PM, "Mccall, Kurt E. (MSFC-EV41) via discuss" <discuss(a)mpich.org> wrote: Hi, The MPICH docs say the following about running under Slurm. Is “sbatch” one of the launchers for which special configuration is necessary? Thanks, Kurt Q: How do I use MPICH with slurm? A: MPICH's default process manager, Hydra, internally detects and functions correctly with SLURM. No special configuration is needed. However, if you want to use SLURM's launchers (such as srun), you will need to configure MPICH with: ./configure --with-pmi=slurm --with-pm=no In addition, if your slurm installation is not in the default location, you will need to pass the actual installation location using: ./configure --with-pmi=slurm --with-pm=no --with-slurm=[path_to_slurm_install]

1 0

Slurm and MPICH configuration
by Mccall, Kurt E. (MSFC-EV41) 03 Jan '22

03 Jan '22

Hi, The MPICH docs say the following about running under Slurm. Is "sbatch" one of the launchers for which special configuration is necessary? Thanks, Kurt Q: How do I use MPICH with slurm? A: MPICH's default process manager, Hydra, internally detects and functions correctly with SLURM. No special configuration is needed. However, if you want to use SLURM's launchers (such as srun), you will need to configure MPICH with: ./configure --with-pmi=slurm --with-pm=no In addition, if your slurm installation is not in the default location, you will need to pass the actual installation location using: ./configure --with-pmi=slurm --with-pm=no --with-slurm=[path_to_slurm_install]

1 0

Re: [mpich-discuss] Installation error: size of array element is not a multiple of its alignment
by Douglas G Dommermuth 29 Dec '21

29 Dec '21

Dear mpich.org<http://mpich.org>, I was able to compile with gcc 10.2.0. Thank you, Doug. On Dec 29, 2021, at 9:40 AM, Douglas G Dommermuth <dgd(a)mit.edu<mailto:[email protected]>> wrote: Dear mpich.org<http://mpich.org/>, I apologize. I sent this email out with the wrong subject matter. Here is the correct subject. Thank you. I am getting an error installing mpich-3.4.3 on a MacBook Air (M1,2020) with macOS Monterey 12.1, gcc 11.2.0, g++ 11.2.0, and gfortran 11.2.0. I used the following commands: ./configure FFLAGS=-fallow-argument-mismatch --prefix=/Users/douglasdommermuth/mpich2/mpich2-install | & tee c.txt make clean make V=1 | & tee m.txt I get the following error: CC src/mpi/attr/lib_libpmpi_la-attr_delete.lo In file included from ./src/mpid/ch4/netmod/include/../ofi/ofi_impl.h:10, from ./src/mpid/ch4/netmod/include/../ofi/ofi_am.h:8, from ./src/mpid/ch4/netmod/include/../ofi/netmod_inline.h:9, from ./src/mpid/ch4/netmod/include/netmod_impl.h:1644, from ./src/mpid/ch4/include/mpidch4.h:322, from ./src/mpid/ch4/include/mpidpost.h:10, from ./src/include/mpiimpl.h:241, from src/mpi/attr/attr_delete.c:6: ./src/mpid/ch4/netmod/include/../ofi/ofi_types.h:326:5: error: size of array element is not a multiple of its alignment 326 | MPIDI_OFI_cacheline_mutex_t mutexes[MAX_OFI_MUTEXES]; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ make[2]: *** [src/mpi/attr/lib_libpmpi_la-attr_delete.lo] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 An online search indicates that others are experiencing this error with gcc 11. For example, here https://github.com/JayDDee/cpuminer-opt/issues/313, https://gitlab.com/xonotic/darkplaces/-/issues/363, and https://github.com/relic-toolkit/relic/issues/202. I have not tried an earlier version of gcc. Compressed versions of c.txt, m.txt, and config log files are attached. Thank you, Doug. <m.txt.gz><c.txt.gz><config.log.gz><src_pm_hydra_config.log.gz><src_mpl_config.log.gz>

1 0

Installation error: size of array element is not a multiple of its alignment
by Douglas G Dommermuth 29 Dec '21

29 Dec '21

Dear mpich.org<http://mpich.org/>, I apologize. I sent this email out with the wrong subject matter. Here is the correct subject. Thank you. I am getting an error installing mpich-3.4.3 on a MacBook Air (M1,2020) with macOS Monterey 12.1, gcc 11.2.0, g++ 11.2.0, and gfortran 11.2.0. I used the following commands: ./configure FFLAGS=-fallow-argument-mismatch --prefix=/Users/douglasdommermuth/mpich2/mpich2-install | & tee c.txt make clean make V=1 | & tee m.txt I get the following error: CC src/mpi/attr/lib_libpmpi_la-attr_delete.lo In file included from ./src/mpid/ch4/netmod/include/../ofi/ofi_impl.h:10, from ./src/mpid/ch4/netmod/include/../ofi/ofi_am.h:8, from ./src/mpid/ch4/netmod/include/../ofi/netmod_inline.h:9, from ./src/mpid/ch4/netmod/include/netmod_impl.h:1644, from ./src/mpid/ch4/include/mpidch4.h:322, from ./src/mpid/ch4/include/mpidpost.h:10, from ./src/include/mpiimpl.h:241, from src/mpi/attr/attr_delete.c:6: ./src/mpid/ch4/netmod/include/../ofi/ofi_types.h:326:5: error: size of array element is not a multiple of its alignment 326 | MPIDI_OFI_cacheline_mutex_t mutexes[MAX_OFI_MUTEXES]; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ make[2]: *** [src/mpi/attr/lib_libpmpi_la-attr_delete.lo] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 An online search indicates that others are experiencing this error with gcc 11. For example, here https://github.com/JayDDee/cpuminer-opt/issues/313, https://gitlab.com/xonotic/darkplaces/-/issues/363, and https://github.com/relic-toolkit/relic/issues/202. I have not tried an earlier version of gcc. Compressed versions of c.txt, m.txt, and config log files are attached. Thank you, Doug.

1 0

Re: [mpich-discuss] Optimizing runtime for 4 mpiexec.hydra jobs
by Douglas G Dommermuth 29 Dec '21

29 Dec '21

Dear mpich.org<http://mpich.org>, I am getting an error installing mpich-3.4.3 on a MacBook Air (M1,2020) with macOS Monterey 12.1, gcc 11.2.0, g++ 11.2.0, and gfortran 11.2.0. I used the following commands: ./configure FFLAGS=-fallow-argument-mismatch --prefix=/Users/douglasdommermuth/mpich2/mpich2-install | & tee c.txt make clean make V=1 | & tee m.txt I get the following error: CC src/mpi/attr/lib_libpmpi_la-attr_delete.lo In file included from ./src/mpid/ch4/netmod/include/../ofi/ofi_impl.h:10, from ./src/mpid/ch4/netmod/include/../ofi/ofi_am.h:8, from ./src/mpid/ch4/netmod/include/../ofi/netmod_inline.h:9, from ./src/mpid/ch4/netmod/include/netmod_impl.h:1644, from ./src/mpid/ch4/include/mpidch4.h:322, from ./src/mpid/ch4/include/mpidpost.h:10, from ./src/include/mpiimpl.h:241, from src/mpi/attr/attr_delete.c:6: ./src/mpid/ch4/netmod/include/../ofi/ofi_types.h:326:5: error: size of array element is not a multiple of its alignment 326 | MPIDI_OFI_cacheline_mutex_t mutexes[MAX_OFI_MUTEXES]; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ make[2]: *** [src/mpi/attr/lib_libpmpi_la-attr_delete.lo] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 An online search indicates that others are experiencing this error with gcc 11. For example, here https://github.com/JayDDee/cpuminer-opt/issues/313, https://gitlab.com/xonotic/darkplaces/-/issues/363, and https://github.com/relic-toolkit/relic/issues/202. I have not tried an earlier version of gcc. Compressed versions of c.txt, m.txt, and config log files are attached. Thank you, Doug.

1 0

Re: [mpich-discuss] [EXTERNAL] Re: Issue with MPICH 4.0b1 and MPI I/O and ROMIO...maybe?
by Zhou, Hui 23 Dec '21

23 Dec '21

> Also, I don't see --with-file-system in the configure help? Does > the MPICH configure "pass down" the option to (I assume) > ROMIO? Just to confirm that MPICH configure will pass down all options including that MPICH itself is not aware. We probably should make MPICH "aware" of these options to provide better user helps. -- Hui Zhou

1 0

Re: [mpich-discuss] [EXTERNAL] Re: Issue with MPICH 4.0b1 and MPI I/O and ROMIO...maybe?
by Latham, Robert J. 22 Dec '21

22 Dec '21

On Wed, 2021-12-22 at 17:35 +0000, Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] wrote: > Rob, > > Thanks. So for a cluster that is mainly "regular linux" (not sure > what they run for non-GPFS), GPFS, some tmpfs probably, and a whole > bunch of NFS, what would recommend as a good set of filesystems? > Would something like: > > --with-file-system=gpfs > > be enough, or once you have that flag does that do enough? If you have any gpfs anywhere, then yeah, you can add the 'gpfs' flag. You probably want "--with-file-system=ufs+gpfs" ROMIO has an NFS driver but I'll warn you it is not -- and in fact cannot be -- very good. There are many cases where if you try to write data to NFS in paralllel you will end up with corrupted data. If you do manage to not get data corruption, your performance will be terrible because ROMIO tried really really hard to ensure data correctness (wrapping each and every I/O call with a lock/unlock in an effort to force client-side data cache flushes). > Also, I don't see --with-file-system in the configure help? Does the > MPICH configure "pass down" the option to (I assume) ROMIO? correct. ROMIO is both it's own thing and part of MPICH. Uh, sorry about that. ==rob > Matt > -- > Matt Thompson, SSAI, Ld Scientific Programmer/Analyst > NASA GSFC, Global Modeling and Assimilation Office > Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 > Phone: 301-614-6712 Fax: 301-614-6246 > http://science.gsfc.nasa.gov/sed/bio/matthew.thompson > > On 12/20/21, 10:21 AM, "Latham, Robert J." <robl(a)mcs.anl.gov> wrote: > > On Fri, 2021-12-17 at 19:10 +0000, Thompson, Matt (GSFC- > 610.1)[SCIENCE > SYSTEMS AND APPLICATIONS INC] via discuss wrote: > > MPICH Discuss, > NetCDF: Error initializing for parallel access > > > > I did a bit of debugging and found that the crash was due to an > > environment variable that was set because my application > mistakenly > > thought I was running Intel MPI (mainly because we didn't have > > detection for MPICH, so it defaulted to our "default" on this > cluster > > of Intel MPI). When it sees Intel MPI, it sets: > > > > ROMIO_FSTYPE_FORCE="gpfs:" > > > > which we've found is useful when running with Intel MPI on our > GPFS > > system. > > Hah I had no idea anyone besides me was using this little > feature. Here's a bit more information for context: > > > https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpress3.mc… > > > > So, of course the "right" thing to do is not to set that. > (Doctor, it > > hurts when I do this. So stop doing that.) > > > > But it got me wondering, is there perhaps a "better" way I > should be > > building MPICH? Should this flag cause this sort of crash? Or > does it > > mean I build MPICH/ROMIO incorrectly or incompletely (no GPFS > > support, say)? > > Intel MPI has its own way of supporting extra file systems, via > the "I_MPI_EXTRA_FILESYSTEM" environment variable. > > > https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpress3.mc… > > I'm open to suggestions on how ROMIO should best handle this > situation. You requested a file system (gpfs) that ROMIO did not > support. ROMIO could fall back to "generic unix file system" but you > asked explicitly for GPFS, presumably for a good reason (or in your > case, by accident but ROMIO is not a mind reader...) > > If you are building your own MPICH, add the `--with-file- > system=...` flag. that is a '+'-delmited list of file systems. for > example: > > `--with-file-system=ufs+testfs+gpfs+lustre+panfs+pvfs2+unify` > > I try to build everything I can so my list is quite long. Yours > will be shorter -- if you pick a file system for which you do not > have development headers and libraries, your build will fail > (hopefully at configure time). > > ==rob >

1 0

Re: [mpich-discuss] [EXTERNAL] Re: Issue with MPICH 4.0b1 and MPI I/O and ROMIO...maybe?
by Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] 22 Dec '21

22 Dec '21

Rob, Thanks. So for a cluster that is mainly "regular linux" (not sure what they run for non-GPFS), GPFS, some tmpfs probably, and a whole bunch of NFS, what would recommend as a good set of filesystems? Would something like: --with-file-system=gpfs be enough, or once you have that flag does that do enough? Also, I don't see --with-file-system in the configure help? Does the MPICH configure "pass down" the option to (I assume) ROMIO? Matt -- Matt Thompson, SSAI, Ld Scientific Programmer/Analyst NASA GSFC, Global Modeling and Assimilation Office Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 Phone: 301-614-6712 Fax: 301-614-6246 http://science.gsfc.nasa.gov/sed/bio/matthew.thompson On 12/20/21, 10:21 AM, "Latham, Robert J." <robl(a)mcs.anl.gov> wrote: On Fri, 2021-12-17 at 19:10 +0000, Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss wrote: > MPICH Discuss, NetCDF: Error initializing for parallel access > > I did a bit of debugging and found that the crash was due to an > environment variable that was set because my application mistakenly > thought I was running Intel MPI (mainly because we didn't have > detection for MPICH, so it defaulted to our "default" on this cluster > of Intel MPI). When it sees Intel MPI, it sets: > > ROMIO_FSTYPE_FORCE="gpfs:" > > which we've found is useful when running with Intel MPI on our GPFS > system. Hah I had no idea anyone besides me was using this little feature. Here's a bit more information for context: https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpress3.mc… > So, of course the "right" thing to do is not to set that. (Doctor, it > hurts when I do this. So stop doing that.) > > But it got me wondering, is there perhaps a "better" way I should be > building MPICH? Should this flag cause this sort of crash? Or does it > mean I build MPICH/ROMIO incorrectly or incompletely (no GPFS > support, say)? Intel MPI has its own way of supporting extra file systems, via the "I_MPI_EXTRA_FILESYSTEM" environment variable. https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpress3.mc… I'm open to suggestions on how ROMIO should best handle this situation. You requested a file system (gpfs) that ROMIO did not support. ROMIO could fall back to "generic unix file system" but you asked explicitly for GPFS, presumably for a good reason (or in your case, by accident but ROMIO is not a mind reader...) If you are building your own MPICH, add the `--with-file-system=...` flag. that is a '+'-delmited list of file systems. for example: `--with-file-system=ufs+testfs+gpfs+lustre+panfs+pvfs2+unify` I try to build everything I can so my list is quite long. Yours will be shorter -- if you pick a file system for which you do not have development headers and libraries, your build will fail (hopefully at configure time). ==rob

1 0

Re: [mpich-discuss] Issue with MPICH 4.0b1 and MPI I/O and ROMIO...maybe?
by Latham, Robert J. 20 Dec '21

20 Dec '21

On Fri, 2021-12-17 at 19:10 +0000, Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via discuss wrote: > MPICH Discuss, NetCDF: Error initializing for parallel access > > I did a bit of debugging and found that the crash was due to an > environment variable that was set because my application mistakenly > thought I was running Intel MPI (mainly because we didn't have > detection for MPICH, so it defaulted to our "default" on this cluster > of Intel MPI). When it sees Intel MPI, it sets: > > ROMIO_FSTYPE_FORCE="gpfs:" > > which we've found is useful when running with Intel MPI on our GPFS > system. Hah I had no idea anyone besides me was using this little feature. Here's a bit more information for context: https://press3.mcs.anl.gov/romio/2019/02/20/useful-environment-variables/ > So, of course the "right" thing to do is not to set that. (Doctor, it > hurts when I do this. So stop doing that.) > > But it got me wondering, is there perhaps a "better" way I should be > building MPICH? Should this flag cause this sort of crash? Or does it > mean I build MPICH/ROMIO incorrectly or incompletely (no GPFS > support, say)? Intel MPI has its own way of supporting extra file systems, via the "I_MPI_EXTRA_FILESYSTEM" environment variable. https://press3.mcs.anl.gov/romio/2014/06/12/romio-and-intel-mpi/ I'm open to suggestions on how ROMIO should best handle this situation. You requested a file system (gpfs) that ROMIO did not support. ROMIO could fall back to "generic unix file system" but you asked explicitly for GPFS, presumably for a good reason (or in your case, by accident but ROMIO is not a mind reader...) If you are building your own MPICH, add the `--with-file-system=...` flag. that is a '+'-delmited list of file systems. for example: `--with-file-system=ufs+testfs+gpfs+lustre+panfs+pvfs2+unify` I try to build everything I can so my list is quite long. Yours will be shorter -- if you pick a file system for which you do not have development headers and libraries, your build will fail (hopefully at configure time). ==rob

1 0