January 2021 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] config error
by Zhou, Hui 20 Jan '21

20 Jan '21

Does this help? [---- quote ----] configure: error: no ch4 netmod selected The default ch4 device could not detect a preferred network library. Supported options are ofi (libfabric) and ucx: --with-device=ch4:ofi or --with-device=ch4:ucx Configure will use an embedded copy of libfabric or ucx if one is not found in the user environment. An installation can be specified by adding --with-libfabric=<path/to/install> or --with-ucx=<path/to/install> to the configuration. The previous MPICH default device (ch3) is also available and supported with option: --with-device=ch3 [ -- end-quote --] -- Hui Zhou From: Maksim Kozlov via discuss <discuss(a)mpich.org> Date: Wednesday, January 20, 2021 at 9:06 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Maksim Kozlov <maksim.kozlov(a)nu.edu.kz> Subject: [mpich-discuss] config error Dear Sir or Madam, could you please help me with installing MPICH. I run ~/mpich-3.4/configure -prefix=/home/maksim/mpich-install |& tee c.txt but I guess something went wrong because makefile was not created. c.txt is attached. thank you very much Maksim Kozlov

1 0

config error
by Maksim Kozlov 20 Jan '21

20 Jan '21

Dear Sir or Madam, could you please help me with installing MPICH. I run ~/mpich-3.4/configure -prefix=/home/maksim/mpich-install |& tee c.txt but I guess something went wrong because makefile was not created. c.txt is attached. thank you very much Maksim Kozlov

1 0

Re: [mpich-discuss] Issue with libcuda.a
by Damiano Casalino - LR 18 Jan '21

18 Jan '21

I undrrstand. Thanks. On a parallel effort I tried to compile cuda and generate the static library, but the compilation failed because of a conflict between a function type in the library and the same function in a python header. After many issues that I had to face with CentOS-8, I must say that this OS is not really a good one! Damiano From: "Zhou, Hui" <zhouh(a)anl.gov> Date: Monday 18 January 2021 at 23:33 To: "discuss(a)mpich.org" <discuss(a)mpich.org> Cc: Damiano Casalino - LR <D.Casalino(a)tudelft.nl> Subject: Re: Issue with libcuda.a I believe the official centos-8 distribution doesn’t have GPU support. The latest relase will support gpu when cuda library is detected during configure. Unfortunately, there isn’t an easy way to switch off cuda when the cuda is in the system path. Maybe try compile it in a container that doesn’t have cuda installed? -- Hui Zhou From: Damiano Casalino - LR via discuss <discuss(a)mpich.org> Date: Monday, January 18, 2021 at 11:29 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Damiano Casalino - LR <D.Casalino(a)tudelft.nl> Subject: [mpich-discuss] Issue with libcuda.a I have built mpich on a centos-8 gcc-8.3.1 machine and then tried to link it statically with my tools. The link failed because -lcuda was missing. The same does not happen if I use mpich from the official centos-8 distrib. Do you have any explanation about this behavior? Thank you, Damiano

1 0

Re: [mpich-discuss] Issue with libcuda.a
by Zhou, Hui 18 Jan '21

18 Jan '21

I believe the official centos-8 distribution doesn’t have GPU support. The latest relase will support gpu when cuda library is detected during configure. Unfortunately, there isn’t an easy way to switch off cuda when the cuda is in the system path. Maybe try compile it in a container that doesn’t have cuda installed? -- Hui Zhou From: Damiano Casalino - LR via discuss <discuss(a)mpich.org> Date: Monday, January 18, 2021 at 11:29 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: Damiano Casalino - LR <D.Casalino(a)tudelft.nl> Subject: [mpich-discuss] Issue with libcuda.a I have built mpich on a centos-8 gcc-8.3.1 machine and then tried to link it statically with my tools. The link failed because -lcuda was missing. The same does not happen if I use mpich from the official centos-8 distrib. Do you have any explanation about this behavior? Thank you, Damiano

1 0

Issue with libcuda.a
by Damiano Casalino - LR 18 Jan '21

18 Jan '21

I have built mpich on a centos-8 gcc-8.3.1 machine and then tried to link it statically with my tools. The link failed because -lcuda was missing. The same does not happen if I use mpich from the official centos-8 distrib. Do you have any explanation about this behavior? Thank you, Damiano

1 0

Re: [mpich-discuss] ask about mpich-3.3.1 installation
by Zhou, Hui 16 Jan '21

16 Jan '21

Try export FC=gfortran. -- Hui Zhou From: 马俊 via discuss <discuss(a)mpich.org> Date: Saturday, January 16, 2021 at 7:44 AM To: discuss(a)mpich.org <discuss(a)mpich.org> Cc: 马俊 <majun123(a)whu.edu.cn> Subject: [mpich-discuss] ask about mpich-3.3.1 installation Dear mpich staff， when I'm installing mpich-3.3.1 using gcc-8.3.1, unset F90 export F90=gfortran export F77=gfortran ./configure --prefix=/home/majun2/dat01/setup_new/mpich/ error shows: nfigure: WARNING: installation or configuration problem: Fortran 90 compiler cannot create executables. checking whether Fortran 90 compiler works with Fortran 77 compiler... configure: WARNING: See config.log for the failed test program and its output. no configure: error: The selected Fortran 90 compiler gfortran does not work with the selected Fortran 77 compiler gfortran. Use the environment variables FC and F77 respectively to select compatible Fortran compilers. The check here tests to see if a main program compiled with the Fortran 90 compiler can link with a subroutine compiled with the Fortran 77 compiler. config.log is attached below, anybody know what's the problem? thanks a lot!

1 0

ask about mpich-3.3.1 installation
by 马俊 16 Jan '21

16 Jan '21

Dear mpich staff， when I'm installing mpich-3.3.1 using gcc-8.3.1, unset F90 export F90=gfortran export F77=gfortran ./configure --prefix=/home/majun2/dat01/setup_new/mpich/ error shows: nfigure: WARNING: installation or configuration problem: Fortran 90 compiler cannot create executables. checking whether Fortran 90 compiler works with Fortran 77 compiler... configure: WARNING: See config.log for the failed test program and its output. no configure: error: The selected Fortran 90 compiler gfortran does not work with the selected Fortran 77 compiler gfortran. Use the environment variables FC and F77 respectively to select compatible Fortran compilers. The check here tests to see if a main program compiled with the Fortran 90 compiler can link with a subroutine compiled with the Fortran 77 compiler. config.log is attached below, anybody know what's the problem? thanks a lot!

1 0

Re: [mpich-discuss] [mpich-devel] mpich3 error
by Zhou, Hui 15 Jan '21

15 Jan '21

Your description only mentions MPI_Gather. If there is indeed problem with MPI_Gather, then you should be able to reproduce the issue with a sample program. Share with us and we can better assist you. If you can’t reproduce the issue with a simple example, then I suspect there are other problems that you are not able to fully describe. We really can’t help much without able to see the code. That said, I am not even sure what is the issue you are describing. 100 process MPI_Gather will be slower than 50 process MPI_Gather. And since it is a collective, if one of your process is delayed due to some computations or else, the whole collective will take longer to finish just due to waiting for the late process. You really need tell us what your program is doing in order for us to even offer an intelligent guess. -- Hui Zhou From: Brent Morgan <brent.taylormorgan(a)gmail.com> Date: Friday, January 15, 2021 at 10:42 PM To: Zhou, Hui <zhouh(a)anl.gov>, discuss(a)mpich.org <discuss(a)mpich.org> Cc: Robert Katona <robert.katona(a)hotmail.com> Subject: Re: [mpich-devel] mpich3 error Hi MPICH community, My team has downloaded mpich 3.3.2 (using ch3 as default) and implemented MPI, and for small # of processes (<50), everything worked fine for our MPI implementation. For >=50 processes, there was a ch3 error and crashed the program after a random amount of seconds (sometimes 10seconds, sometimes 100seconds). So we compiled mpich 3.3.2 with ch4 (instead of default ch3) using '--with-device=ch4:ofi` flag and this got rid of the error- but for >12 processes, the speed would slow down to 2x slower suddenly. Upon Hui's suggestion, we upgraded to mpich 3.4 and compiled with '--with-device=ch4:ofi` flag (where ch4 is default for mpich 3.4). Everything worked fine until we hit 20 processes; after >=20 processes, the 2x slowdown is happening again. We have tried 1 communicator and multiple communicators in an attempt to make the MPI implementation faster, but there's no significant difference in observations. We are using MPI_Gather collector for merely calculating the sum of the result of N processes, but we can't seem to maintain stability within MPI as we increase N processes. Is there something we are missing that is ultimately causing this error? We are at a loss here, thank you. Best, Brent

1 0

Re: [mpich-discuss] Question about non-blocking MPI - 3
by teivml＠gmail.com 15 Jan '21

15 Jan '21

Dear Joachim, Thank you for your response. It is good to know that no special settings are required to use non-blocking MPI collectives in MPICH, and the setting "MPICH_ASYNC_PROGRESS = 1" improves the performance of calculations that overlap communication. I will find tools to confirm this. Best, Viet On Thu, Jan 7, 2021 at 10:09 PM Joachim Protze <protze(a)itc.rwth-aachen.de> wrote: > Hi Viet, > > If MPICH_ASYNC_PROGRESS=1 is necessary for your application to > terminate, i.e., you see a deadlock without this flag, the applications > has a correctness/portability issue. This env variable should only have > a possible impact to performance (better compute/communication overlap) > but should not be necessary to avoid deadlock. > > You can compare this to the classical send-send deadlock (example 3.9 in > the standard), where the application relies on buffering / eager > communication, but all send suddenly block. The portable fix for the > application is not to increase the eager-limit, but to fix the > communication pattern. > > Best > Joachim > > Am 07.01.21 um 13:08 schrieb viet via discuss: > > Dear Hui Zhou, > > > > Thank you for your response to this thread. > > Could you give me some references on this issue? > > > > It seems that the below environment setting is necessary. > > > > MPICH_ASYNC_PROGRESS=1 > > > > Thank you for anything you can provide. > > Best, Viet. > > > > > > > > On Tue, Jan 5, 2021 at 5:59 AM Zhou, Hui <zhouh(a)anl.gov> wrote: > > > >> You don’t need any special settings to use non-blocking MPI collectives > in > >> MPICH. > >> > >> > >> > >> -- > >> Hui Zhou > >> > >> > >> > >> > >> > >> *From: *viet via discuss <discuss(a)mpich.org> > >> *Date: *Friday, December 4, 2020 at 7:12 AM > >> *To: *discuss(a)mpich.org <discuss(a)mpich.org> > >> *Cc: *teivml(a)gmail.com <teivml(a)gmail.com> > >> *Subject: *[mpich-discuss] Question about non-blocking MPI - 3 > >> > >> Hello everyone, > >> > >> What is the environment setting for the use of non-blocking MPI-3 * > >> collectives in mpich3.3? > >> > >> In Intel MPI, the setting is as follows > >> > >> $ export I_MPI_ASYNC_PROGRESS=1 > >> $ export I_MPI_ASYNC_PROGRESS_PIN=<CPU list> > >> > >> Is there an equivalent setting in mpich3.3? > >> Thank you, > >> Viet. > >> > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > -- > Dipl.-Inf. Joachim Protze > > IT Center > Group: High Performance Computing > Division: Computational Science and Engineering > RWTH Aachen University > Seffenter Weg 23 > D 52074 Aachen (Germany) > Tel: +49 241 80- 24765 > Fax: +49 241 80-624765 > protze(a)itc.rwth-aachen.de > www.itc.rwth-aachen.de > >

1 0

Re: [mpich-discuss] Question about non-blocking MPI - 3
by Palmer, Bruce J 07 Jan '21

07 Jan '21

As far as I know, you don’t have to use the MPI_ASYNC_PROGRESS settings with Intel MPI, either. However, you may see a huge boost in performance, at least for the onesided operations using MPI Windows. Bruce Palmer From: viet via discuss <discuss(a)mpich.org> Sent: Thursday, January 7, 2021 4:08 AM To: Zhou, Hui <zhouh(a)anl.gov>; discuss(a)mpich.org Cc: teivml(a)gmail.com Subject: Re: [mpich-discuss] Question about non-blocking MPI - 3 Check twice before you click! This email originated from outside PNNL. Dear Hui Zhou, Thank you for your response to this thread. Could you give me some references on this issue? It seems that the below environment setting is necessary. MPICH_ASYNC_PROGRESS=1 Thank you for anything you can provide. Best, Viet. On Tue, Jan 5, 2021 at 5:59 AM Zhou, Hui <zhouh(a)anl.gov<mailto:[email protected]>> wrote: You don’t need any special settings to use non-blocking MPI collectives in MPICH. -- Hui Zhou From: viet via discuss <discuss(a)mpich.org<mailto:[email protected]>> Date: Friday, December 4, 2020 at 7:12 AM To: discuss(a)mpich.org<mailto:[email protected]> <discuss(a)mpich.org<mailto:[email protected]>> Cc: teivml(a)gmail.com<mailto:[email protected]> <teivml(a)gmail.com<mailto:[email protected]>> Subject: [mpich-discuss] Question about non-blocking MPI - 3 Hello everyone, What is the environment setting for the use of non-blocking MPI-3 * collectives in mpich3.3? In Intel MPI, the setting is as follows $ export I_MPI_ASYNC_PROGRESS=1 $ export I_MPI_ASYNC_PROGRESS_PIN=<CPU list> Is there an equivalent setting in mpich3.3? Thank you, Viet.

1 0