- discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Non-blocking collectives
by Jed Brown 29 Jun '13

29 Jun '13

Rajeev Thakur <thakur(a)mcs.anl.gov> writes: > It doesn't have to be a matching Wait. The implementation is required > to make progress on other pending communication while it is blocked in > a Wait. Or blocked in any of a number of other places, if non-blocking collective requests have the same completion semantics as point-to-point requests. E.g., this shouldn't deadlock: MPI_Iallreduce(...,COMM_WORLD,&req); if (!rank) MPI_Wait(&req); MPI_Allgather(...,COMM_WORLD); if (rank) MPI_Wait(&req);

1 0

Re: [mpich-discuss] Non-blocking collectives
by Rajeev Thakur 29 Jun '13

29 Jun '13

It doesn't have to be a matching Wait. The implementation is required to make progress on other pending communication while it is blocked in a Wait. On Jun 29, 2013, at 2:43 PM, Jed Brown wrote: > Rajeev Thakur <thakur(a)mcs.anl.gov> writes: > >> I think the Wait is allowed to wait until others call Wait. > > Do you read this as allowing an implementation to defer completion until > a _matching_ MPI_Wait is called, or merely until the application > re-enters MPI somehow (sufficiently many times, or blocking)? I had the > latter interpretation, which would allow non-blocking operations on > different communicators to be used without risking deadlock. > > If the intent was really that the implementation can block until a > _matching_ MPI_Wait is called, then I think the standard should clarify > this point because it risks deadlock in a natural use case. > >> See the Advice to Users on 197:27-29: >> >> "Users should be aware that implementations are allowed, but not >> required (with exception of MPI_IBARRIER), to synchronize processes >> during the completion of a nonblocking collective operation."

1 0

Re: [mpich-discuss] Non-blocking collectives
by Jed Brown 29 Jun '13

29 Jun '13

Rajeev Thakur <thakur(a)mcs.anl.gov> writes: > I think the Wait is allowed to wait until others call Wait. Do you read this as allowing an implementation to defer completion until a _matching_ MPI_Wait is called, or merely until the application re-enters MPI somehow (sufficiently many times, or blocking)? I had the latter interpretation, which would allow non-blocking operations on different communicators to be used without risking deadlock. If the intent was really that the implementation can block until a _matching_ MPI_Wait is called, then I think the standard should clarify this point because it risks deadlock in a natural use case. > See the Advice to Users on 197:27-29: > > "Users should be aware that implementations are allowed, but not > required (with exception of MPI_IBARRIER), to synchronize processes > during the completion of a nonblocking collective operation."

1 0

Re: [mpich-discuss] Non-blocking collectives
by Rajeev Thakur 29 Jun '13

29 Jun '13

I think the Wait is allowed to wait until others call Wait. See the Advice to Users on 197:27-29: "Users should be aware that implementations are allowed, but not required (with exception of MPI_IBARRIER), to synchronize processes during the completion of a nonblocking collective operation." On Jun 29, 2013, at 1:25 PM, Jed Brown wrote: > rreddypsc(a)gmail.com writes: > >> I apologize for creating a new thread, I have accidentally deleted the >> original message. >> >> Not to split hairs, can you please clarify the following: >> >> After doing a non-blocking collective: >> >> The MPI_Wait would be satisfied as soon as the other ranks in the >> communicator have called the non-blocking collective (and may not yet have >> called MPI_Wait), right? > > Yes. > >> Or do they wait until all of them have called MPI_Wait? > > No, that would make the MPI_Wait synchronizing, mostly defeating any > potential benefit of non-blocking collectives. Note that the > non-blocking request can complete in other ways, e.g., via MPI_Test. > >> On the contrary, if one of the ranks calls MPI_Wait (without calling the >> non-blocking collective, which is an incorrect program), will the MPI_Wait >> be satisfied? > > This doesn't even make sense. Where would you be getting an MPI_Request > on which to call MPI_Wait? > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Non-blocking collectives
by Jed Brown 29 Jun '13

29 Jun '13

rreddypsc(a)gmail.com writes: > I apologize for creating a new thread, I have accidentally deleted the > original message. > > Not to split hairs, can you please clarify the following: > > After doing a non-blocking collective: > > The MPI_Wait would be satisfied as soon as the other ranks in the > communicator have called the non-blocking collective (and may not yet have > called MPI_Wait), right? Yes. > Or do they wait until all of them have called MPI_Wait? No, that would make the MPI_Wait synchronizing, mostly defeating any potential benefit of non-blocking collectives. Note that the non-blocking request can complete in other ways, e.g., via MPI_Test. > On the contrary, if one of the ranks calls MPI_Wait (without calling the > non-blocking collective, which is an incorrect program), will the MPI_Wait > be satisfied? This doesn't even make sense. Where would you be getting an MPI_Request on which to call MPI_Wait?

1 0

Non-blocking collectives
by rreddypsc＠gmail.com 29 Jun '13

29 Jun '13

I apologize for creating a new thread, I have accidentally deleted the original message. Not to split hairs, can you please clarify the following: After doing a non-blocking collective: The MPI_Wait would be satisfied as soon as the other ranks in the communicator have called the non-blocking collective (and may not yet have called MPI_Wait), right? Or do they wait until all of them have called MPI_Wait? On the contrary, if one of the ranks calls MPI_Wait (without calling the non-blocking collective, which is an incorrect program), will the MPI_Wait be satisfied? Thanks, Raghu [mpich-discuss] Non-blocking Collectives Jiri Simsa jsimsa at cs.cmu.edu Thu Jun 27 10:33:14 CDT 2013 Previous message: [mpich-discuss] Non-blocking Collectives Next message: [mpich-discuss] PMGR_COLLECTIVE ERROR Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Hi Pavan, To rephrase, I am interested in understanding when would MPI_Wait() block indefinitely, waiting for other process to make progress. I believe that your response answers my question. Thanks again. --Jiri On Wed, Jun 26, 2013 at 4:48 PM, Pavan Balaji <balaji at mcs.anl.gov> wrote: > Hi Jiri, > > > On 06/26/2013 03:08 PM, Jiri Simsa wrote: > >> Thank you for your quick answer. I am trying to understand the blocking >> behavior of MPI_Wait in the case of non-blocking collectives. Is it safe >> to assume that, for a non-blocking collective, MPI_Wait is guaranteed to >> return once all other processes call the corresponding completion >> operation (e.g. MPI_Wait or MPI_Test)? >> > > I'm not sure I understand your question. Are you asking if MPI_WAIT in a > process is guaranteed to return after some finite amount of time after > every other process has called MPI_WAIT? Then, yes. > > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] [CIG-SEISMO] MPICH mpif.h incompatible with gfortran-4.8 -std=f2003
by Jed Brown 29 Jun '13

29 Jun '13

Dimitri Komatitsch <komatitsch(a)lma.cnrs-mrs.fr> writes: > Dear all, > > The (only) three things to do to make mpif.h conform to recent standards > (F90, F95, F2003) would be: > > - change all character* declarations to character(len=...) > > - change all real*8 to either double precision or real(kind=8) (both > are OK) > > - change all integer*8 to integer(kind=8) Well, mpif.h is supposed to be backward-compatible to F77. Upon checking the code, I see that MPICH offers the feature of tolerating -i8 and -r8 options. Open MPI uses 'double precision' for MPI_WTIME, which would be promoted to 16 bytes under -r8, so it won't behave correctly if the user expects a real*8 (perhaps specified as real(kind=8) [1]). The MPI standard says the prototype is DOUBLE PRECISION MPI_WTIME() so I would say that MPICH is non-conforming by using REAL*8, though the standard does not make any statements about what changes when compiled with -r8. I haven't checked to see how MPI data types are mapped in case of -r8. For example, MPI_REAL is defined to a constant in mpif.h, and the MPI library (which was compiled in advance) would not be aware of whether user code was being built with -r8. Is the user expected to use MPI_REAL4 and MPI_REAL8 explicitly in this case? Is -r8 and -i8 tested somewhere in MPICH? What exactly is the user expected to do in order to use this feature? I see this comment in src/binding/f90/mpi_sizeofs.f90.in: ! If reals and doubles have been forced to the same size (e.g., with ! -i8 -r8 to compilers like g95), then the compiler may refuse to ! allow interfaces that use real and double precision (failing to ! determine which one is intended) [1] Note that real(kind=8) is not guaranteed by the Fortran standard to be 8 bytes. For example, g77 and the Salford F95 compilers denote 4-bytes reals with real(kind=1), and 8-byte reals with real(kind=2). http://www.silverfrost.com/manuals/salfordftn95.pdf

1 0

Re: [mpich-discuss] MPICH mpif.h incompatible with gfortran-4.8 -std=f2003
by Jeff Hammond 28 Jun '13

28 Jun '13

On Jun 28, 2013, at 5:56 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Jeff Hammond <jeff.science(a)gmail.com> writes: > >> mpif.h is for F77 codes. Use the module for F90+. There is no point >> in fixing mpif.h for F03 because the solution already exists. > > It would be trivially easy to make work, though probably not to get rid > of the warnings. > Great. We look forward to your patch. >> The F08 bindings in MPI-3 are the long-term solution. > > Yup, and your son will be in college by the time MPI-3 is available on > all supported RHEL versions. ;-) You can compile MPICH 3.0.4 on Linux today. It's trivial. Why wait for RPMs? Those might even exist already too. Did you check the MPICH website? Jeff

1 0

Re: [mpich-discuss] MPICH mpif.h incompatible with gfortran-4.8 -std=f2003
by Jed Brown 28 Jun '13

28 Jun '13

Jeff Hammond <jeff.science(a)gmail.com> writes: > mpif.h is for F77 codes. Use the module for F90+. There is no point > in fixing mpif.h for F03 because the solution already exists. It would be trivially easy to make work, though probably not to get rid of the warnings. > The F08 bindings in MPI-3 are the long-term solution. Yup, and your son will be in college by the time MPI-3 is available on all supported RHEL versions. ;-)

1 0

Re: [mpich-discuss] MPICH mpif.h incompatible with gfortran-4.8 -std=f2003
by Jeff Hammond 28 Jun '13

28 Jun '13

mpif.h is for F77 codes. Use the module for F90+. There is no point in fixing mpif.h for F03 because the solution already exists. The F08 bindings in MPI-3 are the long-term solution. Jeff Sent from my iPhone On Jun 28, 2013, at 5:21 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > As shown below in SPECFEM3D, mpif.h cannot be included when using > -std=f2003 with gfortran-4.8. I've Cc'd the MPICH list in case there is > interest in converting mpif.h to be compatible with -std=f2003. Open > MPI uses 'double precision' instead of 'real*8', rendering mpif.h safe > to include. > > On the SPECFEM side, I believe this can be handled by switching from > including mpif.h to using the f90 mpi module. > > > configure:6665: /opt/mpich/bin/mpif90 -c -g -O2 -std=f2003 -fimplicit-none -frange-check -O2 -DFORCE_VECTORIZATION -fmax-errors=10 -pedantic -pedantic-errors -Waliasing -Wam > persand -Wcharacter-truncation -Wline-truncation -Wsurprising -Wno-tabs -Wunderflow -ffpe-trap=invalid,zero,overflow conftest.f90 >&5 > mpif.h:16.18: > Included at conftest.f90:4: > > CHARACTER*1 MPI_ARGVS_NULL(1,1) > 1 > Warning: Obsolescent feature: Old-style character length at (1) > mpif.h:17.18: > Included at conftest.f90:4: > > CHARACTER*1 MPI_ARGV_NULL(1) > 1 > Warning: Obsolescent feature: Old-style character length at (1) > mpif.h:528.16: > Included at conftest.f90:4: > > integer*8 MPI_DISPLACEMENT_CURRENT > 1 > Error: GNU Extension: Nonstandard type declaration INTEGER*8 at (1) > mpif.h:529.42: > Included at conftest.f90:4: > > PARAMETER (MPI_DISPLACEMENT_CURRENT=-54278278) > 1 > Error: Symbol 'mpi_displacement_current' at (1) has no IMPLICIT type > mpif.h:546.13: > Included at conftest.f90:4: > > REAL*8 MPI_WTIME, MPI_WTICK > 1 > Error: GNU Extension: Nonstandard type declaration REAL*8 at (1) > mpif.h:547.13: > Included at conftest.f90:4: > > REAL*8 PMPI_WTIME, PMPI_WTICK > 1 > Error: GNU Extension: Nonstandard type declaration REAL*8 at (1) > mpif.h:550.18: > Included at conftest.f90:4: > > CHARACTER*1 PADS_A(3), PADS_B(3) > 1 > Warning: Obsolescent feature: Old-style character length at (1) > configure:6665: $? = 1 > configure: failed program was: > | > | program main > | > | include 'mpif.h' > | integer, parameter :: CUSTOM_MPI_TYPE = MPI_REAL > | integer ier > | call MPI_INIT(ier) > | call MPI_BARRIER(MPI_COMM_WORLD,ier) > | call MPI_FINALIZE(ier) > | > | end > | > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0