June 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Non-blocking collectives
by rreddypsc＠gmail.com 29 Jun '13

29 Jun '13

This is a very good example! Thanks for the clarification! Thanks, Raghu --On Saturday, June 29, 2013 3:08 PM -0500 Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > Yes. > > On Jun 29, 2013, at 3:05 PM, Jed Brown wrote: > >> Rajeev Thakur <thakur(a)mcs.anl.gov> writes: >> >>> It doesn't have to be a matching Wait. The implementation is required >>> to make progress on other pending communication while it is blocked in >>> a Wait. >> >> Or blocked in any of a number of other places, if non-blocking >> collective requests have the same completion semantics as point-to-point >> requests. E.g., this shouldn't deadlock: >> >> MPI_Iallreduce(...,COMM_WORLD,&req); >> if (!rank) MPI_Wait(&req); >> MPI_Allgather(...,COMM_WORLD); >> if (rank) MPI_Wait(&req); > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Non-blocking collectives
by Rajeev Thakur 29 Jun '13

29 Jun '13

Yes. On Jun 29, 2013, at 3:05 PM, Jed Brown wrote: > Rajeev Thakur <thakur(a)mcs.anl.gov> writes: > >> It doesn't have to be a matching Wait. The implementation is required >> to make progress on other pending communication while it is blocked in >> a Wait. > > Or blocked in any of a number of other places, if non-blocking > collective requests have the same completion semantics as point-to-point > requests. E.g., this shouldn't deadlock: > > MPI_Iallreduce(...,COMM_WORLD,&req); > if (!rank) MPI_Wait(&req); > MPI_Allgather(...,COMM_WORLD); > if (rank) MPI_Wait(&req);

1 0

Re: [mpich-discuss] Non-blocking collectives
by Jed Brown 29 Jun '13

29 Jun '13

Rajeev Thakur <thakur(a)mcs.anl.gov> writes: > It doesn't have to be a matching Wait. The implementation is required > to make progress on other pending communication while it is blocked in > a Wait. Or blocked in any of a number of other places, if non-blocking collective requests have the same completion semantics as point-to-point requests. E.g., this shouldn't deadlock: MPI_Iallreduce(...,COMM_WORLD,&req); if (!rank) MPI_Wait(&req); MPI_Allgather(...,COMM_WORLD); if (rank) MPI_Wait(&req);

1 0

Re: [mpich-discuss] Non-blocking collectives
by Rajeev Thakur 29 Jun '13

29 Jun '13

It doesn't have to be a matching Wait. The implementation is required to make progress on other pending communication while it is blocked in a Wait. On Jun 29, 2013, at 2:43 PM, Jed Brown wrote: > Rajeev Thakur <thakur(a)mcs.anl.gov> writes: > >> I think the Wait is allowed to wait until others call Wait. > > Do you read this as allowing an implementation to defer completion until > a _matching_ MPI_Wait is called, or merely until the application > re-enters MPI somehow (sufficiently many times, or blocking)? I had the > latter interpretation, which would allow non-blocking operations on > different communicators to be used without risking deadlock. > > If the intent was really that the implementation can block until a > _matching_ MPI_Wait is called, then I think the standard should clarify > this point because it risks deadlock in a natural use case. > >> See the Advice to Users on 197:27-29: >> >> "Users should be aware that implementations are allowed, but not >> required (with exception of MPI_IBARRIER), to synchronize processes >> during the completion of a nonblocking collective operation."

1 0

Re: [mpich-discuss] Non-blocking collectives
by Jed Brown 29 Jun '13

29 Jun '13

Rajeev Thakur <thakur(a)mcs.anl.gov> writes: > I think the Wait is allowed to wait until others call Wait. Do you read this as allowing an implementation to defer completion until a _matching_ MPI_Wait is called, or merely until the application re-enters MPI somehow (sufficiently many times, or blocking)? I had the latter interpretation, which would allow non-blocking operations on different communicators to be used without risking deadlock. If the intent was really that the implementation can block until a _matching_ MPI_Wait is called, then I think the standard should clarify this point because it risks deadlock in a natural use case. > See the Advice to Users on 197:27-29: > > "Users should be aware that implementations are allowed, but not > required (with exception of MPI_IBARRIER), to synchronize processes > during the completion of a nonblocking collective operation."

1 0

Re: [mpich-discuss] Non-blocking collectives
by Rajeev Thakur 29 Jun '13

29 Jun '13

I think the Wait is allowed to wait until others call Wait. See the Advice to Users on 197:27-29: "Users should be aware that implementations are allowed, but not required (with exception of MPI_IBARRIER), to synchronize processes during the completion of a nonblocking collective operation." On Jun 29, 2013, at 1:25 PM, Jed Brown wrote: > rreddypsc(a)gmail.com writes: > >> I apologize for creating a new thread, I have accidentally deleted the >> original message. >> >> Not to split hairs, can you please clarify the following: >> >> After doing a non-blocking collective: >> >> The MPI_Wait would be satisfied as soon as the other ranks in the >> communicator have called the non-blocking collective (and may not yet have >> called MPI_Wait), right? > > Yes. > >> Or do they wait until all of them have called MPI_Wait? > > No, that would make the MPI_Wait synchronizing, mostly defeating any > potential benefit of non-blocking collectives. Note that the > non-blocking request can complete in other ways, e.g., via MPI_Test. > >> On the contrary, if one of the ranks calls MPI_Wait (without calling the >> non-blocking collective, which is an incorrect program), will the MPI_Wait >> be satisfied? > > This doesn't even make sense. Where would you be getting an MPI_Request > on which to call MPI_Wait? > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Non-blocking collectives
by Jed Brown 29 Jun '13

29 Jun '13

rreddypsc(a)gmail.com writes: > I apologize for creating a new thread, I have accidentally deleted the > original message. > > Not to split hairs, can you please clarify the following: > > After doing a non-blocking collective: > > The MPI_Wait would be satisfied as soon as the other ranks in the > communicator have called the non-blocking collective (and may not yet have > called MPI_Wait), right? Yes. > Or do they wait until all of them have called MPI_Wait? No, that would make the MPI_Wait synchronizing, mostly defeating any potential benefit of non-blocking collectives. Note that the non-blocking request can complete in other ways, e.g., via MPI_Test. > On the contrary, if one of the ranks calls MPI_Wait (without calling the > non-blocking collective, which is an incorrect program), will the MPI_Wait > be satisfied? This doesn't even make sense. Where would you be getting an MPI_Request on which to call MPI_Wait?

1 0

Non-blocking collectives
by rreddypsc＠gmail.com 29 Jun '13

29 Jun '13

I apologize for creating a new thread, I have accidentally deleted the original message. Not to split hairs, can you please clarify the following: After doing a non-blocking collective: The MPI_Wait would be satisfied as soon as the other ranks in the communicator have called the non-blocking collective (and may not yet have called MPI_Wait), right? Or do they wait until all of them have called MPI_Wait? On the contrary, if one of the ranks calls MPI_Wait (without calling the non-blocking collective, which is an incorrect program), will the MPI_Wait be satisfied? Thanks, Raghu [mpich-discuss] Non-blocking Collectives Jiri Simsa jsimsa at cs.cmu.edu Thu Jun 27 10:33:14 CDT 2013 Previous message: [mpich-discuss] Non-blocking Collectives Next message: [mpich-discuss] PMGR_COLLECTIVE ERROR Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Hi Pavan, To rephrase, I am interested in understanding when would MPI_Wait() block indefinitely, waiting for other process to make progress. I believe that your response answers my question. Thanks again. --Jiri On Wed, Jun 26, 2013 at 4:48 PM, Pavan Balaji <balaji at mcs.anl.gov> wrote: > Hi Jiri, > > > On 06/26/2013 03:08 PM, Jiri Simsa wrote: > >> Thank you for your quick answer. I am trying to understand the blocking >> behavior of MPI_Wait in the case of non-blocking collectives. Is it safe >> to assume that, for a non-blocking collective, MPI_Wait is guaranteed to >> return once all other processes call the corresponding completion >> operation (e.g. MPI_Wait or MPI_Test)? >> > > I'm not sure I understand your question. Are you asking if MPI_WAIT in a > process is guaranteed to return after some finite amount of time after > every other process has called MPI_WAIT? Then, yes. > > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] [CIG-SEISMO] MPICH mpif.h incompatible with gfortran-4.8 -std=f2003
by Jed Brown 29 Jun '13

29 Jun '13

Dimitri Komatitsch <komatitsch(a)lma.cnrs-mrs.fr> writes: > Dear all, > > The (only) three things to do to make mpif.h conform to recent standards > (F90, F95, F2003) would be: > > - change all character* declarations to character(len=...) > > - change all real*8 to either double precision or real(kind=8) (both > are OK) > > - change all integer*8 to integer(kind=8) Well, mpif.h is supposed to be backward-compatible to F77. Upon checking the code, I see that MPICH offers the feature of tolerating -i8 and -r8 options. Open MPI uses 'double precision' for MPI_WTIME, which would be promoted to 16 bytes under -r8, so it won't behave correctly if the user expects a real*8 (perhaps specified as real(kind=8) [1]). The MPI standard says the prototype is DOUBLE PRECISION MPI_WTIME() so I would say that MPICH is non-conforming by using REAL*8, though the standard does not make any statements about what changes when compiled with -r8. I haven't checked to see how MPI data types are mapped in case of -r8. For example, MPI_REAL is defined to a constant in mpif.h, and the MPI library (which was compiled in advance) would not be aware of whether user code was being built with -r8. Is the user expected to use MPI_REAL4 and MPI_REAL8 explicitly in this case? Is -r8 and -i8 tested somewhere in MPICH? What exactly is the user expected to do in order to use this feature? I see this comment in src/binding/f90/mpi_sizeofs.f90.in: ! If reals and doubles have been forced to the same size (e.g., with ! -i8 -r8 to compilers like g95), then the compiler may refuse to ! allow interfaces that use real and double precision (failing to ! determine which one is intended) [1] Note that real(kind=8) is not guaranteed by the Fortran standard to be 8 bytes. For example, g77 and the Salford F95 compilers denote 4-bytes reals with real(kind=1), and 8-byte reals with real(kind=2). http://www.silverfrost.com/manuals/salfordftn95.pdf

1 0

Re: [mpich-discuss] MPICH mpif.h incompatible with gfortran-4.8 -std=f2003
by Jeff Hammond 28 Jun '13

28 Jun '13

On Jun 28, 2013, at 5:56 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Jeff Hammond <jeff.science(a)gmail.com> writes: > >> mpif.h is for F77 codes. Use the module for F90+. There is no point >> in fixing mpif.h for F03 because the solution already exists. > > It would be trivially easy to make work, though probably not to get rid > of the warnings. > Great. We look forward to your patch. >> The F08 bindings in MPI-3 are the long-term solution. > > Yup, and your son will be in college by the time MPI-3 is available on > all supported RHEL versions. ;-) You can compile MPICH 3.0.4 on Linux today. It's trivial. Why wait for RPMs? Those might even exist already too. Did you check the MPICH website? Jeff

1 0