December 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPI_Send and recv
by Matthieu Dorier 06 Dec '13

06 Dec '13

That's another idea. The messages I'm talking about are less than 256 bytes. Matthieu ----- Mail original ----- > De: "Jeff Hammond" <jeff.science(a)gmail.com> > À: discuss(a)mpich.org > Envoyé: Vendredi 6 Décembre 2013 14:10:15 > Objet: Re: [mpich-discuss] MPI_Send and recv > > For large messages, MPI_Send may behave like MPI_Ssend on Blue Gene > and probably other platforms. You have to specify the message sizes > your interested in for me to provide more accurate information. > > If you don't want data transfer to block, just use very small > Send-Recv for control and move data with RMA. > > Jeff > > On Fri, Dec 6, 2013 at 12:46 AM, Matthieu Dorier > <matthieu.dorier(a)irisa.fr> wrote: > > Thanks Jeff for your answer. > > By "systematically" I mean "whether a remote buffer is full or not" > > (i.e. > > "every time"), as opposed to an MPI_Send that "may block under some > > conditions, but not always". > > > > Matthieu > > > > ________________________________ > > > > De: "Jeff Hammond" <jeff.science(a)gmail.com> > > À: discuss(a)mpich.org > > Envoyé: Vendredi 6 Décembre 2013 03:34:00 > > > > Objet: Re: [mpich-discuss] MPI_Send and recv > > > > Blue Gene does not always block on mpi_send. I don't know what you > > mean by > > "systematically"... > > > > If you are sending small messages (less than ~1KB), they will use > > eager > > protocol and not block until you blow up the remote buffer. > > > > Rendezvous very likely blocks until the recv is posted. > > > > There are slides online about Cray MPI protocols that may show > > similar > > behavior. > > > > If you want to be sure you don't block on send, you need to use > > isend. That > > much should be obvious. > > > > Jeff > > > > Sent from my iPhone > > > > On Dec 5, 2013, at 2:59 PM, Matthieu Dorier > > <matthieu.dorier(a)irisa.fr> > > wrote: > > > > Hi, > > > > I come back to this old discussion, regarding MPI_Send potentially > > blocking > > until a matching MPI_Recv is posted. > > > > I happen to write a master-worker type program where workers > > periodically > > send progress status to the master through two MPI_Send. > > The master has an MPI_Irecv posted for the first one and when this > > one > > completes, it posts an MPI_Recv for the second one. Those messages > > being > > small and the worker not expecting any answer before continuing to > > work, you > > can imagine that a systematically blocking MPI_Send could degrade > > the > > performance if the master is busy doing something else. > > > > So my question is: to your knowledge, do implementations of MPI (in > > particular Mpich, and those running on Cray Linux and BlueGene/P > > and Q) > > systematically block on MPI_Send? (I would understand if it blocked > > because > > too many MPI_Send have not been matched on the receiver) > > > > Otherwise I might have to use MPI_Isend everywhere... > > > > Thanks! > > > > Matthieu > > > > ________________________________ > > > > De: "Wesley Bland" <wbland(a)mcs.anl.gov> > > À: discuss(a)mpich.org > > Envoyé: Mercredi 3 Juillet 2013 15:40:11 > > Objet: Re: [mpich-discuss] MPI_Send and recv > > > > On Jul 2, 2013, at 9:03 PM, Matthieu Dorier > > <matthieu.dorier(a)irisa.fr> > > wrote: > > > > Hi, > > > > These messages piqued my curiosity; > > > > If I have two processes, process 0 does an MPI_Send to process 1 > > and > > eventually, process 1 posts a matching MPI_Recv or MPI_Irecv; does > > process 0 > > waits for the MPI_Recv/Irecv to be posted before returning from the > > MPI_Send? (in other words if process 0 does 2 MPI_Send and process > > 1 does 2 > > MPI_Recv, is the linearization "Send; Send; Recv; Recv" possible?) > > > > > > It is possible that process 0 could return from the MPI_Send before > > process > > 1 posts the corresponding MPI_Recv, but it's not guaranteed. The > > only thing > > that is guaranteed from the standard is that when MPI_Send returns, > > the > > buffer is available to use again. It is valid to wait for the > > MPI_Recv to > > post as well. So to answer your example, S - S - R - R is valid, as > > is S - R > > - S - R. > > > > > > Second question: in the situation described by Sufeng, where a > > process does > > a MPI_Send to itself, MPI has enough information to see that the > > process is > > sending to itself while it has not posted any matching receive, why > > would > > MPI block instead of printing an error and aborting? Is it because > > in a > > multithreaded program, the matching receive could be posted by > > another > > thread? (Yet in that case I always thought it was unsafe to use the > > same > > communicator from multiple threads…) > > > > > > There are probably many reasons why MPI doesn't try to do internal > > error > > checking. The first is performance. If a program is correct, then > > checking > > for all of the cases that might cause a deadlock would be not only > > very > > difficult, but also expensive. Therefore, we just say that the user > > needs to > > write a correct program and MPI makes no guarantees that anything > > smart will > > happen in the background. It's possible that someone could write > > Nice-MPI > > that does lots of error checking and verification, but I would > > think that > > most users wouldn't want it as it would add overhead. > > > > Second, it's not always unsafe to use the same communicator in > > multiple > > threads. If you use different tags for each thread, then it > > shouldn't be a > > problem to communicate between them. > > > > Wesley > > > > > > Thanks, > > > > Matthieu > > > > ________________________________ > > > > De: "Sufeng Niu" <sniu(a)hawk.iit.edu> > > À: "Pavan Balaji" <balaji(a)mcs.anl.gov> > > Cc: discuss(a)mpich.org > > Envoyé: Mardi 2 Juillet 2013 17:20:25 > > Objet: Re: [mpich-discuss] MPI_Send and recv > > > > Thank you so much! Now I make sense > > > > Sufeng > > > > > > On Tue, Jul 2, 2013 at 5:19 PM, Pavan Balaji <balaji(a)mcs.anl.gov> > > wrote: > >> > >> > >> On 07/02/2013 05:18 PM, Sufeng Niu wrote: > >>> > >>> Thanks a lot, that is most confusing part for me in MPI. I would > >>> like to > >>> clear this type of problems. > >>> > >>> So the blocking begins to send message to itself, but there is no > >>> receive posted. Thus, the MPI block it. Am I right here? > >> > >> > >> Correct. > >> > >> > >>> Thus, I need a nonblocking receive to wait the message here. Can > >>> I use > >>> blocking receive? > >> > >> > >> If you use blocking receive, you'll block in the receive; how will > >> you > >> reach the send? > >> > >> -- Pavan > >> > >> > >> -- > >> Pavan Balaji > >> http://www.mcs.anl.gov/~balaji > > > > > > > > > > -- > > Best Regards, > > Sufeng Niu > > ECASP lab, ECE department, Illinois Institute of Technology > > Tel: 312-731-7219 > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpifort wrapper compiler?
by Jeff Hammond 06 Dec '13

06 Dec '13

>> You said there haven't been F77 compilers for 30 years. This assumes >> backwards compatibility of Fortran completely non-conflicting >> features. I am asserting this is false. > > I think we're getting into finer points, and I think we're both (at least essentially) right: remember that F77 restricted symbols to ***6 characters in length***. Hence, MPI has *never* been compliant with Fortran 77. And if you can find a fortran compiler that runs on a system built within the last 10 years that rejects 7 character symbols without any extra CLI options, well, then, more power to you. :-) I don't think you understand how serious IBM is about language standards. If you ask them for a Fortran 77 compile, they will give you a Fortran 77 compiler: [jhammond@vestalac1 FORTRAN]$ cat hello77.f program hello implicit none integer x double precision i integer JeffSquyresIsUnfortunatelyRightSometimes print*,'this is ', & ' a Fortran 77 program' end program [jhammond@vestalac1 FORTRAN]$ bgxlf -qlanglvl=77std hello77.f "hello77.f", line 2.6: 1518-285 (L) The IMPLICIT NONE statement is not permitted by the FORTRAN 77 standard. "hello77.f", line 5.15: 1518-253 (L) The length of an identifier must not exceed 6 characters in the FORTRAN 77 standard. "hello77.f", line 8.6: 1518-285 (L) The END PROGRAM statement is not permitted by the FORTRAN 77 standard. "hello77.f", 1518-264 (L) Lowercase characters have been detected in the source. Lowercase characters are not permitted by the FORTRAN 77 standard. "hello77.f", 1518-317 (L) The POSITION option can cause the compiler to accept code that does not conform to the FORTRAN 77 standard. ** hello === End of Compilation 1 === 1501-510 Compilation successful for file hello77.f. I recall that one can set the warning threshold such that this compilation will fail. Of course, by default, you will not see this strict behavior. > However, your overall point may still stand: > >> I'm going to write some tests and get back to you. But I know for a >> fact that the IBM XLF compiler chokes on F77 comment style when it is >> compiling in F90+ mode. > > If this is correct (and doesn't just rely on the .suffix), then yes, your overall point is correct -- a single "mpifort" wrapper compiler may not be sufficient on such platforms (without putting at least a little more knowledge in the wrapper -- which doesn't sound desirable). In such cases, I would think that you either need 2 MPICH installations (because, at least in one light, you could look at those incompatible Fortran compilers as actually different compilers), or you need to make an exception on that platform for mpif77 to be different than mpifort. The problem here is that one cannot rely upon filename suffixes to detect language features. First, this is not specified as something one can rely upon in the Fortran standard (or if it is, it is stated without using any of the words I thought to search for). Second, it is not even convention [http://fortranwiki.org/fortran/show/source+conventions#filename_suffix] The IBM XLF compiler can compile any valid Fortran code by specifying the language requested explicitly with a flag. I suppose I can just do the following: alias mpif77='mpifort -gimme-f77' alias mpif90='mpifort -gimme-f90' alias mpif95='mpifort -gimme-f95' alias mpif03='mpifort -gimme-f03' alias mpif08='mpifort -gimme-f08' I just find this annoying since it adds a previously unnecessary step to users' workflow and breaks backwards compatibility for anyone that relies upon their MPI implementation to provide mpif77. Jeff -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] MPI_Send and recv
by Jeff Hammond 06 Dec '13

06 Dec '13

For large messages, MPI_Send may behave like MPI_Ssend on Blue Gene and probably other platforms. You have to specify the message sizes your interested in for me to provide more accurate information. If you don't want data transfer to block, just use very small Send-Recv for control and move data with RMA. Jeff On Fri, Dec 6, 2013 at 12:46 AM, Matthieu Dorier <matthieu.dorier(a)irisa.fr> wrote: > Thanks Jeff for your answer. > By "systematically" I mean "whether a remote buffer is full or not" (i.e. > "every time"), as opposed to an MPI_Send that "may block under some > conditions, but not always". > > Matthieu > > ________________________________ > > De: "Jeff Hammond" <jeff.science(a)gmail.com> > À: discuss(a)mpich.org > Envoyé: Vendredi 6 Décembre 2013 03:34:00 > > Objet: Re: [mpich-discuss] MPI_Send and recv > > Blue Gene does not always block on mpi_send. I don't know what you mean by > "systematically"... > > If you are sending small messages (less than ~1KB), they will use eager > protocol and not block until you blow up the remote buffer. > > Rendezvous very likely blocks until the recv is posted. > > There are slides online about Cray MPI protocols that may show similar > behavior. > > If you want to be sure you don't block on send, you need to use isend. That > much should be obvious. > > Jeff > > Sent from my iPhone > > On Dec 5, 2013, at 2:59 PM, Matthieu Dorier <matthieu.dorier(a)irisa.fr> > wrote: > > Hi, > > I come back to this old discussion, regarding MPI_Send potentially blocking > until a matching MPI_Recv is posted. > > I happen to write a master-worker type program where workers periodically > send progress status to the master through two MPI_Send. > The master has an MPI_Irecv posted for the first one and when this one > completes, it posts an MPI_Recv for the second one. Those messages being > small and the worker not expecting any answer before continuing to work, you > can imagine that a systematically blocking MPI_Send could degrade the > performance if the master is busy doing something else. > > So my question is: to your knowledge, do implementations of MPI (in > particular Mpich, and those running on Cray Linux and BlueGene/P and Q) > systematically block on MPI_Send? (I would understand if it blocked because > too many MPI_Send have not been matched on the receiver) > > Otherwise I might have to use MPI_Isend everywhere... > > Thanks! > > Matthieu > > ________________________________ > > De: "Wesley Bland" <wbland(a)mcs.anl.gov> > À: discuss(a)mpich.org > Envoyé: Mercredi 3 Juillet 2013 15:40:11 > Objet: Re: [mpich-discuss] MPI_Send and recv > > On Jul 2, 2013, at 9:03 PM, Matthieu Dorier <matthieu.dorier(a)irisa.fr> > wrote: > > Hi, > > These messages piqued my curiosity; > > If I have two processes, process 0 does an MPI_Send to process 1 and > eventually, process 1 posts a matching MPI_Recv or MPI_Irecv; does process 0 > waits for the MPI_Recv/Irecv to be posted before returning from the > MPI_Send? (in other words if process 0 does 2 MPI_Send and process 1 does 2 > MPI_Recv, is the linearization "Send; Send; Recv; Recv" possible?) > > > It is possible that process 0 could return from the MPI_Send before process > 1 posts the corresponding MPI_Recv, but it's not guaranteed. The only thing > that is guaranteed from the standard is that when MPI_Send returns, the > buffer is available to use again. It is valid to wait for the MPI_Recv to > post as well. So to answer your example, S - S - R - R is valid, as is S - R > - S - R. > > > Second question: in the situation described by Sufeng, where a process does > a MPI_Send to itself, MPI has enough information to see that the process is > sending to itself while it has not posted any matching receive, why would > MPI block instead of printing an error and aborting? Is it because in a > multithreaded program, the matching receive could be posted by another > thread? (Yet in that case I always thought it was unsafe to use the same > communicator from multiple threads…) > > > There are probably many reasons why MPI doesn't try to do internal error > checking. The first is performance. If a program is correct, then checking > for all of the cases that might cause a deadlock would be not only very > difficult, but also expensive. Therefore, we just say that the user needs to > write a correct program and MPI makes no guarantees that anything smart will > happen in the background. It's possible that someone could write Nice-MPI > that does lots of error checking and verification, but I would think that > most users wouldn't want it as it would add overhead. > > Second, it's not always unsafe to use the same communicator in multiple > threads. If you use different tags for each thread, then it shouldn't be a > problem to communicate between them. > > Wesley > > > Thanks, > > Matthieu > > ________________________________ > > De: "Sufeng Niu" <sniu(a)hawk.iit.edu> > À: "Pavan Balaji" <balaji(a)mcs.anl.gov> > Cc: discuss(a)mpich.org > Envoyé: Mardi 2 Juillet 2013 17:20:25 > Objet: Re: [mpich-discuss] MPI_Send and recv > > Thank you so much! Now I make sense > > Sufeng > > > On Tue, Jul 2, 2013 at 5:19 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >> >> >> On 07/02/2013 05:18 PM, Sufeng Niu wrote: >>> >>> Thanks a lot, that is most confusing part for me in MPI. I would like to >>> clear this type of problems. >>> >>> So the blocking begins to send message to itself, but there is no >>> receive posted. Thus, the MPI block it. Am I right here? >> >> >> Correct. >> >> >>> Thus, I need a nonblocking receive to wait the message here. Can I use >>> blocking receive? >> >> >> If you use blocking receive, you'll block in the receive; how will you >> reach the send? >> >> -- Pavan >> >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji > > > > > -- > Best Regards, > Sufeng Niu > ECASP lab, ECE department, Illinois Institute of Technology > Tel: 312-731-7219 > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] mpifort wrapper compiler?
by Jeff Squyres (jsquyres) 06 Dec '13

06 Dec '13

On Dec 6, 2013, at 8:00 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> I don't understand the point you're trying to make -- please explain? > > You said there haven't been F77 compilers for 30 years. This assumes > backwards compatibility of Fortran completely non-conflicting > features. I am asserting this is false. I think we're getting into finer points, and I think we're both (at least essentially) right: remember that F77 restricted symbols to ***6 characters in length***. Hence, MPI has *never* been compliant with Fortran 77. And if you can find a fortran compiler that runs on a system built within the last 10 years that rejects 7 character symbols without any extra CLI options, well, then, more power to you. :-) However, your overall point may still stand: > I'm going to write some tests and get back to you. But I know for a > fact that the IBM XLF compiler chokes on F77 comment style when it is > compiling in F90+ mode. If this is correct (and doesn't just rely on the .suffix), then yes, your overall point is correct -- a single "mpifort" wrapper compiler may not be sufficient on such platforms (without putting at least a little more knowledge in the wrapper -- which doesn't sound desirable). In such cases, I would think that you either need 2 MPICH installations (because, at least in one light, you could look at those incompatible Fortran compilers as actually different compilers), or you need to make an exception on that platform for mpif77 to be different than mpifort. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpich-discuss] mpifort wrapper compiler?
by Jeff Hammond 06 Dec '13

06 Dec '13

On Fri, Dec 6, 2013 at 5:55 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: > On Dec 5, 2013, at 7:48 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > >>> Keep in mind that there have not been any "Fortran 77" compilers for over 30 years. >>> The distinction is artificial and arbitrary... >> >> I suppose you'd say we no longer need a C89 compiler as long as we have a C++03 compiler? >> >> Do _strictly_ F2008 compilers support Hollerith constants, entry points, and computed goto? > > I don't understand the point you're trying to make -- please explain? You said there haven't been F77 compilers for 30 years. This assumes backwards compatibility of Fortran completely non-conflicting features. I am asserting this is false. I'm going to write some tests and get back to you. But I know for a fact that the IBM XLF compiler chokes on F77 comment style when it is compiling in F90+ mode. Jeff > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-discuss] mpifort wrapper compiler?
by Jeff Squyres (jsquyres) 06 Dec '13

06 Dec '13

On Dec 5, 2013, at 7:48 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> Keep in mind that there have not been any "Fortran 77" compilers for over 30 years. >> The distinction is artificial and arbitrary... > > I suppose you'd say we no longer need a C89 compiler as long as we have a C++03 compiler? > > Do _strictly_ F2008 compilers support Hollerith constants, entry points, and computed goto? I don't understand the point you're trying to make -- please explain? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpich-discuss] MPI_Send and recv
by Matthieu Dorier 06 Dec '13

06 Dec '13

Thanks Jeff for your answer. By "systematically" I mean "whether a remote buffer is full or not" (i.e. "every time"), as opposed to an MPI_Send that "may block under some conditions, but not always". Matthieu ----- Mail original ----- > De: "Jeff Hammond" <jeff.science(a)gmail.com> > À: discuss(a)mpich.org > Envoyé: Vendredi 6 Décembre 2013 03:34:00 > Objet: Re: [mpich-discuss] MPI_Send and recv > Blue Gene does not always block on mpi_send. I don't know what you > mean by "systematically"... > If you are sending small messages (less than ~1KB), they will use > eager protocol and not block until you blow up the remote buffer. > Rendezvous very likely blocks until the recv is posted. > There are slides online about Cray MPI protocols that may show > similar behavior. > If you want to be sure you don't block on send, you need to use > isend. That much should be obvious. > Jeff > Sent from my iPhone > On Dec 5, 2013, at 2:59 PM, Matthieu Dorier < > matthieu.dorier(a)irisa.fr > wrote: > > Hi, > > > I come back to this old discussion, regarding MPI_Send potentially > > blocking until a matching MPI_Recv is posted. > > > I happen to write a master-worker type program where workers > > periodically send progress status to the master through two > > MPI_Send. > > > The master has an MPI_Irecv posted for the first one and when this > > one completes, it posts an MPI_Recv for the second one. Those > > messages being small and the worker not expecting any answer before > > continuing to work, you can imagine that a systematically blocking > > MPI_Send could degrade the performance if the master is busy doing > > something else. > > > So my question is: to your knowledge, do implementations of MPI (in > > particular Mpich, and those running on Cray Linux and BlueGene/P > > and > > Q) systematically block on MPI_Send? (I would understand if it > > blocked because too many MPI_Send have not been matched on the > > receiver) > > > Otherwise I might have to use MPI_Isend everywhere... > > > Thanks! > > > Matthieu > > > ----- Mail original ----- > > > > De: "Wesley Bland" < wbland(a)mcs.anl.gov > > > > > > > À: discuss(a)mpich.org > > > > > > Envoyé: Mercredi 3 Juillet 2013 15:40:11 > > > > > > Objet: Re: [mpich-discuss] MPI_Send and recv > > > > > > On Jul 2, 2013, at 9:03 PM, Matthieu Dorier < > > > matthieu.dorier(a)irisa.fr > wrote: > > > > > > > Hi, > > > > > > > > > > These messages piqued my curiosity; > > > > > > > > > > If I have two processes, process 0 does an MPI_Send to process > > > > 1 > > > > and > > > > eventually, process 1 posts a matching MPI_Recv or MPI_Irecv; > > > > does > > > > process 0 waits for the MPI_Recv/Irecv to be posted before > > > > returning > > > > from the MPI_Send? (in other words if process 0 does 2 MPI_Send > > > > and > > > > process 1 does 2 MPI_Recv, is the linearization "Send; Send; > > > > Recv; > > > > Recv" possible?) > > > > > > > > > It is possible that process 0 could return from the MPI_Send > > > before > > > process 1 posts the corresponding MPI_Recv, but it's not > > > guaranteed. > > > The only thing that is guaranteed from the standard is that when > > > MPI_Send returns, the buffer is available to use again. It is > > > valid > > > to wait for the MPI_Recv to post as well. So to answer your > > > example, > > > S - S - R - R is valid, as is S - R - S - R. > > > > > > > Second question: in the situation described by Sufeng, where a > > > > process does a MPI_Send to itself, MPI has enough information > > > > to > > > > see > > > > that the process is sending to itself while it has not posted > > > > any > > > > matching receive, why would MPI block instead of printing an > > > > error > > > > and aborting? Is it because in a multithreaded program, the > > > > matching > > > > receive could be posted by another thread? (Yet in that case I > > > > always thought it was unsafe to use the same communicator from > > > > multiple threads…) > > > > > > > > > There are probably many reasons why MPI doesn't try to do > > > internal > > > error checking. The first is performance. If a program is > > > correct, > > > then checking for all of the cases that might cause a deadlock > > > would > > > be not only very difficult, but also expensive. Therefore, we > > > just > > > say that the user needs to write a correct program and MPI makes > > > no > > > guarantees that anything smart will happen in the background. > > > It's > > > possible that someone could write Nice-MPI that does lots of > > > error > > > checking and verification, but I would think that most users > > > wouldn't want it as it would add overhead. > > > > > > Second, it's not always unsafe to use the same communicator in > > > multiple threads. If you use different tags for each thread, then > > > it > > > shouldn't be a problem to communicate between them. > > > > > > Wesley > > > > > > > Thanks, > > > > > > > > > > Matthieu > > > > > > > > > > ----- Mail original ----- > > > > > > > > > > > De: "Sufeng Niu" < sniu(a)hawk.iit.edu > > > > > > > > > > > > > > > > À: "Pavan Balaji" < balaji(a)mcs.anl.gov > > > > > > > > > > > > > > > > Cc: discuss(a)mpich.org > > > > > > > > > > > > > > > Envoyé: Mardi 2 Juillet 2013 17:20:25 > > > > > > > > > > > > > > > Objet: Re: [mpich-discuss] MPI_Send and recv > > > > > > > > > > > > > > > Thank you so much! Now I make sense > > > > > > > > > > > > > > > Sufeng > > > > > > > > > > > > > > > On Tue, Jul 2, 2013 at 5:19 PM, Pavan Balaji < > > > > > balaji(a)mcs.anl.gov > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > On 07/02/2013 05:18 PM, Sufeng Niu wrote: > > > > > > > > > > > > > > > > > > > > > > Thanks a lot, that is most confusing part for me in MPI. > > > > > > > I > > > > > > > would > > > > > > > like > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > > > clear this type of problems. > > > > > > > > > > > > > > > > > > > > > > > > > > > > So the blocking begins to send message to itself, but > > > > > > > there > > > > > > > is > > > > > > > no > > > > > > > > > > > > > > > > > > > > > > > > > > > > receive posted. Thus, the MPI block it. Am I right here? > > > > > > > > > > > > > > > > > > > > > > > > > > > Correct. > > > > > > > > > > > > > > > > > > > > > > Thus, I need a nonblocking receive to wait the message > > > > > > > here. > > > > > > > Can > > > > > > > I > > > > > > > use > > > > > > > > > > > > > > > > > > > > > > > > > > > > blocking receive? > > > > > > > > > > > > > > > > > > > > > > > > > > > If you use blocking receive, you'll block in the receive; > > > > > > how > > > > > > will > > > > > > you reach the send? > > > > > > > > > > > > > > > > > > > > > -- Pavan > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > Pavan Balaji > > > > > > > > > > > > > > > > > > > > > http://www.mcs.anl.gov/~balaji > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > Best Regards, > > > > > > > > > > > > > > > Sufeng Niu > > > > > > > > > > > > > > > ECASP lab, ECE department, Illinois Institute of Technology > > > > > > > > > > > > > > > Tel: 312-731-7219 > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > > > > discuss mailing list discuss(a)mpich.org > > > > > > > > > > > > > > > To manage subscription options or unsubscribe: > > > > > > > > > > > > > > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > discuss mailing list discuss(a)mpich.org > > > > > > > > > > To manage subscription options or unsubscribe: > > > > > > > > > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > > _______________________________________________ > > > > > > discuss mailing list discuss(a)mpich.org > > > > > > To manage subscription options or unsubscribe: > > > > > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > _______________________________________________ > > > discuss mailing list discuss(a)mpich.org > > > To manage subscription options or unsubscribe: > > > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_Send and recv
by Jeff Hammond 05 Dec '13

05 Dec '13

Blue Gene does not always block on mpi_send. I don't know what you mean by "systematically"... If you are sending small messages (less than ~1KB), they will use eager protocol and not block until you blow up the remote buffer. Rendezvous very likely blocks until the recv is posted. There are slides online about Cray MPI protocols that may show similar behavior. If you want to be sure you don't block on send, you need to use isend. That much should be obvious. Jeff Sent from my iPhone > On Dec 5, 2013, at 2:59 PM, Matthieu Dorier <matthieu.dorier(a)irisa.fr> wrote: > > Hi, > > I come back to this old discussion, regarding MPI_Send potentially blocking until a matching MPI_Recv is posted. > > I happen to write a master-worker type program where workers periodically send progress status to the master through two MPI_Send. > The master has an MPI_Irecv posted for the first one and when this one completes, it posts an MPI_Recv for the second one. Those messages being small and the worker not expecting any answer before continuing to work, you can imagine that a systematically blocking MPI_Send could degrade the performance if the master is busy doing something else. > > So my question is: to your knowledge, do implementations of MPI (in particular Mpich, and those running on Cray Linux and BlueGene/P and Q) systematically block on MPI_Send? (I would understand if it blocked because too many MPI_Send have not been matched on the receiver) > > Otherwise I might have to use MPI_Isend everywhere... > > Thanks! > > Matthieu > > De: "Wesley Bland" <wbland(a)mcs.anl.gov> > À: discuss(a)mpich.org > Envoyé: Mercredi 3 Juillet 2013 15:40:11 > Objet: Re: [mpich-discuss] MPI_Send and recv > > On Jul 2, 2013, at 9:03 PM, Matthieu Dorier <matthieu.dorier(a)irisa.fr> wrote: > > Hi, > > These messages piqued my curiosity; > > If I have two processes, process 0 does an MPI_Send to process 1 and eventually, process 1 posts a matching MPI_Recv or MPI_Irecv; does process 0 waits for the MPI_Recv/Irecv to be posted before returning from the MPI_Send? (in other words if process 0 does 2 MPI_Send and process 1 does 2 MPI_Recv, is the linearization "Send; Send; Recv; Recv" possible?) > > It is possible that process 0 could return from the MPI_Send before process 1 posts the corresponding MPI_Recv, but it's not guaranteed. The only thing that is guaranteed from the standard is that when MPI_Send returns, the buffer is available to use again. It is valid to wait for the MPI_Recv to post as well. So to answer your example, S - S - R - R is valid, as is S - R - S - R. > > > Second question: in the situation described by Sufeng, where a process does a MPI_Send to itself, MPI has enough information to see that the process is sending to itself while it has not posted any matching receive, why would MPI block instead of printing an error and aborting? Is it because in a multithreaded program, the matching receive could be posted by another thread? (Yet in that case I always thought it was unsafe to use the same communicator from multiple threads…) > > There are probably many reasons why MPI doesn't try to do internal error checking. The first is performance. If a program is correct, then checking for all of the cases that might cause a deadlock would be not only very difficult, but also expensive. Therefore, we just say that the user needs to write a correct program and MPI makes no guarantees that anything smart will happen in the background. It's possible that someone could write Nice-MPI that does lots of error checking and verification, but I would think that most users wouldn't want it as it would add overhead. > > Second, it's not always unsafe to use the same communicator in multiple threads. If you use different tags for each thread, then it shouldn't be a problem to communicate between them. > > Wesley > > > Thanks, > > Matthieu > > De: "Sufeng Niu" <sniu(a)hawk.iit.edu> > À: "Pavan Balaji" <balaji(a)mcs.anl.gov> > Cc: discuss(a)mpich.org > Envoyé: Mardi 2 Juillet 2013 17:20:25 > Objet: Re: [mpich-discuss] MPI_Send and recv > > Thank you so much! Now I make sense > > Sufeng > > >> On Tue, Jul 2, 2013 at 5:19 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >> >>> On 07/02/2013 05:18 PM, Sufeng Niu wrote: >>> Thanks a lot, that is most confusing part for me in MPI. I would like to >>> clear this type of problems. >>> >>> So the blocking begins to send message to itself, but there is no >>> receive posted. Thus, the MPI block it. Am I right here? >> >> Correct. >> >> >>> Thus, I need a nonblocking receive to wait the message here. Can I use >>> blocking receive? >> >> If you use blocking receive, you'll block in the receive; how will you reach the send? >> >> -- Pavan >> >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji > > > > -- > Best Regards, > Sufeng Niu > ECASP lab, ECE department, Illinois Institute of Technology > Tel: 312-731-7219 > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpifort wrapper compiler?
by Jeff Hammond 05 Dec '13

05 Dec '13

I recall offhand that .F is for code requiring preprocessor and .f not. I recall numerically-adorned suffixes do not do anything useful but I'll need to verify that. Jeff Sent from my iPhone > On Dec 5, 2013, at 10:59 AM, "Jeff Squyres (jsquyres)" <jsquyres(a)cisco.com> wrote: > > I'm unfamiliar with the xl suite -- don't they have xlf that handles .f, .f90, and .F90 source files? > > >> On Dec 5, 2013, at 11:57 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >> >> >>> On Dec 5, 2013, at 10:43 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: >>> >>>> On Dec 5, 2013, at 10:22 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>>> >>>> It is important to separate out the F77 wrapper from the F90+ one to support the case where the F77 compiler cannot handle F90+ code and vice versa. The IBM compiler is this way because of fixed/free source form handling. >>> >>> I'm not sure I understand this -- my understanding is that compilers differentiate between the different forms of Fortran (fixed vs. free) via the .suffix. >>> >>> This is very much in line with compilers like ifort, pgifort, etc. >> >> xl is in a league of it’s own. :-) >> >> Maybe they’ll fix it in future versions. But we can’t abandon it if that’s the way it is right now. Many users still use those compilers. >> >> — Pavan >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] mpifort wrapper compiler?
by Jeff Hammond 05 Dec '13

05 Dec '13

> On Dec 5, 2013, at 10:57 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > >> On Dec 5, 2013, at 10:43 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: >> >>> On Dec 5, 2013, at 10:22 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>> >>> It is important to separate out the F77 wrapper from the F90+ one to support the case where the F77 compiler cannot handle F90+ code and vice versa. The IBM compiler is this way because of fixed/free source form handling. >> >> I'm not sure I understand this -- my understanding is that compilers differentiate between the different forms of Fortran (fixed vs. free) via the .suffix. >> >> This is very much in line with compilers like ifort, pgifort, etc. > > xl is in a league of it’s own. :-) > > Maybe they’ll fix it in future versions. But we can’t abandon it if that’s the way it is right now. Many users still use those compilers. > There is nothing to be fixed. XLF is extremely strict about language syntax. You cannot give it source that isn't compliant with one and only one Fortran standard. People that don't use XLF on a daily basis should not make wild generalizations based upon Intel and GCC, both of which are notorious sleezeballs when it comes to language specification enforcement. For example, gfortran will accept inline C syntax in some cases... Jeff > — Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0