August 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPI_Get on the same memory location
by Brian Van Straalen 23 Aug '14

23 Aug '14

I’d recommend @inproceedings{dotsenko2004multi, title={A multi-platform co-array fortran compiler}, author={Dotsenko, Yuri and Coarfa, Cristian and Mellor-Crummey, John}, booktitle={Proceedings of the 13th International Conference on Parallel Architectures and Compilation Techniques}, pages={29--40}, year={2004}, organization={IEEE Computer Society} } @inproceedings{coarfa2005evaluation, title={An evaluation of global address space languages: co-array fortran and unified parallel C}, author={Coarfa, Cristian and Dotsenko, Yuri and Mellor-Crummey, John and Cantonnet, Fran{\c{c}}ois and El-Ghazawi, Tarek and Mohanti, Ashrujit and Yao, Yiyi and Chavarr{\'\i}a-Miranda, Daniel}, booktitle={Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming}, pages={36--47}, year={2005}, organization={ACM} } Which has a large discussion of aliasing and performance with the Rice CAF compiler and Berkeley UPC compiler Short answer is to a) not go through the communication layer when you are in a shared address space, and b) you have to trick Fortran with dummy variables so it thinks you are not aliasing, or drop directly to load/store instructions when handling accesses that share an address space. You could also look at some messaging layers that are designed specifically to support global address space language development, like GASNet or ARMCI. Brian On Aug 23, 2014, at 7:44 AM, William Gropp <wgropp(a)illinois.edu> wrote: > It is amusing that the original reason for the restriction on aliasing of arguments comes from Fortran - MPI-1 wanted the language-independent spec to be expressible in both Fortran and C, and this is a property of Fortran (and one which aids in performance). > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > University of Illinois Urbana-Champaign > > > > > > On Aug 22, 2014, at 6:04 PM, Brian Van Straalen <bvstraalen(a)lbl.gov> wrote: > >> >> Disabling the check is probably a bad idea. if you disable it in MPI you will still push a call to memcpy that has aliased buffers. For many systems memcpy still does the right thing, but modern fancy memcpy implementations have permission to produce wrong results if the regions overlap. >> >> The MPI spec is clear, and the coarray spec is also clear, and clearly different. I would recommend handling this in your library implementation and leave your MPI build as standard. Other MPI builds will be done standard and you will just be delaying the work. You can’t call MPI_Get with aliased locations. >> >> Brian >> >> >> >> On Aug 22, 2014, at 3:18 PM, Alessandro Fanfarillo <fanfarillo(a)ing.uniroma2.it> wrote: >> >>> Could you tell me which flag disables the check? I would suggest this >>> procedure as temporary workaround. >>> >>> Thanks >>> >>> On Fri, Aug 22, 2014 at 2:41 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>> >>>> On Aug 22, 2014, at 1:43 PM, Brian Van Straalen <bvstraalen(a)lbl.gov> wrote: >>>>> Is there a significant cost to have the MPI implementation provide a check for aliasing? >>>> >>>> We do check for this error by default in MPICH (which is why this email thread started). >>>> >>>> There’s a way to turn off the checking through a configure flag as well as an environment variable, if needed. But that’s essentially saying — I know it’s wrong, but I’m too lazy to fix my application. >>>> >>>> — Pavan >>>> >>>> -- >>>> Pavan Balaji ✉️ >>>> http://www.mcs.anl.gov/~balaji >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> >>> >>> -- >>> >>> Alessandro Fanfarillo >>> Dip. di Ingegneria Civile ed Ingegneria Informatica >>> Università di Roma "Tor Vergata" >>> NCAR Office: +1 (303) 497-2442 >>> Tel: +39-06-7259 7719 >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> Brian Van Straalen Lawrence Berkeley Lab >> BVStraalen(a)lbl.gov Computational Research >> (510) 486-4976 Division (crd.lbl.gov) >> >> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss Brian Van Straalen Lawrence Berkeley Lab BVStraalen(a)lbl.gov Computational Research (510) 486-4976 Division (crd.lbl.gov)

1 0

Re: [mpich-discuss] MPE Profiling?
by Rob Latham 23 Aug '14

23 Aug '14

On 08/23/2014 09:58 AM, William Gropp wrote: > I also have some updates, and they preserve the ability to work with > MPI-2 and MPI-3 implementations. please send them to me, to the list, or to the bug report and I'll integrate them. ==rob > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > University of Illinois Urbana-Champaign > > > > > > On Aug 22, 2014, at 5:07 PM, James <jamesqf(a)charter.net > <mailto:[email protected]>> wrote: > >> That seems to have helped: I'm down from ~1600 lines of error messages >> to under 500. >> >> I'll probably have to do the rest by hand, and will try to make a >> patch file of the changes >> >> James >> >> >> On Fri, 22 Aug 2014 13:49:14 -0700, Balaji, Pavan <balaji(a)anl.gov >> <mailto:[email protected]>> wrote: >> >>> >>> Can you try the patch from this ticket? >>> >>> https://trac.mpich.org/projects/mpich/ticket/2090 >>> >>> ï¿½ Pavan >>> >>> On Aug 22, 2014, at 3:47 PM, James <jamesqf(a)charter.net> wrote: >>> >>>> Hi, >>>> >>>> (Apologies if this is the wrong place to post, but all the links >>>> on the MPE page seem to redirect to MPICH.) >>>> >>>> I am trying to build the MPE profiling libraries to work with MPICH. >>>> However, I am getting about a thousand lines of compile error messages, >>>> because some MPE routines use K&R-style function declarations, which >>>> don't match up with the prototypes in mpi.h >>>> >>>> 00106 log_mpi_core.c: In function 'MPI_Allgather': >>>> 00107 log_mpi_core.c:1550:8: error: argument 'sendbuf' doesn't match >>>> prototype >>>> 00108 void * sendbuf; >>>> 00109 ^ >>>> 00110 In file included from log_mpi_core.c:10:0: >>>> 00111 /opt/mpich/include/mpi.h:1002:5: error: prototype declaration >>>> 00112 int MPI_Allgather(const void *sendbuf, int sendcount, >>>> MPI_Datatype sendtype, void *recvbuf, >>>> 00113 ^ >>>> >>>> Before I go and try to fix all these myself, could I ask 1) Has it >>>> already been done somewhere? or 2) Has it not been done because there's >>>> now something better than MPE for tracing/profiling? >>>> >>>> To be clear, I'm not interested in profiling MPI itself, but what the >>>> program is doing between calls to MPI routines. >>>> >>>> Thanks, >>>> James >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-discuss] MPE Profiling?
by William Gropp 23 Aug '14

23 Aug '14

I also have some updates, and they preserve the ability to work with MPI-2 and MPI-3 implementations. Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign On Aug 22, 2014, at 5:07 PM, James <jamesqf(a)charter.net> wrote: > That seems to have helped: I'm down from ~1600 lines of error messages > to under 500. > > I'll probably have to do the rest by hand, and will try to make a > patch file of the changes > > James > > > On Fri, 22 Aug 2014 13:49:14 -0700, Balaji, Pavan <balaji(a)anl.gov> wrote: > >> >> Can you try the patch from this ticket? >> >> https://trac.mpich.org/projects/mpich/ticket/2090 >> >> — Pavan >> >> On Aug 22, 2014, at 3:47 PM, James <jamesqf(a)charter.net> wrote: >> >>> Hi, >>> >>> (Apologies if this is the wrong place to post, but all the links >>> on the MPE page seem to redirect to MPICH.) >>> >>> I am trying to build the MPE profiling libraries to work with MPICH. >>> However, I am getting about a thousand lines of compile error messages, >>> because some MPE routines use K&R-style function declarations, which >>> don't match up with the prototypes in mpi.h >>> >>> 00106 log_mpi_core.c: In function 'MPI_Allgather': >>> 00107 log_mpi_core.c:1550:8: error: argument 'sendbuf' doesn't match prototype >>> 00108 void * sendbuf; >>> 00109 ^ >>> 00110 In file included from log_mpi_core.c:10:0: >>> 00111 /opt/mpich/include/mpi.h:1002:5: error: prototype declaration >>> 00112 int MPI_Allgather(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, >>> 00113 ^ >>> >>> Before I go and try to fix all these myself, could I ask 1) Has it >>> already been done somewhere? or 2) Has it not been done because there's >>> now something better than MPE for tracing/profiling? >>> >>> To be clear, I'm not interested in profiling MPI itself, but what the >>> program is doing between calls to MPI routines. >>> >>> Thanks, >>> James > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_Get on the same memory location
by William Gropp 23 Aug '14

23 Aug '14

It is amusing that the original reason for the restriction on aliasing of arguments comes from Fortran - MPI-1 wanted the language-independent spec to be expressible in both Fortran and C, and this is a property of Fortran (and one which aids in performance). Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign On Aug 22, 2014, at 6:04 PM, Brian Van Straalen <bvstraalen(a)lbl.gov> wrote: > > Disabling the check is probably a bad idea. if you disable it in MPI you will still push a call to memcpy that has aliased buffers. For many systems memcpy still does the right thing, but modern fancy memcpy implementations have permission to produce wrong results if the regions overlap. > > The MPI spec is clear, and the coarray spec is also clear, and clearly different. I would recommend handling this in your library implementation and leave your MPI build as standard. Other MPI builds will be done standard and you will just be delaying the work. You can’t call MPI_Get with aliased locations. > > Brian > > > > On Aug 22, 2014, at 3:18 PM, Alessandro Fanfarillo <fanfarillo(a)ing.uniroma2.it> wrote: > >> Could you tell me which flag disables the check? I would suggest this >> procedure as temporary workaround. >> >> Thanks >> >> On Fri, Aug 22, 2014 at 2:41 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>> >>> On Aug 22, 2014, at 1:43 PM, Brian Van Straalen <bvstraalen(a)lbl.gov> wrote: >>>> Is there a significant cost to have the MPI implementation provide a check for aliasing? >>> >>> We do check for this error by default in MPICH (which is why this email thread started). >>> >>> There’s a way to turn off the checking through a configure flag as well as an environment variable, if needed. But that’s essentially saying — I know it’s wrong, but I’m too lazy to fix my application. >>> >>> — Pavan >>> >>> -- >>> Pavan Balaji ✉️ >>> http://www.mcs.anl.gov/~balaji >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> >> -- >> >> Alessandro Fanfarillo >> Dip. di Ingegneria Civile ed Ingegneria Informatica >> Università di Roma "Tor Vergata" >> NCAR Office: +1 (303) 497-2442 >> Tel: +39-06-7259 7719 >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > Brian Van Straalen Lawrence Berkeley Lab > BVStraalen(a)lbl.gov Computational Research > (510) 486-4976 Division (crd.lbl.gov) > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_Get on the same memory location
by Alessandro Fanfarillo 22 Aug '14

22 Aug '14

Thanks, I will suggest a temporary workaround to apply in the coarray program. On Fri, Aug 22, 2014 at 5:20 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > Sorry, I was wrong, there’s no runtime option to disable this. There’s only a runtime option to disable a similar check for collective operations. > > — Pavan > > On Aug 22, 2014, at 5:18 PM, Alessandro Fanfarillo <fanfarillo(a)ing.uniroma2.it> wrote: > >> Could you tell me which flag disables the check? I would suggest this >> procedure as temporary workaround. >> >> Thanks >> >> On Fri, Aug 22, 2014 at 2:41 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>> >>> On Aug 22, 2014, at 1:43 PM, Brian Van Straalen <bvstraalen(a)lbl.gov> wrote: >>>> Is there a significant cost to have the MPI implementation provide a check for aliasing? >>> >>> We do check for this error by default in MPICH (which is why this email thread started). >>> >>> There’s a way to turn off the checking through a configure flag as well as an environment variable, if needed. But that’s essentially saying — I know it’s wrong, but I’m too lazy to fix my application. >>> >>> — Pavan >>> >>> -- >>> Pavan Balaji ✉️ >>> http://www.mcs.anl.gov/~balaji >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> >> -- >> >> Alessandro Fanfarillo >> Dip. di Ingegneria Civile ed Ingegneria Informatica >> Università di Roma "Tor Vergata" >> NCAR Office: +1 (303) 497-2442 >> Tel: +39-06-7259 7719 >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > -- > Pavan Balaji ✉️ > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Alessandro Fanfarillo Dip. di Ingegneria Civile ed Ingegneria Informatica Università di Roma "Tor Vergata" NCAR Office: +1 (303) 497-2442 Tel: +39-06-7259 7719

1 0

Re: [mpich-discuss] MPI_Get on the same memory location
by Balaji, Pavan 22 Aug '14

22 Aug '14

Sorry, I was wrong, there’s no runtime option to disable this. There’s only a runtime option to disable a similar check for collective operations. — Pavan On Aug 22, 2014, at 5:18 PM, Alessandro Fanfarillo <fanfarillo(a)ing.uniroma2.it> wrote: > Could you tell me which flag disables the check? I would suggest this > procedure as temporary workaround. > > Thanks > > On Fri, Aug 22, 2014 at 2:41 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >> >> On Aug 22, 2014, at 1:43 PM, Brian Van Straalen <bvstraalen(a)lbl.gov> wrote: >>> Is there a significant cost to have the MPI implementation provide a check for aliasing? >> >> We do check for this error by default in MPICH (which is why this email thread started). >> >> There’s a way to turn off the checking through a configure flag as well as an environment variable, if needed. But that’s essentially saying — I know it’s wrong, but I’m too lazy to fix my application. >> >> — Pavan >> >> -- >> Pavan Balaji ✉️ >> http://www.mcs.anl.gov/~balaji >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > > Alessandro Fanfarillo > Dip. di Ingegneria Civile ed Ingegneria Informatica > Università di Roma "Tor Vergata" > NCAR Office: +1 (303) 497-2442 > Tel: +39-06-7259 7719 > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] MPI_Get on the same memory location
by Brian Van Straalen 22 Aug '14

22 Aug '14

Disabling the check is probably a bad idea. if you disable it in MPI you will still push a call to memcpy that has aliased buffers. For many systems memcpy still does the right thing, but modern fancy memcpy implementations have permission to produce wrong results if the regions overlap. The MPI spec is clear, and the coarray spec is also clear, and clearly different. I would recommend handling this in your library implementation and leave your MPI build as standard. Other MPI builds will be done standard and you will just be delaying the work. You can’t call MPI_Get with aliased locations. Brian On Aug 22, 2014, at 3:18 PM, Alessandro Fanfarillo <fanfarillo(a)ing.uniroma2.it> wrote: > Could you tell me which flag disables the check? I would suggest this > procedure as temporary workaround. > > Thanks > > On Fri, Aug 22, 2014 at 2:41 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >> >> On Aug 22, 2014, at 1:43 PM, Brian Van Straalen <bvstraalen(a)lbl.gov> wrote: >>> Is there a significant cost to have the MPI implementation provide a check for aliasing? >> >> We do check for this error by default in MPICH (which is why this email thread started). >> >> There’s a way to turn off the checking through a configure flag as well as an environment variable, if needed. But that’s essentially saying — I know it’s wrong, but I’m too lazy to fix my application. >> >> — Pavan >> >> -- >> Pavan Balaji ✉️ >> http://www.mcs.anl.gov/~balaji >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > > Alessandro Fanfarillo > Dip. di Ingegneria Civile ed Ingegneria Informatica > Università di Roma "Tor Vergata" > NCAR Office: +1 (303) 497-2442 > Tel: +39-06-7259 7719 > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss Brian Van Straalen Lawrence Berkeley Lab BVStraalen(a)lbl.gov Computational Research (510) 486-4976 Division (crd.lbl.gov)

1 0

Re: [mpich-discuss] MPI_Get on the same memory location
by Alessandro Fanfarillo 22 Aug '14

22 Aug '14

Could you tell me which flag disables the check? I would suggest this procedure as temporary workaround. Thanks On Fri, Aug 22, 2014 at 2:41 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > On Aug 22, 2014, at 1:43 PM, Brian Van Straalen <bvstraalen(a)lbl.gov> wrote: >> Is there a significant cost to have the MPI implementation provide a check for aliasing? > > We do check for this error by default in MPICH (which is why this email thread started). > > There’s a way to turn off the checking through a configure flag as well as an environment variable, if needed. But that’s essentially saying — I know it’s wrong, but I’m too lazy to fix my application. > > — Pavan > > -- > Pavan Balaji ✉️ > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Alessandro Fanfarillo Dip. di Ingegneria Civile ed Ingegneria Informatica Università di Roma "Tor Vergata" NCAR Office: +1 (303) 497-2442 Tel: +39-06-7259 7719

1 0

Re: [mpich-discuss] MPE Profiling?
by James 22 Aug '14

22 Aug '14

That seems to have helped: I'm down from ~1600 lines of error messages to under 500. I'll probably have to do the rest by hand, and will try to make a patch file of the changes James On Fri, 22 Aug 2014 13:49:14 -0700, Balaji, Pavan <balaji(a)anl.gov> wrote: > > Can you try the patch from this ticket? > > https://trac.mpich.org/projects/mpich/ticket/2090 > > — Pavan > > On Aug 22, 2014, at 3:47 PM, James <jamesqf(a)charter.net> wrote: > >> Hi, >> >> (Apologies if this is the wrong place to post, but all the links >> on the MPE page seem to redirect to MPICH.) >> >> I am trying to build the MPE profiling libraries to work with MPICH. >> However, I am getting about a thousand lines of compile error messages, >> because some MPE routines use K&R-style function declarations, which >> don't match up with the prototypes in mpi.h >> >> 00106 log_mpi_core.c: In function 'MPI_Allgather': >> 00107 log_mpi_core.c:1550:8: error: argument 'sendbuf' doesn't match >> prototype >> 00108 void * sendbuf; >> 00109 ^ >> 00110 In file included from log_mpi_core.c:10:0: >> 00111 /opt/mpich/include/mpi.h:1002:5: error: prototype declaration >> 00112 int MPI_Allgather(const void *sendbuf, int sendcount, >> MPI_Datatype sendtype, void *recvbuf, >> 00113 ^ >> >> Before I go and try to fix all these myself, could I ask 1) Has it >> already been done somewhere? or 2) Has it not been done because there's >> now something better than MPE for tracing/profiling? >> >> To be clear, I'm not interested in profiling MPI itself, but what the >> program is doing between calls to MPI routines. >> >> Thanks, >> James

1 0

Re: [mpich-discuss] MPE Profiling?
by Balaji, Pavan 22 Aug '14

22 Aug '14

Can you try the patch from this ticket? https://trac.mpich.org/projects/mpich/ticket/2090 — Pavan On Aug 22, 2014, at 3:47 PM, James <jamesqf(a)charter.net> wrote: > Hi, > > (Apologies if this is the wrong place to post, but all the links > on the MPE page seem to redirect to MPICH.) > > I am trying to build the MPE profiling libraries to work with MPICH. > However, I am getting about a thousand lines of compile error messages, > because some MPE routines use K&R-style function declarations, which > don't match up with the prototypes in mpi.h > > 00106 log_mpi_core.c: In function 'MPI_Allgather': > 00107 log_mpi_core.c:1550:8: error: argument 'sendbuf' doesn't match prototype > 00108 void * sendbuf; > 00109 ^ > 00110 In file included from log_mpi_core.c:10:0: > 00111 /opt/mpich/include/mpi.h:1002:5: error: prototype declaration > 00112 int MPI_Allgather(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, > 00113 ^ > > Before I go and try to fix all these myself, could I ask 1) Has it > already been done somewhere? or 2) Has it not been done because there's > now something better than MPE for tracing/profiling? > > To be clear, I'm not interested in profiling MPI itself, but what the > program is doing between calls to MPI routines. > > Thanks, > James > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji

1 0