- mpiwg-sessions - mailman3.cels.anl.gov

FW: Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum
by Holmes, Daniel John 08 Jun '23

08 Jun '23

Hi Howard (& all), In the MPI Virtual meeting yesterday, Rolf presented a PR that tries to define the word "pending" in MPI. The discussion raised a whole bunch of issues and concerns. I took an action to raise some of them in the Sessions WG because they are introduced by (or made worse by) the sessions model. Quick summary of options in Rolf's proposal: 1) Remove the word "pending" from the MPI Standard -- replace it with "active" where that makes sense, or something else when appropriate. 2) Define "pending" to be a strict alias of "active" -- it can only be applied to MPI operations and it means "starting stage has been done but completion stage has not yet been done" 2) Define "pending" to be anything task/activity on MPI 's to-do list -- encompassing all active operations but also all decoupled activities (i.e. progress). We discovered in the discussion of these options, in the context of MPI_FINALIZE and MPI_DISCONNECT (which must delay their return until "pending" things have been done) that an important restriction has been unintentionally removed by the introduction of sessions and we probably need to re-introduce parts of that restriction. When there was only the World Model, it was erroneous to call things like MPI_REQUEST_FREE after MPI_FINALIZE. Questions like "can I free a request after finalize?" -- derived from questions like "what happens to inactive persistent requests when I finalize MPI?" -- made no sense because you can look at the code with MPI_REQUEST_FREE after MPI_FINALIZE and immediate declare it is an erroneous program. Once sessions came into existence though, we can now call MPI_REQUEST_FREE after MPI_FINALIZE, as long as there is a session: ``` MPI_SESSION_INIT(&sh) MPI_INIT() MPI_RECV_INIT(..., MPI_COMM_WORLD, &req) MPI_FINALIZE() MPI_REQUEST_FREE(&req) MPI_SESSION_FINALIZE() ``` This is clearly A Bad Idea (tm), but which MPI rule has been broken in the above pseudo-code? Without the two sessions lines of code, this is clearly erroneous, but with them it is not. Do we want this pattern to be erroneous or well-defined? There is no semantic reason why this cannot be deemed legal -- MPI_REQUEST_FREE is a local procedure, so the fact MPI_COMM_WORLD is no longer functional should be irrelevant -- it could just clean up locally allocated resources (the connections to other processes were cleaned up during MPI_FINALIZE). One technical reason to define this pattern to be erroneous is that the request probably has a ref-counted reference to the communicator and MPI_FINALIZE will hang waiting for that ref-count to reach zero, which it cannot do because the request will not be freed until after it returns. This is an implementation detail, but it is how at least one major MPI library is currently implemented. An implementation could instead choose to store a weak_ptr (https://en.cppreference.com/w/cpp/memory/weak_ptr) to the communicator in the request when the operation is inactive, attempt to get a shared_ptr from the weak_ptr (https://en.cppreference.com/w/cpp/memory/weak_ptr/lock) during the starting stage, and give up the shared_ptr during the completion stage. In such an implementation, an inactive request does not own the communicator associated with the operation, so finalize can destroy the communicator, and a freeing an inactive request never needs to dereference to get the communicator, so it doesn't care whether the communicator still exists or not. Much more important than technical feasibility is the question "why should we enable this pattern?" mumble mumble C++ destructors mumble garbage collected languages mumble. Taking a step back, there is a whole category of programs that used to be erroneous by definition (cannot make MPI calls after MPI_FINALIZE) that are no longer obviously erroneous. What do we do about them all? There are related issues/concerns here: if a rule is added to MPI that forces MPI_FINALIZE to free all requests associated with the World Model, should sessions follow suit and have MPI_SESSION_FINALIZE free requests from that session? There is text exhorting the user to enable and complete all MPI operations (in the World Mode) before calling MPI_FINALIZE, but there is no such exhortation for the sessions model; should there be one? Does finalizing a session destroy all the communicators from that session? Can a communicator handle be used after the session is finalized (e.g. in MPI_COMM_FREE)? Best wishes, Dan. -----Original Message----- From: Rolf Rabenseifner <rabenseifner(a)hlrs.de> Sent: Wednesday, June 7, 2023 6:10 PM To: Holmes, Daniel John <daniel.john.holmes(a)intel.com> Subject: Fwd: Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum The pdf is https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… especially the questions on page 482 and 524. ----- Forwarded Message ----- From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> Sent: Wednesday, June 7, 2023 1:51:04 PM Subject: Re: Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum Dear MPI forum members, last update for #710/PR823: second option (alternative B) and open questions for MPI_Session_finalize, see https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… for the virtual meeting today. Kind regards Rolf ________ email from yesterday: _________ for the virtual meeting this week (June 7, 2023) the following readings and discussions are scheduled. Some technical decisions are needed and it may be best to do these decisions as institutional staw votes in an additional virtual meeting (e.g. next week), rather to do them ad hoc this week. The topics are #705/PR822, #710/PR823 with two options, #676/PR824 or PR825. Details: __________________________ #705 Errata: Fortran has only compile-time constants Rolf,Joseph Issue https://github.com/mpi-forum/mpi-issues/issues/705 (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff H.) PR https://github.com/mpi-forum/mpi-standard/pull/822 PDF https://github.com/mpi-forum/mpi-standard/files/11450065/mpi41-report_Issue… This proposal seems to be already stable. __________________________ #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf Issue https://github.com/mpi-forum/mpi-issues/issues/710 PR https://github.com/mpi-forum/mpi-standard/pull/823 updated: PDF https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… There is severe critism about the current proposal in PR 823: "In this PR here, as far as I can see, the implementations are mainly affected if they incorrectly require that there must not be any inactive request handle using the given comm. Whether an MPI lib really internally frees the inactive handles or not, is mainly a question of having dangling handles or not." "Is there broad agreement that this is incorrect implementation behavior? I'm skeptical." And therefore, there are two different solutions: - An advice to users, telling the consequences if they do not free inactive handles before calling MPI_Comm_disconnect or MPI_Session_finalize. or - Automatically freeing them. And furthermore: "Whatever we decide, I imagine this needs to extend to MPI_SESSION_FINALIZE as well." I'll try to update the proposal before the meeting to have text for both possible solutions. __________________________ #676 Errata: 'Pending operation' not defined, pending proper definition Rolf,Joseph Issue https://github.com/mpi-forum/mpi-issues/issues/676 PR https://github.com/mpi-forum/mpi-standard/pull/824 PDF https://github.com/mpi-forum/mpi-standard/files/11668075/mpi41-report_Issue… Substituting "pending operation" by "active operation" or PR https://github.com/mpi-forum/mpi-standard/pull/825 PDF https://github.com/mpi-forum/mpi-standard/files/11669354/mpi41-report_Issue… Defining "pending operation" as "active operation" (see page 12) Best regards Rolf ----- Original Message ----- > From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" <pvbangalore(a)ua.edu>, "Joseph Schuchart" > <schuchart(a)icl.utk.edu> > Sent: Wednesday, May 24, 2023 1:36:42 PM > Subject: Re: Update: Vote announcement for the July 2023 meeting of > the MPI Forum > Martin, Wes, and all, > > I expect that we should reserve also June 14, 2023 for a possible > continuation of open questions resulting from the discussions of > >> #705/PR822, #710/PR823, and #676/PR824 are now finalized at least for >> the virtual forum meeting, June 7, 2023 > > and other pending items for MPI-4.1 from the June 7 virtual forum meeting. > > Best regards > Rolf > > > > > ----- Original Message ----- >> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" >> <pvbangalore(a)ua.edu>, "Joseph Schuchart" >> <schuchart(a)icl.utk.edu> >> Sent: Thursday, May 11, 2023 8:05:07 PM >> Subject: Update: Vote announcement for the July 2023 meeting of the >> MPI Forum > >> Dear forum members, >> >> #705/PR822, #710/PR823, and #676/PR824 are now finalized at least for >> the virtual forum meeting, June 7, 2023, details below: >> >> #705 Errata: Fortran has only compile-time constants Rolf,Joseph >> Issue https://github.com/mpi-forum/mpi-issues/issues/705 >> (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff >> H.) >> PR https://github.com/mpi-forum/mpi-standard/pull/822 >> PDF >> >> https://github.com/mpi-forum/mpi-standard/files/11450065/mpi41-report >> _Issue705%2B657_PR822.pdf >> >> #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf >> Issue https://github.com/mpi-forum/mpi-issues/issues/710 >> PR https://github.com/mpi-forum/mpi-standard/pull/823 >> PDF >> >> https://github.com/mpi-forum/mpi-standard/files/11454896/mpi41-report >> _Issue710_PR823.pdf >> >> #676 Errata: 'Pending operation' not defined, pending proper definition >> Rolf,Joseph >> Issue https://github.com/mpi-forum/mpi-issues/issues/676 >> PR https://github.com/mpi-forum/mpi-standard/pull/824 >> PDF >> >> https://github.com/mpi-forum/mpi-standard/files/11443976/mpi41-report >> _Issue676_PR824.pdf >> >> The agenda of the virtual meeting should be reading and discussions if needed. >> I did the proposals based on the discussion and the major goals: >> - backward compatible, >> - no performance drawbacks on critical paths, >> - full consistency of the solution with all related parts of the MPI standard. >> >> If I have overseen something, then my apologies. >> >> If somebody wants to provide a completely different solution to one >> of these issues, please feel free to do it. >> - Thanks to Joseph who did PR 822, which was such a counter proposal for #705. >> - This method is often better than doing many change requests for the given PR. >> and it helps to completely check whether it is really consistent over >> all related parts of the MPI standard. >> - And thanks for all the comments so far. They really helped me a lot >> for preparing the three PRs. >> >> I'll be the next three weeks on vacation. >> >> Best regards >> Rolf Rabenseifner >> >> >> ----- Original Message ----- >>> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >>> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >>> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" >>> <pvbangalore(a)ua.edu>, "Joseph Schuchart" >>> <schuchart(a)icl.utk.edu> >>> Sent: Tuesday, May 9, 2023 6:07:04 PM >>> Subject: Vote announcement for the July 2023 meeting of the MPI >>> Forum >> >>> Dear forum members, >>> (correction: it is for the July 2023 meeting) >>> >>> I would like to make the following announcements for the next MPI >>> Forum Meeting (July 10-13, 2023): >>> >>> - 2nd votes on >>> #669 Add operation state 'enabled' and 'local calls' into Terms Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/669 >>> PR https://github.com/mpi-forum/mpi-standard/pull/788 >>> >>> #457 Improvements around the word "rank" in the Process Topologies chapter >>> Christoph,Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/457 >>> PR https://github.com/mpi-forum/mpi-standard/pull/804 >>> >>> #485 Fix Incorrect Usage of Rank/Task/etc. in Language Bindings Chapter >>> Rolf,Puri >>> Issue https://github.com/mpi-forum/mpi-issues/issues/485 >>> PR https://github.com/mpi-forum/mpi-standard/pull/803 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/10831598/mpi41-repor >>> t_Issue485_PR803.pdf >>> >>> - errata reading and vote on >>> >>> #679 Errata: Noncollective (for procedure) and nonpersistent are not defined >>> Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/679 >>> PR https://github.com/mpi-forum/mpi-standard/pull/820 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/11432286/mpi41-repor >>> t_Issue679_PR820.pdf >>> >>> - pre-announcement for errata reading and vote on --> See also >>> virtual meeting, June 7, 2023 !!! >>> >>> #705 Errata: Fortran has only compile-time constants Rolf,Joseph >>> Issue https://github.com/mpi-forum/mpi-issues/issues/705 >>> (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff >>> H.) >>> PR https://github.com/mpi-forum/mpi-standard/pull/822 >>> >>> #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/710 >>> PR https://github.com/mpi-forum/mpi-standard/pull/823 >>> >>> #676 Errata: 'Pending operation' not defined, pending proper definition >>> Rolf,Joseph >>> Issue https://github.com/mpi-forum/mpi-issues/issues/676 >>> PR https://github.com/mpi-forum/mpi-standard/pull/824 >>> >>> Best regards >>> Rolf Rabenseifner -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, 70569 Stuttgart, Germany -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, 70569 Stuttgart, Germany

1 0

Doodle poll for new time options for Sessions WG meetings (still on Mondays)
by Pritchard Jr., Howard 07 Jun '23

07 Jun '23

HI Folks, Per discussions at the last Sessions WG meeting about moving to an earlier time in the day for our regular meetings, I’ve hopefully created a doodle poll we can use to pick a time. https://doodle.com/meeting/participate/id/dLvMNLXa Times are all earlier than the current 12 PM US central time in recognition of the fact that typically at least half of the participants on the calls are from UK or central European time zones. I’ll ask Martin Schulz to adjust the zoom meeting once we’ve settled on a new time. We will stick with the current 12 PM US central time if we meet next Monday. Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_2927682628]<https://www.instagram.com/losalamosnatlab/>[signature_1438618913]<https://twitter.com/LosAlamosNatLab>[signature_4143922630]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_911225356]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Sessions WG - request for topics for 6/5/23 meeting
by Pritchard Jr., Howard 01 Jun '23

01 Jun '23

HI All, If you have a topic you’d like to discuss at next weeks Sessions WG meeting let me know and I’ll put it on the agenda. I’ve got one item to discuss, namely rewording of Sessions related functionality to lift some restrictions that currently exist on allowed pset name arguments to MPI_Group_from_session_pset. I’ve started work on a branch where these sorts of changes will be made. See https://github.com/hppritcha/mpi-standard/commit/b637545dc5f0c47ca897d5b8ce… Thanks, Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_2141780240]<https://www.instagram.com/losalamosnatlab/>[signature_2668660589]<https://twitter.com/LosAlamosNatLab>[signature_3191167880]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_432265969]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Sessions WG meeting 5/29
by Pritchard Jr., Howard 25 May '23

25 May '23

Hello All, This coming Monday is a US holiday so many of the US folks will be taking the day off. So I propose canceling next Monday’s Sessions WG meeting. For the following Monday, 6/5, I’d like to discuss this small change to the standard https://github.com/hppritcha/mpi-standard/commit/b637545dc5f0c47ca897d5b8ce… Its small but gives us an out for some of the dynamic pset work some of us are exploring. Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_1090544726]<https://www.instagram.com/losalamosnatlab/>[signature_1006799064]<https://twitter.com/LosAlamosNatLab>[signature_129244297]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_939819964]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Cancel today's Sessions WG meeting
by Pritchard Jr., Howard 15 May '23

15 May '23

Hi All, No topics on the agenda so we’ll cancel today’s Sessions WG meeting. Since ISC is next week, we’ll also go ahead and cancel next week’s meeting as well. We’ll reconvene on 5/29. Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_3825652674]<https://www.instagram.com/losalamosnatlab/>[signature_3690866012]<https://twitter.com/LosAlamosNatLab>[signature_4071740062]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_452210332]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Agenda items for Sessions WG meeting 5/15
by Pritchard Jr., Howard 12 May '23

12 May '23

Hi All, Please let me know if you have items you’d like to discuss at the Sessions WG meeting scheduled for 5/15. Thanks very much, Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_2354025222]<https://www.instagram.com/losalamosnatlab/>[signature_4151899719]<https://twitter.com/LosAlamosNatLab>[signature_3914425316]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_262735055]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Re: [mpiwg-sessions] Agenda items for Sessions WG meeting - 5/8/23
by Pritchard Jr., Howard 08 May '23

08 May '23

Hi All, We’ll meet today. I’ve one other item for the agenda – FlexMPI. Has anyone used this? Google scholar has pointed me to a recent paper by Carretero, Exposito, et a. – Malleability Techniques for HPC systems. Howard From: "Pritchard Jr., Howard" <howardp(a)lanl.gov> Date: Thursday, May 4, 2023 at 4:45 PM To: MPI Sessions working group <mpiwg-sessions(a)lists.mpi-forum.org> Subject: Agenda items for Sessions WG meeting - 5/8/23 Hi All, Please send me agenda items you’d like to discuss at our WG meeting on 5/8/23. I have one item already. I’d like to discuss https://github.com/mpi-forum/mpi-issues/issues/710 and its possible impact on sessions. Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_2816248013]<https://www.instagram.com/losalamosnatlab/>[signature_10222113]<https://twitter.com/LosAlamosNatLab>[signature_2944524454]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_3333939719]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Re: [mpiwg-sessions] Agenda items for Sessions WG meeting - 5/8/23
by Holmes, Daniel John 05 May '23

05 May '23

Hi Howard (et al), I wrote down my thoughts on #710 (including the sessions viewpoint) here: https://github.com/mpi-forum/mpi-issues/issues/710#issuecomment-1536306552 Addendum, just for sessions folks: ``` C MPI_Init(); MPI_Recv_init(..., MPI_COMM_WORLD, &req); MPI_Finalize(); MPI_Request_free(&req); // erroneous, MPI is no longer initialised, so calling MPI_Request_free is prohibited MPI_Session_init(); MPI_Request_free(&req); // oh dear lord, what is this? Surely this is also erroneous, but where ...flicks through the MPI Standard... is the prohibition in the standard? MPI_Session_finalize(); ``` Best wishes, Dan. [cid:[email protected]] From: mpiwg-sessions <mpiwg-sessions-bounces(a)lists.mpi-forum.org> On Behalf Of Pritchard Jr., Howard via mpiwg-sessions Sent: 04 May 2023 23:46 To: MPI Sessions working group <mpiwg-sessions(a)lists.mpi-forum.org> Cc: Pritchard Jr., Howard <howardp(a)lanl.gov> Subject: [mpiwg-sessions] Agenda items for Sessions WG meeting - 5/8/23 Hi All, Please send me agenda items you'd like to discuss at our WG meeting on 5/8/23. I have one item already. I'd like to discuss https://github.com/mpi-forum/mpi-issues/issues/710 and its possible impact on sessions. Howard - [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov<mailto:[email protected]> [signature_2816248013]<https://www.instagram.com/losalamosnatlab/>[signature_10222113]<https://twitter.com/LosAlamosNatLab>[signature_2944524454]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_3333939719]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Agenda items for Sessions WG meeting - 5/8/23
by Pritchard Jr., Howard 04 May '23

04 May '23

Hi All, Please send me agenda items you’d like to discuss at our WG meeting on 5/8/23. I have one item already. I’d like to discuss https://github.com/mpi-forum/mpi-issues/issues/710 and its possible impact on sessions. Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_2816248013]<https://www.instagram.com/losalamosnatlab/>[signature_10222113]<https://twitter.com/LosAlamosNatLab>[signature_2944524454]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_3333939719]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Sessions WG - no meeting on May 1
by Pritchard Jr., Howard 30 Apr '23

30 Apr '23

Hi All,, Canceling this coming Monday Sessions WG meeting as May 1 is a holiday for many of us, and we’ll be spending plenty of time on MPI the remainder of the week. Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_2504329336]<https://www.instagram.com/losalamosnatlab/>[signature_3992758059]<https://twitter.com/LosAlamosNatLab>[signature_2799185150]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_1995536454]<https://www.facebook.com/LosAlamosNationalLab/>

1 0