Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
We (Jeff, Bill L., Craig, Hubert, Martin, Tobias, and me (Rolf)) had a telecon today and the outcome are following options: Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem: Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization. Option 2 - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.) Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface. Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc. Pro: I'm not sure whether there is a real pro? Bill, can you add the pros, it is your option. Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface. My favorite in the moment, i.e. at the starting of the pro-con-discussion: Option 2. Best regards Rolf ----- Original Message -----
From: "Rolf Rabenseifner" <[email protected]> To: [email protected] Cc: "Jeff Squyres (jsquyres)" <[email protected]>, "Craig E Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Monday, March 25, 2013 12:21:39 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Yes, this freedom (1)-(4) in Bill's email is important.
In our discussion, we forgot one additional reason why we defined it as it is written in MPI-3.0 Section 17.1.5, page 605:
- It should be possible to implement the interfaces with these non-standard Fortran-void-* directives, and later on, to substitute this implementation by new ones that are based on TS 29113. - When doing this substitution, it should be save for applications when the application is not re-compiled. - More than that, it should be possible to combine older .o files with newly TS 29113 based .o files. - For this, we decided, that the implementation has to switch from non-BIND(C) to BIND(C) at least in the MPI_ISEND routine group when switching to MPI_SUBARRAYS_SUPPORTED==.TRUE. - The real trick is that the linker name must be changed. - This all is valid for both, the new mpi_f08 module, and the old mpi module.
A second issue in our discussion is always, which changes we want to do to the mpi_f08 MPI-3.0 user interface. I want to remember that this user interface is defined according following rules. If a user modifies his source to use everywhere mpi_f08, he/she has to modify: - He/she has to substitute the "include 'mpif.h'" or "use mpi" statements by "use mpi_f08". - He/she has to modify all handle and status declarations. - In the executable statements, he/she has to substitute all status(MPI_TAG), status(MPI_SOURCE), status(MPI_ERROR) accesses into status%MPI_TAG, status%MPI_SOURCE, status%MPI_ERROR. - Callback routines need more effort due to the abstract interface definitions. This is touched by ticket #364, i.e., the MPI-3.0 BIND(C) of callbacks had to be removed. - Changes in the interfaces are only -- all handles use the new handle types and in -- MPI_User_function, -- MPI_Datarep_conversion_function, the buffers are done with TYPE(C_TR), VALUE.
This should be all. All other user correct code should directly compile with the new mpi_f08 module. We should not change these things, except the necessary errata in #364.
I see that we have some freedom only in the area, what we do internally (together with linker names) and with PMPI, but not in the user interface. If we add additional user interfaces (e.g. overloading the LOGICAL with INTEGER arguments) we would only make it more complicate.
Best regards Rolf
----- Original Message -----
From: "Bill Long" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "Jeff Squyres (jsquyres)" <[email protected]>, "Craig E Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Sunday, March 24, 2013 5:59:38 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind This time is OK for me. I'll look for the webex email.
I would like to note that the various options discussed are not necessarily "all or nothing" propositions. An implementation could use the thin wrappers in a module approach to eliminate the bad Fortran types problems for most of the MPI routines, but also have some routines implemented in a different way, such as (1) all in Fortran that is callable from C, (2) all in C that is called through an interface directly from Fortran, (3) as a fat wrapper in Fortran, or (4) a completely replicated routine for the Fortran implementation. There is no reason that all of the routines have to be the same method - in fact probably good arguments against that in some cases. We just need to have the rules such that the tools have some known name to intercept. Along the lines of "If implementation method X is chosen for routine MPI_xxx, then these are the linker name requirements:.....".
Cheers, Bill
On 3/23/13 2:11 AM, Rolf Rabenseifner wrote:
Dear Jeff, Craig, Bill, Hubert, Martin, and Tobias,
I expect that this is the total group that discusses the Fortran LOGICAL etc. problem. We learnt that the solutions also touch the PMPI concept. There was a telecon between Craig, Jeff and Bill with the proposal at the end of this email. I like to understand this proposal and therefore I asked Jeff for explanations. I would like to open this for the whole group.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time. Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Monday March 25, - 4 PM Central European Time CET = 11 AM Eastern Daylight Time EST/EDT = 8 AM Pacific Time
I hope these relations are correct. Would this work for you?
At least for Jeff and me, because I want to understand what they discussed with their proposal.
Jeff, can you set up a webex?
Best regards Rolf
----- Original Message -----
From: "Jeff Squyres (jsquyres)" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "Craig E Rasmussen" <[email protected]> Sent: Friday, March 22, 2013 8:01:23 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Btw, I'm not sure how this is not the least surprise.
We can't change the params to not be logical, yes we must still allow for calling to c. So the overloaded solution is the only one.
And the new Integer param signature must have a new bind name so as to did ambiguity it from the one we published in MPI3. See my previous mail about this.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time.
Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Sent from my phone. No type good.
On Mar 22, 2013, at 8:51 AM, "Rolf Rabenseifner" <[email protected]> wrote:
Jeff,
can I call you by phone? I would like to talk to you as the binding responsible and MPI chair.
2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not ... This is a hard step and far away from "last surprise".
I would like to learn how it came out as a result of your telecon, before I try to go the next step (e.g. email answer).
Best regards Rolf
----- Original Message -----
From: "Craig Rasmussen" <[email protected]> To: "MPI-3 Fortran working group" <[email protected]> Cc: "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]> Sent: Friday, March 22, 2013 4:29:36 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
Bill Long, Jeff Squyres, and I had a phone conversation yesterday and came up with a straw man proposal. We believe it fixes the LOGICAL argument problem while satisfying three major goals:
1. Backwards compatibility. 2. Flexibility so that efficient implementations can be produced on a variety of hardware. 3. Not dramatically increase the complexity for the tools community. 4. Ensure that the changes won't be seen by ordinary Fortran MPI users.
The straw man proposal is:
1. Remove the current binding names for MPI functions with LOGICAL dummy arguments (. This must be done because these interfaces break the Fortran standard. 2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not conflict with the current name with LOGICAL arguments. For example, the binding name "MPI_Test_f08" will be replaced by "MPI_Test_int_f08" (exact replacement name open for discussion). The Fortran implementation of the function MPI_Test must call MPI_Test_int_f08 as soon as possible for the tools implementers. 3. Implementation of the Fortran PMPI calling convention will be made optional (we think it already it). If an MPI implementer doesn't provide the MPI_F08 binding names, then it must call the C MPI implementation in a thin Fortran wrapper so that the tools implementers can intercept the call in C. An MPI implementation shall not provide/call both the MPI_F08 binding name and the C name.
Option 2 allows some MPI vendors to implement MPI functions entirely within Fortran (using the MPI_f08 binding names), if this is more efficient.
Option 3 allows some MPI implementations to place Fortran wrappers inside modules where they may be inlined by the compiler for efficiency.
Craig Rasmussen CAS Scientific Programmer [email protected]
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
My summary of the three options: 1. Absolute minimal change, remove BIND(C) only for routines with LOGICAL arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL arguments. 2. Remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines. 3. Remove BIND(C) from all routines. Plus allow Fortran implementations to substitute the C PMPI interface standard. Is this a fair summary? My vote: I'm strongly in favor of option 3. It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API. There are several advantages with this implementation: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well. Craig Rasmussen CAS Scientific Programmer [email protected] On Mar 25, 2013, at 10:11 AM, Rolf Rabenseifner wrote:
We (Jeff, Bill L., Craig, Hubert, Martin, Tobias, and me (Rolf)) had a telecon today and the outcome are following options:
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem: Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2 - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.)
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: I'm not sure whether there is a real pro?
Bill, can you add the pros, it is your option.
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
My favorite in the moment, i.e. at the starting of the pro-con-discussion: Option 2.
Best regards Rolf
----- Original Message -----
From: "Rolf Rabenseifner" <[email protected]> To: [email protected] Cc: "Jeff Squyres (jsquyres)" <[email protected]>, "Craig E Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Monday, March 25, 2013 12:21:39 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Yes, this freedom (1)-(4) in Bill's email is important.
In our discussion, we forgot one additional reason why we defined it as it is written in MPI-3.0 Section 17.1.5, page 605:
- It should be possible to implement the interfaces with these non-standard Fortran-void-* directives, and later on, to substitute this implementation by new ones that are based on TS 29113. - When doing this substitution, it should be save for applications when the application is not re-compiled. - More than that, it should be possible to combine older .o files with newly TS 29113 based .o files. - For this, we decided, that the implementation has to switch from non-BIND(C) to BIND(C) at least in the MPI_ISEND routine group when switching to MPI_SUBARRAYS_SUPPORTED==.TRUE. - The real trick is that the linker name must be changed. - This all is valid for both, the new mpi_f08 module, and the old mpi module.
A second issue in our discussion is always, which changes we want to do to the mpi_f08 MPI-3.0 user interface. I want to remember that this user interface is defined according following rules. If a user modifies his source to use everywhere mpi_f08, he/she has to modify: - He/she has to substitute the "include 'mpif.h'" or "use mpi" statements by "use mpi_f08". - He/she has to modify all handle and status declarations. - In the executable statements, he/she has to substitute all status(MPI_TAG), status(MPI_SOURCE), status(MPI_ERROR) accesses into status%MPI_TAG, status%MPI_SOURCE, status%MPI_ERROR. - Callback routines need more effort due to the abstract interface definitions. This is touched by ticket #364, i.e., the MPI-3.0 BIND(C) of callbacks had to be removed. - Changes in the interfaces are only -- all handles use the new handle types and in -- MPI_User_function, -- MPI_Datarep_conversion_function, the buffers are done with TYPE(C_TR), VALUE.
This should be all. All other user correct code should directly compile with the new mpi_f08 module. We should not change these things, except the necessary errata in #364.
I see that we have some freedom only in the area, what we do internally (together with linker names) and with PMPI, but not in the user interface. If we add additional user interfaces (e.g. overloading the LOGICAL with INTEGER arguments) we would only make it more complicate.
Best regards Rolf
----- Original Message -----
From: "Bill Long" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "Jeff Squyres (jsquyres)" <[email protected]>, "Craig E Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Sunday, March 24, 2013 5:59:38 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind This time is OK for me. I'll look for the webex email.
I would like to note that the various options discussed are not necessarily "all or nothing" propositions. An implementation could use the thin wrappers in a module approach to eliminate the bad Fortran types problems for most of the MPI routines, but also have some routines implemented in a different way, such as (1) all in Fortran that is callable from C, (2) all in C that is called through an interface directly from Fortran, (3) as a fat wrapper in Fortran, or (4) a completely replicated routine for the Fortran implementation. There is no reason that all of the routines have to be the same method - in fact probably good arguments against that in some cases. We just need to have the rules such that the tools have some known name to intercept. Along the lines of "If implementation method X is chosen for routine MPI_xxx, then these are the linker name requirements:.....".
Cheers, Bill
On 3/23/13 2:11 AM, Rolf Rabenseifner wrote:
Dear Jeff, Craig, Bill, Hubert, Martin, and Tobias,
I expect that this is the total group that discusses the Fortran LOGICAL etc. problem. We learnt that the solutions also touch the PMPI concept. There was a telecon between Craig, Jeff and Bill with the proposal at the end of this email. I like to understand this proposal and therefore I asked Jeff for explanations. I would like to open this for the whole group.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time. Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Monday March 25, - 4 PM Central European Time CET = 11 AM Eastern Daylight Time EST/EDT = 8 AM Pacific Time
I hope these relations are correct. Would this work for you?
At least for Jeff and me, because I want to understand what they discussed with their proposal.
Jeff, can you set up a webex?
Best regards Rolf
----- Original Message -----
From: "Jeff Squyres (jsquyres)" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "Craig E Rasmussen" <[email protected]> Sent: Friday, March 22, 2013 8:01:23 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Btw, I'm not sure how this is not the least surprise.
We can't change the params to not be logical, yes we must still allow for calling to c. So the overloaded solution is the only one.
And the new Integer param signature must have a new bind name so as to did ambiguity it from the one we published in MPI3. See my previous mail about this.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time.
Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Sent from my phone. No type good.
On Mar 22, 2013, at 8:51 AM, "Rolf Rabenseifner" <[email protected]> wrote:
Jeff,
can I call you by phone? I would like to talk to you as the binding responsible and MPI chair.
2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not ... This is a hard step and far away from "last surprise".
I would like to learn how it came out as a result of your telecon, before I try to go the next step (e.g. email answer).
Best regards Rolf
----- Original Message -----
From: "Craig Rasmussen" <[email protected]> To: "MPI-3 Fortran working group" <[email protected]> Cc: "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]> Sent: Friday, March 22, 2013 4:29:36 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
Bill Long, Jeff Squyres, and I had a phone conversation yesterday and came up with a straw man proposal. We believe it fixes the LOGICAL argument problem while satisfying three major goals:
1. Backwards compatibility. 2. Flexibility so that efficient implementations can be produced on a variety of hardware. 3. Not dramatically increase the complexity for the tools community. 4. Ensure that the changes won't be seen by ordinary Fortran MPI users.
The straw man proposal is:
1. Remove the current binding names for MPI functions with LOGICAL dummy arguments (. This must be done because these interfaces break the Fortran standard. 2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not conflict with the current name with LOGICAL arguments. For example, the binding name "MPI_Test_f08" will be replaced by "MPI_Test_int_f08" (exact replacement name open for discussion). The Fortran implementation of the function MPI_Test must call MPI_Test_int_f08 as soon as possible for the tools implementers. 3. Implementation of the Fortran PMPI calling convention will be made optional (we think it already it). If an MPI implementer doesn't provide the MPI_F08 binding names, then it must call the C MPI implementation in a thin Fortran wrapper so that the tools implementers can intercept the call in C. An MPI implementation shall not provide/call both the MPI_F08 binding name and the C name.
Option 2 allows some MPI vendors to implement MPI functions entirely within Fortran (using the MPI_f08 binding names), if this is more efficient.
Option 3 allows some MPI implementations to place Fortran wrappers inside modules where they may be inlined by the compiler for efficiency.
Craig Rasmussen CAS Scientific Programmer [email protected]
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
On 3/25/13 12:50 PM, Craig Rasmussen wrote:
My summary of the three options:
1. Absolute minimal change, remove BIND(C) only for routines with LOGICAL arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL arguments.
2. Remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines.
3. Remove BIND(C) from all routines. Plus allow Fortran implementations to substitute the C PMPI interface standard.
Is this a fair summary?
My vote:
I'm strongly in favor of option 3. It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API. There are several advantages with this implementation: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well.
I like option 3 as well. It uses the C interoperability feature in Fortran the way it was intended/designed. I would add to the above: 6. You can isolate the issues of Fortran types (compiled with -i8, meaning of LOGICAL, character arguments with len > 1) into the Fortran wrappers so that the C library writers do not have to know about these issues. Simplifies the maintenance and testing issues for the C library code. I would modify option 3 slightly. The implementer might want to write some routine ONLY in Fortran. The routines that convert the status formats between the Fortran and the C forms comes to mind. In that case, BIND(C) would be required, and it would specify the C binding name as the external linker name. This would be needed so that C codes could call the routine. Cheers, Bill
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
On Mar 25, 2013, at 10:11 AM, Rolf Rabenseifner wrote:
We (Jeff, Bill L., Craig, Hubert, Martin, Tobias, and me (Rolf)) had a telecon today and the outcome are following options:
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem: Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2 - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.)
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: I'm not sure whether there is a real pro?
Bill, can you add the pros, it is your option.
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
My favorite in the moment, i.e. at the starting of the pro-con-discussion: Option 2.
Best regards Rolf
----- Original Message -----
From: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> To: [email protected] <mailto:[email protected]> Cc: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>>, "Craig E Rasmussen" <[email protected] <mailto:[email protected]>>, "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>>, "Tobias Burnus" <[email protected] <mailto:[email protected]>> Sent: Monday, March 25, 2013 12:21:39 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Yes, this freedom (1)-(4) in Bill's email is important.
In our discussion, we forgot one additional reason why we defined it as it is written in MPI-3.0 Section 17.1.5, page 605:
- It should be possible to implement the interfaces with these non-standard Fortran-void-* directives, and later on, to substitute this implementation by new ones that are based on TS 29113. - When doing this substitution, it should be save for applications when the application is not re-compiled. - More than that, it should be possible to combine older .o files with newly TS 29113 based .o files. - For this, we decided, that the implementation has to switch from non-BIND(C) to BIND(C) at least in the MPI_ISEND routine group when switching to MPI_SUBARRAYS_SUPPORTED==.TRUE. - The real trick is that the linker name must be changed. - This all is valid for both, the new mpi_f08 module, and the old mpi module.
A second issue in our discussion is always, which changes we want to do to the mpi_f08 MPI-3.0 user interface. I want to remember that this user interface is defined according following rules. If a user modifies his source to use everywhere mpi_f08, he/she has to modify: - He/she has to substitute the "include 'mpif.h'" or "use mpi" statements by "use mpi_f08". - He/she has to modify all handle and status declarations. - In the executable statements, he/she has to substitute all status(MPI_TAG), status(MPI_SOURCE), status(MPI_ERROR) accesses into status%MPI_TAG, status%MPI_SOURCE, status%MPI_ERROR. - Callback routines need more effort due to the abstract interface definitions. This is touched by ticket #364, i.e., the MPI-3.0 BIND(C) of callbacks had to be removed. - Changes in the interfaces are only -- all handles use the new handle types and in -- MPI_User_function, -- MPI_Datarep_conversion_function, the buffers are done with TYPE(C_TR), VALUE.
This should be all. All other user correct code should directly compile with the new mpi_f08 module. We should not change these things, except the necessary errata in #364.
I see that we have some freedom only in the area, what we do internally (together with linker names) and with PMPI, but not in the user interface. If we add additional user interfaces (e.g. overloading the LOGICAL with INTEGER arguments) we would only make it more complicate.
Best regards Rolf
----- Original Message -----
From: "Bill Long" <[email protected] <mailto:[email protected]>> To: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> Cc: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>>, "Craig E Rasmussen" <[email protected] <mailto:[email protected]>>, "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>>, "Tobias Burnus" <[email protected] <mailto:[email protected]>> Sent: Sunday, March 24, 2013 5:59:38 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind This time is OK for me. I'll look for the webex email.
I would like to note that the various options discussed are not necessarily "all or nothing" propositions. An implementation could use the thin wrappers in a module approach to eliminate the bad Fortran types problems for most of the MPI routines, but also have some routines implemented in a different way, such as (1) all in Fortran that is callable from C, (2) all in C that is called through an interface directly from Fortran, (3) as a fat wrapper in Fortran, or (4) a completely replicated routine for the Fortran implementation. There is no reason that all of the routines have to be the same method - in fact probably good arguments against that in some cases. We just need to have the rules such that the tools have some known name to intercept. Along the lines of "If implementation method X is chosen for routine MPI_xxx, then these are the linker name requirements:.....".
Cheers, Bill
On 3/23/13 2:11 AM, Rolf Rabenseifner wrote:
Dear Jeff, Craig, Bill, Hubert, Martin, and Tobias,
I expect that this is the total group that discusses the Fortran LOGICAL etc. problem. We learnt that the solutions also touch the PMPI concept. There was a telecon between Craig, Jeff and Bill with the proposal at the end of this email. I like to understand this proposal and therefore I asked Jeff for explanations. I would like to open this for the whole group.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time. Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Monday March 25, - 4 PM Central European Time CET = 11 AM Eastern Daylight Time EST/EDT = 8 AM Pacific Time
I hope these relations are correct. Would this work for you?
At least for Jeff and me, because I want to understand what they discussed with their proposal.
Jeff, can you set up a webex?
Best regards Rolf
----- Original Message -----
From: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>> To: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> Cc: "Craig E Rasmussen" <[email protected] <mailto:[email protected]>> Sent: Friday, March 22, 2013 8:01:23 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Btw, I'm not sure how this is not the least surprise.
We can't change the params to not be logical, yes we must still allow for calling to c. So the overloaded solution is the only one.
And the new Integer param signature must have a new bind name so as to did ambiguity it from the one we published in MPI3. See my previous mail about this.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time.
Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Sent from my phone. No type good.
On Mar 22, 2013, at 8:51 AM, "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> wrote:
Jeff,
can I call you by phone? I would like to talk to you as the binding responsible and MPI chair.
2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not ... This is a hard step and far away from "last surprise".
I would like to learn how it came out as a result of your telecon, before I try to go the next step (e.g. email answer).
Best regards Rolf
----- Original Message -----
From: "Craig Rasmussen" <[email protected] <mailto:[email protected]>> To: "MPI-3 Fortran working group" <[email protected] <mailto:[email protected]>> Cc: "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>> Sent: Friday, March 22, 2013 4:29:36 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
Bill Long, Jeff Squyres, and I had a phone conversation yesterday and came up with a straw man proposal. We believe it fixes the LOGICAL argument problem while satisfying three major goals:
1. Backwards compatibility. 2. Flexibility so that efficient implementations can be produced on a variety of hardware. 3. Not dramatically increase the complexity for the tools community. 4. Ensure that the changes won't be seen by ordinary Fortran MPI users.
The straw man proposal is:
1. Remove the current binding names for MPI functions with LOGICAL dummy arguments (. This must be done because these interfaces break the Fortran standard. 2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not conflict with the current name with LOGICAL arguments. For example, the binding name "MPI_Test_f08" will be replaced by "MPI_Test_int_f08" (exact replacement name open for discussion). The Fortran implementation of the function MPI_Test must call MPI_Test_int_f08 as soon as possible for the tools implementers. 3. Implementation of the Fortran PMPI calling convention will be made optional (we think it already it). If an MPI implementer doesn't provide the MPI_F08 binding names, then it must call the C MPI implementation in a thin Fortran wrapper so that the tools implementers can intercept the call in C. An MPI implementation shall not provide/call both the MPI_F08 binding name and the C name.
Option 2 allows some MPI vendors to implement MPI functions entirely within Fortran (using the MPI_f08 binding names), if this is more efficient.
Option 3 allows some MPI implementations to place Fortran wrappers inside modules where they may be inlined by the compiler for efficiency.
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
-- Bill Long [email protected] <mailto:[email protected]> Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] <mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] <mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
I vote for Option 2. Option 3 is not possible for an errata. This is a severe change in the standard. Your modification of Option 3 makes the standard more difficult (we have again mixing of BIND(C) and non BIND(C) functions) and I can't see a real application advantage. To your items: Item 2: You have to perform also additional work in the wrappers (for example memory allocation and transformation for sources compiled with -i8; additional error checks) Item 3: Support of PMPI version of Fortran is not really an issue. Item 4b: The tools have to support routines compiled with mpi.h and old objects. Hubert ________________________________________ From: Bill Long [[email protected]] Sent: Monday, March 25, 2013 7:35 PM To: Craig Rasmussen Cc: Rolf Rabenseifner; MPI-3 Fortran working group; Jeff Squyres (jsquyres); Hubert Ritzdorf; Martin Schulz; Tobias Burnus Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind On 3/25/13 12:50 PM, Craig Rasmussen wrote:
My summary of the three options:
1. Absolute minimal change, remove BIND(C) only for routines with LOGICAL arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL arguments.
2. Remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines.
3. Remove BIND(C) from all routines. Plus allow Fortran implementations to substitute the C PMPI interface standard.
Is this a fair summary?
My vote:
I'm strongly in favor of option 3. It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API. There are several advantages with this implementation: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well.
I like option 3 as well. It uses the C interoperability feature in Fortran the way it was intended/designed. I would add to the above: 6. You can isolate the issues of Fortran types (compiled with -i8, meaning of LOGICAL, character arguments with len > 1) into the Fortran wrappers so that the C library writers do not have to know about these issues. Simplifies the maintenance and testing issues for the C library code. I would modify option 3 slightly. The implementer might want to write some routine ONLY in Fortran. The routines that convert the status formats between the Fortran and the C forms comes to mind. In that case, BIND(C) would be required, and it would specify the C binding name as the external linker name. This would be needed so that C codes could call the routine. Cheers, Bill
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
On Mar 25, 2013, at 10:11 AM, Rolf Rabenseifner wrote:
We (Jeff, Bill L., Craig, Hubert, Martin, Tobias, and me (Rolf)) had a telecon today and the outcome are following options:
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem: Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2 - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.)
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: I'm not sure whether there is a real pro?
Bill, can you add the pros, it is your option.
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
My favorite in the moment, i.e. at the starting of the pro-con-discussion: Option 2.
Best regards Rolf
----- Original Message -----
From: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> To: [email protected] <mailto:[email protected]> Cc: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>>, "Craig E Rasmussen" <[email protected] <mailto:[email protected]>>, "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>>, "Tobias Burnus" <[email protected] <mailto:[email protected]>> Sent: Monday, March 25, 2013 12:21:39 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Yes, this freedom (1)-(4) in Bill's email is important.
In our discussion, we forgot one additional reason why we defined it as it is written in MPI-3.0 Section 17.1.5, page 605:
- It should be possible to implement the interfaces with these non-standard Fortran-void-* directives, and later on, to substitute this implementation by new ones that are based on TS 29113. - When doing this substitution, it should be save for applications when the application is not re-compiled. - More than that, it should be possible to combine older .o files with newly TS 29113 based .o files. - For this, we decided, that the implementation has to switch from non-BIND(C) to BIND(C) at least in the MPI_ISEND routine group when switching to MPI_SUBARRAYS_SUPPORTED==.TRUE. - The real trick is that the linker name must be changed. - This all is valid for both, the new mpi_f08 module, and the old mpi module.
A second issue in our discussion is always, which changes we want to do to the mpi_f08 MPI-3.0 user interface. I want to remember that this user interface is defined according following rules. If a user modifies his source to use everywhere mpi_f08, he/she has to modify: - He/she has to substitute the "include 'mpif.h'" or "use mpi" statements by "use mpi_f08". - He/she has to modify all handle and status declarations. - In the executable statements, he/she has to substitute all status(MPI_TAG), status(MPI_SOURCE), status(MPI_ERROR) accesses into status%MPI_TAG, status%MPI_SOURCE, status%MPI_ERROR. - Callback routines need more effort due to the abstract interface definitions. This is touched by ticket #364, i.e., the MPI-3.0 BIND(C) of callbacks had to be removed. - Changes in the interfaces are only -- all handles use the new handle types and in -- MPI_User_function, -- MPI_Datarep_conversion_function, the buffers are done with TYPE(C_TR), VALUE.
This should be all. All other user correct code should directly compile with the new mpi_f08 module. We should not change these things, except the necessary errata in #364.
I see that we have some freedom only in the area, what we do internally (together with linker names) and with PMPI, but not in the user interface. If we add additional user interfaces (e.g. overloading the LOGICAL with INTEGER arguments) we would only make it more complicate.
Best regards Rolf
----- Original Message -----
From: "Bill Long" <[email protected] <mailto:[email protected]>> To: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> Cc: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>>, "Craig E Rasmussen" <[email protected] <mailto:[email protected]>>, "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>>, "Tobias Burnus" <[email protected] <mailto:[email protected]>> Sent: Sunday, March 24, 2013 5:59:38 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind This time is OK for me. I'll look for the webex email.
I would like to note that the various options discussed are not necessarily "all or nothing" propositions. An implementation could use the thin wrappers in a module approach to eliminate the bad Fortran types problems for most of the MPI routines, but also have some routines implemented in a different way, such as (1) all in Fortran that is callable from C, (2) all in C that is called through an interface directly from Fortran, (3) as a fat wrapper in Fortran, or (4) a completely replicated routine for the Fortran implementation. There is no reason that all of the routines have to be the same method - in fact probably good arguments against that in some cases. We just need to have the rules such that the tools have some known name to intercept. Along the lines of "If implementation method X is chosen for routine MPI_xxx, then these are the linker name requirements:.....".
Cheers, Bill
On 3/23/13 2:11 AM, Rolf Rabenseifner wrote:
Dear Jeff, Craig, Bill, Hubert, Martin, and Tobias,
I expect that this is the total group that discusses the Fortran LOGICAL etc. problem. We learnt that the solutions also touch the PMPI concept. There was a telecon between Craig, Jeff and Bill with the proposal at the end of this email. I like to understand this proposal and therefore I asked Jeff for explanations. I would like to open this for the whole group.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time. Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Monday March 25, - 4 PM Central European Time CET = 11 AM Eastern Daylight Time EST/EDT = 8 AM Pacific Time
I hope these relations are correct. Would this work for you?
At least for Jeff and me, because I want to understand what they discussed with their proposal.
Jeff, can you set up a webex?
Best regards Rolf
----- Original Message -----
From: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>> To: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> Cc: "Craig E Rasmussen" <[email protected] <mailto:[email protected]>> Sent: Friday, March 22, 2013 8:01:23 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Btw, I'm not sure how this is not the least surprise.
We can't change the params to not be logical, yes we must still allow for calling to c. So the overloaded solution is the only one.
And the new Integer param signature must have a new bind name so as to did ambiguity it from the one we published in MPI3. See my previous mail about this.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time.
Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Sent from my phone. No type good.
On Mar 22, 2013, at 8:51 AM, "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> wrote:
Jeff,
can I call you by phone? I would like to talk to you as the binding responsible and MPI chair.
2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not ... This is a hard step and far away from "last surprise".
I would like to learn how it came out as a result of your telecon, before I try to go the next step (e.g. email answer).
Best regards Rolf
----- Original Message -----
From: "Craig Rasmussen" <[email protected] <mailto:[email protected]>> To: "MPI-3 Fortran working group" <[email protected] <mailto:[email protected]>> Cc: "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>> Sent: Friday, March 22, 2013 4:29:36 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
Bill Long, Jeff Squyres, and I had a phone conversation yesterday and came up with a straw man proposal. We believe it fixes the LOGICAL argument problem while satisfying three major goals:
1. Backwards compatibility. 2. Flexibility so that efficient implementations can be produced on a variety of hardware. 3. Not dramatically increase the complexity for the tools community. 4. Ensure that the changes won't be seen by ordinary Fortran MPI users.
The straw man proposal is:
1. Remove the current binding names for MPI functions with LOGICAL dummy arguments (. This must be done because these interfaces break the Fortran standard. 2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not conflict with the current name with LOGICAL arguments. For example, the binding name "MPI_Test_f08" will be replaced by "MPI_Test_int_f08" (exact replacement name open for discussion). The Fortran implementation of the function MPI_Test must call MPI_Test_int_f08 as soon as possible for the tools implementers. 3. Implementation of the Fortran PMPI calling convention will be made optional (we think it already it). If an MPI implementer doesn't provide the MPI_F08 binding names, then it must call the C MPI implementation in a thin Fortran wrapper so that the tools implementers can intercept the call in C. An MPI implementation shall not provide/call both the MPI_F08 binding name and the C name.
Option 2 allows some MPI vendors to implement MPI functions entirely within Fortran (using the MPI_f08 binding names), if this is more efficient.
Option 3 allows some MPI implementations to place Fortran wrappers inside modules where they may be inlined by the compiler for efficiency.
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
-- Bill Long [email protected] <mailto:[email protected]> Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] <mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] <mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101 Click https://www.mailcontrol.com/sr/PSMHhYrWn2nGX2PQPOmvUmmLQnadxIEhPqNHPbHRv8fTz... to report this email as spam.
I am torn between Option 2 and 3 - option 1 seems less clean, but doesn't add much help for tools. Once we have to implement Fortran name mangling for some routines, we can also do it for all. The addition of the optional use of the C interface makes sense, though, since this would long term allow a transition to that, but also gives MPI implementation the option to directly support C only tools. After this discussion and all our back and forth on the Fortran bindings I actually doubt that there is currently a single tool out there that managed to implement the correct Fortran bindings in MPI 2.2, let alone 3.0. This is a very practical argument and I am not sure if it's enough to justify supporting this for standardization, though. I also agree with Hubert - option 3 goes beyond an errata. A fourth option could be to allow BIND-C in the linker names, but leave it up to the MPI implementation to use it or not. This would basically keep the standard as is, but would remove the table on page 609 (plus surrounding text). In this case, tools would still check which version is implemented, but there wouldn't be a mandate of what is used when. If we wanted to combine this with option 3, then there could also be a set of constants that would state that a particular function group is only intercepted in C and all Fortran routines go through them. Martin On Mar 25, 2013, at 1:28 PM, Hubert Ritzdorf <[email protected]> wrote:
I vote for Option 2.
Option 3 is not possible for an errata. This is a severe change in the standard. Your modification of Option 3 makes the standard more difficult (we have again mixing of BIND(C) and non BIND(C) functions) and I can't see a real application advantage.
To your items: Item 2: You have to perform also additional work in the wrappers (for example memory allocation and transformation for sources compiled with -i8; additional error checks) Item 3: Support of PMPI version of Fortran is not really an issue. Item 4b: The tools have to support routines compiled with mpi.h and old objects.
Hubert
________________________________________ From: Bill Long [[email protected]] Sent: Monday, March 25, 2013 7:35 PM To: Craig Rasmussen Cc: Rolf Rabenseifner; MPI-3 Fortran working group; Jeff Squyres (jsquyres); Hubert Ritzdorf; Martin Schulz; Tobias Burnus Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
On 3/25/13 12:50 PM, Craig Rasmussen wrote:
My summary of the three options:
1. Absolute minimal change, remove BIND(C) only for routines with LOGICAL arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL arguments.
2. Remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines.
3. Remove BIND(C) from all routines. Plus allow Fortran implementations to substitute the C PMPI interface standard.
Is this a fair summary?
My vote:
I'm strongly in favor of option 3. It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API. There are several advantages with this implementation: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well.
I like option 3 as well. It uses the C interoperability feature in Fortran the way it was intended/designed. I would add to the above:
6. You can isolate the issues of Fortran types (compiled with -i8, meaning of LOGICAL, character arguments with len > 1) into the Fortran wrappers so that the C library writers do not have to know about these issues. Simplifies the maintenance and testing issues for the C library code.
I would modify option 3 slightly. The implementer might want to write some routine ONLY in Fortran. The routines that convert the status formats between the Fortran and the C forms comes to mind. In that case, BIND(C) would be required, and it would specify the C binding name as the external linker name. This would be needed so that C codes could call the routine.
Cheers, Bill
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
On Mar 25, 2013, at 10:11 AM, Rolf Rabenseifner wrote:
We (Jeff, Bill L., Craig, Hubert, Martin, Tobias, and me (Rolf)) had a telecon today and the outcome are following options:
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem: Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2 - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.)
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: I'm not sure whether there is a real pro?
Bill, can you add the pros, it is your option.
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
My favorite in the moment, i.e. at the starting of the pro-con-discussion: Option 2.
Best regards Rolf
----- Original Message -----
From: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> To: [email protected] <mailto:[email protected]> Cc: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>>, "Craig E Rasmussen" <[email protected] <mailto:[email protected]>>, "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>>, "Tobias Burnus" <[email protected] <mailto:[email protected]>> Sent: Monday, March 25, 2013 12:21:39 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Yes, this freedom (1)-(4) in Bill's email is important.
In our discussion, we forgot one additional reason why we defined it as it is written in MPI-3.0 Section 17.1.5, page 605:
- It should be possible to implement the interfaces with these non-standard Fortran-void-* directives, and later on, to substitute this implementation by new ones that are based on TS 29113. - When doing this substitution, it should be save for applications when the application is not re-compiled. - More than that, it should be possible to combine older .o files with newly TS 29113 based .o files. - For this, we decided, that the implementation has to switch from non-BIND(C) to BIND(C) at least in the MPI_ISEND routine group when switching to MPI_SUBARRAYS_SUPPORTED==.TRUE. - The real trick is that the linker name must be changed. - This all is valid for both, the new mpi_f08 module, and the old mpi module.
A second issue in our discussion is always, which changes we want to do to the mpi_f08 MPI-3.0 user interface. I want to remember that this user interface is defined according following rules. If a user modifies his source to use everywhere mpi_f08, he/she has to modify: - He/she has to substitute the "include 'mpif.h'" or "use mpi" statements by "use mpi_f08". - He/she has to modify all handle and status declarations. - In the executable statements, he/she has to substitute all status(MPI_TAG), status(MPI_SOURCE), status(MPI_ERROR) accesses into status%MPI_TAG, status%MPI_SOURCE, status%MPI_ERROR. - Callback routines need more effort due to the abstract interface definitions. This is touched by ticket #364, i.e., the MPI-3.0 BIND(C) of callbacks had to be removed. - Changes in the interfaces are only -- all handles use the new handle types and in -- MPI_User_function, -- MPI_Datarep_conversion_function, the buffers are done with TYPE(C_TR), VALUE.
This should be all. All other user correct code should directly compile with the new mpi_f08 module. We should not change these things, except the necessary errata in #364.
I see that we have some freedom only in the area, what we do internally (together with linker names) and with PMPI, but not in the user interface. If we add additional user interfaces (e.g. overloading the LOGICAL with INTEGER arguments) we would only make it more complicate.
Best regards Rolf
----- Original Message -----
From: "Bill Long" <[email protected] <mailto:[email protected]>> To: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> Cc: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>>, "Craig E Rasmussen" <[email protected] <mailto:[email protected]>>, "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>>, "Tobias Burnus" <[email protected] <mailto:[email protected]>> Sent: Sunday, March 24, 2013 5:59:38 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind This time is OK for me. I'll look for the webex email.
I would like to note that the various options discussed are not necessarily "all or nothing" propositions. An implementation could use the thin wrappers in a module approach to eliminate the bad Fortran types problems for most of the MPI routines, but also have some routines implemented in a different way, such as (1) all in Fortran that is callable from C, (2) all in C that is called through an interface directly from Fortran, (3) as a fat wrapper in Fortran, or (4) a completely replicated routine for the Fortran implementation. There is no reason that all of the routines have to be the same method - in fact probably good arguments against that in some cases. We just need to have the rules such that the tools have some known name to intercept. Along the lines of "If implementation method X is chosen for routine MPI_xxx, then these are the linker name requirements:.....".
Cheers, Bill
On 3/23/13 2:11 AM, Rolf Rabenseifner wrote:
Dear Jeff, Craig, Bill, Hubert, Martin, and Tobias,
I expect that this is the total group that discusses the Fortran LOGICAL etc. problem. We learnt that the solutions also touch the PMPI concept. There was a telecon between Craig, Jeff and Bill with the proposal at the end of this email. I like to understand this proposal and therefore I asked Jeff for explanations. I would like to open this for the whole group.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time. Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Monday March 25, - 4 PM Central European Time CET = 11 AM Eastern Daylight Time EST/EDT = 8 AM Pacific Time
I hope these relations are correct. Would this work for you?
At least for Jeff and me, because I want to understand what they discussed with their proposal.
Jeff, can you set up a webex?
Best regards Rolf
----- Original Message -----
From: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>> To: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> Cc: "Craig E Rasmussen" <[email protected] <mailto:[email protected]>> Sent: Friday, March 22, 2013 8:01:23 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Btw, I'm not sure how this is not the least surprise.
We can't change the params to not be logical, yes we must still allow for calling to c. So the overloaded solution is the only one.
And the new Integer param signature must have a new bind name so as to did ambiguity it from the one we published in MPI3. See my previous mail about this.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time.
Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Sent from my phone. No type good.
On Mar 22, 2013, at 8:51 AM, "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> wrote:
Jeff,
can I call you by phone? I would like to talk to you as the binding responsible and MPI chair.
2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not ... This is a hard step and far away from "last surprise".
I would like to learn how it came out as a result of your telecon, before I try to go the next step (e.g. email answer).
Best regards Rolf
----- Original Message -----
From: "Craig Rasmussen" <[email protected] <mailto:[email protected]>> To: "MPI-3 Fortran working group" <[email protected] <mailto:[email protected]>> Cc: "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>> Sent: Friday, March 22, 2013 4:29:36 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
Bill Long, Jeff Squyres, and I had a phone conversation yesterday and came up with a straw man proposal. We believe it fixes the LOGICAL argument problem while satisfying three major goals:
1. Backwards compatibility. 2. Flexibility so that efficient implementations can be produced on a variety of hardware. 3. Not dramatically increase the complexity for the tools community. 4. Ensure that the changes won't be seen by ordinary Fortran MPI users.
The straw man proposal is:
1. Remove the current binding names for MPI functions with LOGICAL dummy arguments (. This must be done because these interfaces break the Fortran standard. 2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not conflict with the current name with LOGICAL arguments. For example, the binding name "MPI_Test_f08" will be replaced by "MPI_Test_int_f08" (exact replacement name open for discussion). The Fortran implementation of the function MPI_Test must call MPI_Test_int_f08 as soon as possible for the tools implementers. 3. Implementation of the Fortran PMPI calling convention will be made optional (we think it already it). If an MPI implementer doesn't provide the MPI_F08 binding names, then it must call the C MPI implementation in a thin Fortran wrapper so that the tools implementers can intercept the call in C. An MPI implementation shall not provide/call both the MPI_F08 binding name and the C name.
Option 2 allows some MPI vendors to implement MPI functions entirely within Fortran (using the MPI_f08 binding names), if this is more efficient.
Option 3 allows some MPI implementations to place Fortran wrappers inside modules where they may be inlined by the compiler for efficiency.
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
-- Bill Long [email protected] <mailto:[email protected]> Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] <mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] <mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
Click https://www.mailcontrol.com/sr/PSMHhYrWn2nGX2PQPOmvUmmLQnadxIEhPqNHPbHRv8fTz... to report this email as spam.
________________________________________________________________________ Martin Schulz, [email protected], http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA
On 3/25/13 3:28 PM, Hubert Ritzdorf wrote:
I vote for Option 2.
Option 3 is not possible for an errata. This is a severe change in the standard. Your modification of Option 3 makes the standard more difficult (we have again mixing of BIND(C) and non BIND(C) functions) and I can't see a real application advantage.
To your items: Item 2: You have to perform also additional work in the wrappers (for example memory allocation and transformation for sources compiled with -i8; additional error checks)
There is a trivial amount of work in the wrapper - this is intentional. Here is a simple example from the mpi_f08 module: subroutine MPI_Abort(comm, errorcode, ierror) use,intrinsic :: iso_c_binding, only: c_int use :: mpi_f08_types, only: MPI_Comm use :: mpi_C_interfaces_nobuf, only: C_Comm, MPI_Abort_C TYPE(MPI_Comm), INTENT(IN) :: comm INTEGER, INTENT(IN) :: errorcode INTEGER, OPTIONAL, INTENT(OUT) :: ierror integer(C_Comm) :: comm_c integer(c_int) :: errorcode_c integer(c_int) :: res comm_c = comm%MPI_VAL errorcode_c = errorcode res = MPI_Abort_C (comm_c, errorcode_c) if (present(ierror)) ierror = res end subroutine MPI_Abort The local xxx_c variables are the ones for the C routine; the ones without the _c are the Fortran dummy arguments. I have not encountered a case where memory allocation is needed, at least yet. To accommodate -i8, you only recompile the modules with -i8. There is no change to the module source. This is the intent of the design. The three assignment statements are, in effect, 'casts' between the Fortran and C types. If there is no difference, the compiler could simplify the executable code. In this particular case, the code following the MPI_Abort call to C might not be necessary, since I would expect the routine to not return. But the same pattern is used elsewhere to deal with the error return value.
Item 3: Support of PMPI version of Fortran is not really an issue. Item 4b: The tools have to support routines compiled with mpi.h and old objects.
That is true. For that mode, the text corresponding text from the mpi module (with INTEGER handle for comm and the use of mpi_f08_types removed) is extracted to a file without the module wrapping. Compiling that file will result in a routine with an external symbol of mpi_abort_ (or something similar, depending on the conventions of the compiler). That version would be called only in the case that the user included mpif.h. A particularly aggressive compiler that defers inlining until the link step might even inline this case. Cheers, Bill
Hubert
________________________________________ From: Bill Long [[email protected]] Sent: Monday, March 25, 2013 7:35 PM To: Craig Rasmussen Cc: Rolf Rabenseifner; MPI-3 Fortran working group; Jeff Squyres (jsquyres); Hubert Ritzdorf; Martin Schulz; Tobias Burnus Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
On 3/25/13 12:50 PM, Craig Rasmussen wrote:
My summary of the three options:
1. Absolute minimal change, remove BIND(C) only for routines with LOGICAL arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL arguments.
2. Remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines.
3. Remove BIND(C) from all routines. Plus allow Fortran implementations to substitute the C PMPI interface standard.
Is this a fair summary?
My vote:
I'm strongly in favor of option 3. It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API. There are several advantages with this implementation: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well.
I like option 3 as well. It uses the C interoperability feature in Fortran the way it was intended/designed. I would add to the above:
6. You can isolate the issues of Fortran types (compiled with -i8, meaning of LOGICAL, character arguments with len > 1) into the Fortran wrappers so that the C library writers do not have to know about these issues. Simplifies the maintenance and testing issues for the C library code.
I would modify option 3 slightly. The implementer might want to write some routine ONLY in Fortran. The routines that convert the status formats between the Fortran and the C forms comes to mind. In that case, BIND(C) would be required, and it would specify the C binding name as the external linker name. This would be needed so that C codes could call the routine.
Cheers, Bill
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
On Mar 25, 2013, at 10:11 AM, Rolf Rabenseifner wrote:
We (Jeff, Bill L., Craig, Hubert, Martin, Tobias, and me (Rolf)) had a telecon today and the outcome are following options:
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem: Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2 - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.)
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: I'm not sure whether there is a real pro?
Bill, can you add the pros, it is your option.
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
My favorite in the moment, i.e. at the starting of the pro-con-discussion: Option 2.
Best regards Rolf
----- Original Message -----
From: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> To: [email protected] <mailto:[email protected]> Cc: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>>, "Craig E Rasmussen" <[email protected] <mailto:[email protected]>>, "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>>, "Tobias Burnus" <[email protected] <mailto:[email protected]>> Sent: Monday, March 25, 2013 12:21:39 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Yes, this freedom (1)-(4) in Bill's email is important.
In our discussion, we forgot one additional reason why we defined it as it is written in MPI-3.0 Section 17.1.5, page 605:
- It should be possible to implement the interfaces with these non-standard Fortran-void-* directives, and later on, to substitute this implementation by new ones that are based on TS 29113. - When doing this substitution, it should be save for applications when the application is not re-compiled. - More than that, it should be possible to combine older .o files with newly TS 29113 based .o files. - For this, we decided, that the implementation has to switch from non-BIND(C) to BIND(C) at least in the MPI_ISEND routine group when switching to MPI_SUBARRAYS_SUPPORTED==.TRUE. - The real trick is that the linker name must be changed. - This all is valid for both, the new mpi_f08 module, and the old mpi module.
A second issue in our discussion is always, which changes we want to do to the mpi_f08 MPI-3.0 user interface. I want to remember that this user interface is defined according following rules. If a user modifies his source to use everywhere mpi_f08, he/she has to modify: - He/she has to substitute the "include 'mpif.h'" or "use mpi" statements by "use mpi_f08". - He/she has to modify all handle and status declarations. - In the executable statements, he/she has to substitute all status(MPI_TAG), status(MPI_SOURCE), status(MPI_ERROR) accesses into status%MPI_TAG, status%MPI_SOURCE, status%MPI_ERROR. - Callback routines need more effort due to the abstract interface definitions. This is touched by ticket #364, i.e., the MPI-3.0 BIND(C) of callbacks had to be removed. - Changes in the interfaces are only -- all handles use the new handle types and in -- MPI_User_function, -- MPI_Datarep_conversion_function, the buffers are done with TYPE(C_TR), VALUE.
This should be all. All other user correct code should directly compile with the new mpi_f08 module. We should not change these things, except the necessary errata in #364.
I see that we have some freedom only in the area, what we do internally (together with linker names) and with PMPI, but not in the user interface. If we add additional user interfaces (e.g. overloading the LOGICAL with INTEGER arguments) we would only make it more complicate.
Best regards Rolf
----- Original Message -----
From: "Bill Long" <[email protected] <mailto:[email protected]>> To: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> Cc: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>>, "Craig E Rasmussen" <[email protected] <mailto:[email protected]>>, "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>>, "Tobias Burnus" <[email protected] <mailto:[email protected]>> Sent: Sunday, March 24, 2013 5:59:38 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind This time is OK for me. I'll look for the webex email.
I would like to note that the various options discussed are not necessarily "all or nothing" propositions. An implementation could use the thin wrappers in a module approach to eliminate the bad Fortran types problems for most of the MPI routines, but also have some routines implemented in a different way, such as (1) all in Fortran that is callable from C, (2) all in C that is called through an interface directly from Fortran, (3) as a fat wrapper in Fortran, or (4) a completely replicated routine for the Fortran implementation. There is no reason that all of the routines have to be the same method - in fact probably good arguments against that in some cases. We just need to have the rules such that the tools have some known name to intercept. Along the lines of "If implementation method X is chosen for routine MPI_xxx, then these are the linker name requirements:.....".
Cheers, Bill
On 3/23/13 2:11 AM, Rolf Rabenseifner wrote:
Dear Jeff, Craig, Bill, Hubert, Martin, and Tobias,
I expect that this is the total group that discusses the Fortran LOGICAL etc. problem. We learnt that the solutions also touch the PMPI concept. There was a telecon between Craig, Jeff and Bill with the proposal at the end of this email. I like to understand this proposal and therefore I asked Jeff for explanations. I would like to open this for the whole group.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time. Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Monday March 25, - 4 PM Central European Time CET = 11 AM Eastern Daylight Time EST/EDT = 8 AM Pacific Time
I hope these relations are correct. Would this work for you?
At least for Jeff and me, because I want to understand what they discussed with their proposal.
Jeff, can you set up a webex?
Best regards Rolf
----- Original Message -----
From: "Jeff Squyres (jsquyres)" <[email protected] <mailto:[email protected]>> To: "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> Cc: "Craig E Rasmussen" <[email protected] <mailto:[email protected]>> Sent: Friday, March 22, 2013 8:01:23 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Btw, I'm not sure how this is not the least surprise.
We can't change the params to not be logical, yes we must still allow for calling to c. So the overloaded solution is the only one.
And the new Integer param signature must have a new bind name so as to did ambiguity it from the one we published in MPI3. See my previous mail about this.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time.
Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Sent from my phone. No type good.
On Mar 22, 2013, at 8:51 AM, "Rolf Rabenseifner" <[email protected] <mailto:[email protected]>> wrote:
Jeff,
can I call you by phone? I would like to talk to you as the binding responsible and MPI chair.
2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not ... This is a hard step and far away from "last surprise".
I would like to learn how it came out as a result of your telecon, before I try to go the next step (e.g. email answer).
Best regards Rolf
----- Original Message -----
From: "Craig Rasmussen" <[email protected] <mailto:[email protected]>> To: "MPI-3 Fortran working group" <[email protected] <mailto:[email protected]>> Cc: "Hubert Ritzdorf" <[email protected] <mailto:[email protected]>>, "Martin Schulz" <[email protected] <mailto:[email protected]>> Sent: Friday, March 22, 2013 4:29:36 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
Bill Long, Jeff Squyres, and I had a phone conversation yesterday and came up with a straw man proposal. We believe it fixes the LOGICAL argument problem while satisfying three major goals:
1. Backwards compatibility. 2. Flexibility so that efficient implementations can be produced on a variety of hardware. 3. Not dramatically increase the complexity for the tools community. 4. Ensure that the changes won't be seen by ordinary Fortran MPI users.
The straw man proposal is:
1. Remove the current binding names for MPI functions with LOGICAL dummy arguments (. This must be done because these interfaces break the Fortran standard. 2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not conflict with the current name with LOGICAL arguments. For example, the binding name "MPI_Test_f08" will be replaced by "MPI_Test_int_f08" (exact replacement name open for discussion). The Fortran implementation of the function MPI_Test must call MPI_Test_int_f08 as soon as possible for the tools implementers. 3. Implementation of the Fortran PMPI calling convention will be made optional (we think it already it). If an MPI implementer doesn't provide the MPI_F08 binding names, then it must call the C MPI implementation in a thin Fortran wrapper so that the tools implementers can intercept the call in C. An MPI implementation shall not provide/call both the MPI_F08 binding name and the C name.
Option 2 allows some MPI vendors to implement MPI functions entirely within Fortran (using the MPI_f08 binding names), if this is more efficient.
Option 3 allows some MPI implementations to place Fortran wrappers inside modules where they may be inlined by the compiler for efficiency.
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
-- Bill Long [email protected] <mailto:[email protected]> Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] <mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] <mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
Click https://www.mailcontrol.com/sr/PSMHhYrWn2nGX2PQPOmvUmmLQnadxIEhPqNHPbHRv8fTz... to report this email as spam.
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
On Mar 25, 2013, at 2:35 PM, Bill Long <[email protected]> wrote:
I'm strongly in favor of option 3. It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API.
Remember that Open MPI can never do this approach since our C handles != Fortran handles, and Fortran wrappers will not be able to reach back into the back-end C MPI objects (at least, not without a truckload of extra interop code -- the Fortran bindings layer would have to gain an understanding of the back-end MPI C objects, and that seems like a terrible idea).
6. You can isolate the issues of Fortran types (compiled with -i8, meaning of LOGICAL, character arguments with len > 1) into the Fortran wrappers so that the C library writers do not have to know about these issues. Simplifies the maintenance and testing issues for the C library code.
There's a flaw in this logic. Yes, you divorce the *bindings* between C and Fortran, but the MPI library still has to know whether INTEGER is 4 or 8 bytes (for example) so that it can send/receive them properly. I.e., the MPI datatypes are intrinsically tied to the language's native types, and are orthogonal to how the bindings are implemented. -- Jeff Squyres [email protected] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
On 3/26/13 8:38 AM, Jeff Squyres (jsquyres) wrote:
On Mar 25, 2013, at 2:35 PM, Bill Long <[email protected]> wrote:
6. You can isolate the issues of Fortran types (compiled with -i8, meaning of LOGICAL, character arguments with len > 1) into the Fortran wrappers so that the C library writers do not have to know about these issues. Simplifies the maintenance and testing issues for the C library code.
There's a flaw in this logic.
Yes, you divorce the *bindings* between C and Fortran, but the MPI library still has to know whether INTEGER is 4 or 8 bytes (for example) so that it can send/receive them properly. I.e., the MPI datatypes are intrinsically tied to the language's native types, and are orthogonal to how the bindings are implemented.
I don't think there is a problem. To accommodate -i8 you have to recompile the MPI module with the -i8 flag set. We've been shipping two mpi modules as long as I can remember; the compiler scripts pick the right one based on how the user code is compiled. Same as how we select the right version of other libraries that also have Fortran default type arguments. Within the MPI module source the values of MPI_INTEGER and the similar data types for other Fortran default types are set based on detection of the actual sizes of the default types (which change with the compiler options). If you are using the module compiled with -i8, MPI_INTEGER will encode the information for an 8-byte integer. Cheers, Bill -- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
On Mar 26, 2013, at 12:16 PM, Bill Long <[email protected]> wrote:
I don't think there is a problem. To accommodate -i8 you have to recompile the MPI module with the -i8 flag set. We've been shipping two mpi modules as long as I can remember; the compiler scripts pick the right one based on how the user code is compiled. Same as how we select the right version of other libraries that also have Fortran default type arguments. Within the MPI module source the values of MPI_INTEGER and the similar data types for other Fortran default types are set based on detection of the actual sizes of the default types (which change with the compiler options). If you are using the module compiled with -i8, MPI_INTEGER will encode the information for an 8-byte integer.
But not all Fortran types have corresponding C types (from a standards/strict perspective), right? (e.g., logical -- and I'm not sure about the various complex types...?) So the MPI implementation needs to know that it has to create some synthetic / "good enough" MPI datatypes based on actual Fortran types. ...at least, that's what OMPI does... -- Jeff Squyres [email protected] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
On 3/26/13 11:26 AM, Jeff Squyres (jsquyres) wrote:
On Mar 26, 2013, at 12:16 PM, Bill Long <[email protected]> wrote:
I don't think there is a problem. To accommodate -i8 you have to recompile the MPI module with the -i8 flag set. We've been shipping two mpi modules as long as I can remember; the compiler scripts pick the right one based on how the user code is compiled. Same as how we select the right version of other libraries that also have Fortran default type arguments. Within the MPI module source the values of MPI_INTEGER and the similar data types for other Fortran default types are set based on detection of the actual sizes of the default types (which change with the compiler options). If you are using the module compiled with -i8, MPI_INTEGER will encode the information for an 8-byte integer.
But not all Fortran types have corresponding C types (from a standards/strict perspective), right? (e.g., logical -- and I'm not sure about the various complex types...?) So the MPI implementation needs to know that it has to create some synthetic / "good enough" MPI datatypes based on actual Fortran types.
...at least, that's what OMPI does...
What I did was less sophisticated (at least in terms of mpich internals). The values for the various handles at page 666 (how appropriate) lines 5-10, and 34, and page 667 lines 15-17 are not new. I took the values out of the mpich mpif.h files generated for 32-bit defaults and 64-bit defaults. Those values are assigned to internal constants in the module. The definition of MPI_INTEGER selects between the two options depending on KIND(0), and similarly for the others. The goal is to produce the same values in the module as in the mpif.h files, only automatically selected based on the default type size specified by the compiler options used to compile the module. Cheers, Bill
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
After all the replies, I tried to update the list of options. Please answer at the end the list of "straw votes", may be also with "more clarification needed". Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem. #364 removes BIND(C) only for routines with LOGICAL arguments and for all predefined callback routines and on all ABSTRACT INTERFACE definitions for user written callbacks, and all MPI routines that have such callback PROCEDURE dummy arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL and PROCEDURE arguments, and for all predefined callbacks. Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization. Option 2a - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.) The real linker name is defined through the compiler's name mangling for these names. This means, we remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines. Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface. Option 2b - Same as Option 2a, but allow the four options MPI_Send_f08d (mpi_f08 interface with array descriptor for buffers) MPI_Send_fd (mpi and mpif.h with array descriptor for buffers) MPI_Send_f08 (mpi_f08 interface with void *, via IGNORE_TKR etc.) MPI_Send (mpi and mpif.h with void *, via IGNORE_TKR etc.) only for routines in the two routine groups MPI_SEND and MPI_ISEND. For all other routine groups, we provide only the options MPI_Send_f08 (mpi_f08 interface) MPI_Send (mpi and mpif.h) because there are no DIMENSION(..),TYPE(*) buffers. Additional Pro: Reduces significantly the routine sets that must be handled with the PMPI interface. No additional Con. Option 2c - Same as 2b, but we allow that all routine groups are done with BIND(C), except the callback routine group MPI_COMM_DUP_FN (and also MPI_TEST (with the LOGICAL args), MPI_OP_CREATE, MPI_REGISTER_DATAREP, and MPI_COMM_CREATE_KEYVAL as long as the required Fortran-C support is missing). For this, the following linker names are used, but without name mangling: MPI_Send_f08d (mpi_f08 interface with array descriptor for buffers) MPI_Send_fd (mpi and mpif.h with array descriptor for buffers) MPI_Send_f08 (mpi_f08 interface with void *, via IGNORE_TKR etc.) MPI_Send_f (mpi and mpif.h with void *, via IGNORE_TKR etc.) Pro: In long term, when also LOGICAL and Fortran-callback wiil have become interoperable, an implementation can choose to internally implement all in C based on BIND(C), except these 9 predefined callbacks. This is an ideal basis for option 2d. Con: The tools people have more work. Option 2d - Same as 2c, but we define one additional small routine group: Routine group MPI_STATUS_F2F08 contains MPI_STATUS_F2F08 and MPI_STATUS_F082F. Pro: An implementor may want to choose that this routine group is implemented in Fortran with BIND(C) and the C MPI_Status... routines internally call these BIND(C) Fortran routines. Con: One more routine group (a very weak argument). Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc. Pro: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well. Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface. Open question: Does Option 3 require MPI-3.1 or can it be done as an MPI-3.0 erratum. Pro erratum: One goal of MPI-3.0 mpi_f08 was to allow a fully standardized implementation, i.e., without compiler dependent software (e.g., name mangling). MPI-3.0 was supposed to do this. The Option 3 will keep this quality. It is a change of the PMPI Fortran interface, but MPI-3.0 allows such changes (See removal of C++). It is an erratum that requires at least double-vote, if we allow this also for the mpi module and mpif.h. Con erratum: It is a real change for the user-interface of Fortran's PMPI. Therefore: - Option 3a: allowing this only for the mpi_f08 module - Option 3b: allowing it for all three Fortran support methods. - Option 3c: To allow this Option 3 independently for each routine group. Option 4: (needed for 2c, for all others it is independent) - Substituting in mpi_f08 all CHARACTER(LEN=xxx) by CHARACTER(LEN=1),DIMENSION(xxx) with xxx = a fixed constant. As far as I understand, this cannot be done for CHARACTER(LEN=*) because otherwise the MPI routine cannot learn about the length of the string, see, e.g., argv array in MPI_COMM_SPAWN. Pro: As far as I understand, this is needed for BIND(C) for the MPI routines that have CHARACTER(LEN=xxx) dummy arguments. Con: Non-consistent solution for a bug in TS29113 that should be fixed in TS29113. The mpi and the mpi_f08 moudlues will be inconsistent to each other. We do not want to change the old mpi module and mpif.h Fortran interface. With current CHARACTER(LEN=xxx), the MPI routine can check at runtime whether the provided actual argument is long enough. With CHARACTER(LEN=1),DIMENSION(xxx), such a check is impossible. Bill and Craig, please correct me if I'm wrong. Is this bug fixable in TS29113, or is there a reason, why CHARACTER(LEN=xxx) is not interoperable with C? Open questions: - Is anybody for Option 1? - If we are doing Option 2, which version is your favorite 2a, 2b, 2c, 2d? - For each Option 2a-d do you say - absolute necessary - is okay for me - is not okay for me - Do you want Option 3? - If yes, do you want 3a, or 3b? - Can we do 3a as erratum (instead of postponing to MPI-3.1)? - Can we do 3b as erratum (instead of postponing to MPI-3.1)? - If yes, do you additionally want 3c? - Do you want Option 4? ----------- My personal answers: - Is anybody for Option 1? ** No, it is to few. - If we are doing Option 2, which version is your favorite 2a, 2b, 2c, 2d? ** I would prefer 2d. - For each Option 2a-d do you say - absolute necessary - is okay for me - is not okay for me ** 2a: absolute necessary ** 2b: absolute necessary ** 2c: is okay ** 2d: is okay - Do you want Option 3? ** Yes - If yes, do you want 3a, or 3b? ** 3b - Can we do 3a as erratum (instead of postponing to MPI-3.1)? ** Yes - Can we do 3b as erratum (instead of postponing to MPI-3.1)? ** Yes - If yes to 3a or b, do you additionally want 3c? ** Yes - Do you want Option 4? ** No, should be done by the Fortran standardization as correction to TS29113. Best regards Rolf ----- Original Message -----
From: "Bill Long" <[email protected]> To: "Jeff Squyres (jsquyres)" <[email protected]> Cc: "Craig Rasmussen" <[email protected]>, "Rolf Rabenseifner" <[email protected]>, "MPI-3 Fortran working group" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Wednesday, March 27, 2013 12:03:27 AM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind On 3/26/13 11:26 AM, Jeff Squyres (jsquyres) wrote:
On Mar 26, 2013, at 12:16 PM, Bill Long <[email protected]> wrote:
I don't think there is a problem. To accommodate -i8 you have to recompile the MPI module with the -i8 flag set. We've been shipping two mpi modules as long as I can remember; the compiler scripts pick the right one based on how the user code is compiled. Same as how we select the right version of other libraries that also have Fortran default type arguments. Within the MPI module source the values of MPI_INTEGER and the similar data types for other Fortran default types are set based on detection of the actual sizes of the default types (which change with the compiler options). If you are using the module compiled with -i8, MPI_INTEGER will encode the information for an 8-byte integer.
But not all Fortran types have corresponding C types (from a standards/strict perspective), right? (e.g., logical -- and I'm not sure about the various complex types...?) So the MPI implementation needs to know that it has to create some synthetic / "good enough" MPI datatypes based on actual Fortran types.
...at least, that's what OMPI does...
What I did was less sophisticated (at least in terms of mpich internals). The values for the various handles at page 666 (how appropriate) lines 5-10, and 34, and page 667 lines 15-17 are not new. I took the values out of the mpich mpif.h files generated for 32-bit defaults and 64-bit defaults. Those values are assigned to internal constants in the module. The definition of MPI_INTEGER selects between the two options depending on KIND(0), and similarly for the others. The goal is to produce the same values in the module as in the mpif.h files, only automatically selected based on the default type size specified by the compiler options used to compile the module.
Cheers, Bill
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
Rolf, I'm sorry to say this but it all seems way too complicated to me. There are too many subgroups, options, special cases, ... When things are too complicated, usually this means that something is wrong. Therefore I vote for new option 2x: There is a problem with the PMPI interfaces for MPI-3 in Fortran. Therefore as an erratum, we pull them from the standard and fix things up in MPI 3.1/4.0. This means that the BIND(C) names are removed AND they are not replaced by anything in the erratum. I'm of the strong opinion that while you CAN remove something that is broken in an erratum, you CAN'T replace the old names with a complete hash of NEW names that will be placed in the MPI namespace forever. Furthermore, these proposed NEW names are completely outside of any language standard and will place too many restrictions on how the MPI 3.0 standard must be implemented. For example, you have to be very careful in how you implement the new names, otherwise an aggressive compiler will optimize the calls away and the proposed NEW, NON-STANDARD symbol names, will disappear (i.e., they will never be called). I plan on attending the next MPI meeting in June to discuss these issues in front of the full MPI Forum. Craig Rasmussen CAS Scientific Programmer [email protected] On Mar 27, 2013, at 11:57 AM, Rolf Rabenseifner wrote:
After all the replies, I tried to update the list of options. Please answer at the end the list of "straw votes", may be also with "more clarification needed".
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem. #364 removes BIND(C) only for routines with LOGICAL arguments and for all predefined callback routines and on all ABSTRACT INTERFACE definitions for user written callbacks, and all MPI routines that have such callback PROCEDURE dummy arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL and PROCEDURE arguments, and for all predefined callbacks.
Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2a - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.) The real linker name is defined through the compiler's name mangling for these names.
This means, we remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines.
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 2b - Same as Option 2a, but allow the four options MPI_Send_f08d (mpi_f08 interface with array descriptor for buffers) MPI_Send_fd (mpi and mpif.h with array descriptor for buffers) MPI_Send_f08 (mpi_f08 interface with void *, via IGNORE_TKR etc.) MPI_Send (mpi and mpif.h with void *, via IGNORE_TKR etc.) only for routines in the two routine groups MPI_SEND and MPI_ISEND. For all other routine groups, we provide only the options MPI_Send_f08 (mpi_f08 interface) MPI_Send (mpi and mpif.h) because there are no DIMENSION(..),TYPE(*) buffers.
Additional Pro: Reduces significantly the routine sets that must be handled with the PMPI interface. No additional Con.
Option 2c - Same as 2b, but we allow that all routine groups are done with BIND(C), except the callback routine group MPI_COMM_DUP_FN (and also MPI_TEST (with the LOGICAL args), MPI_OP_CREATE, MPI_REGISTER_DATAREP, and MPI_COMM_CREATE_KEYVAL as long as the required Fortran-C support is missing). For this, the following linker names are used, but without name mangling: MPI_Send_f08d (mpi_f08 interface with array descriptor for buffers) MPI_Send_fd (mpi and mpif.h with array descriptor for buffers) MPI_Send_f08 (mpi_f08 interface with void *, via IGNORE_TKR etc.) MPI_Send_f (mpi and mpif.h with void *, via IGNORE_TKR etc.)
Pro: In long term, when also LOGICAL and Fortran-callback wiil have become interoperable, an implementation can choose to internally implement all in C based on BIND(C), except these 9 predefined callbacks. This is an ideal basis for option 2d. Con: The tools people have more work.
Option 2d - Same as 2c, but we define one additional small routine group:
Routine group MPI_STATUS_F2F08 contains MPI_STATUS_F2F08 and MPI_STATUS_F082F.
Pro: An implementor may want to choose that this routine group is implemented in Fortran with BIND(C) and the C MPI_Status... routines internally call these BIND(C) Fortran routines. Con: One more routine group (a very weak argument).
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well.
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
Open question: Does Option 3 require MPI-3.1 or can it be done as an MPI-3.0 erratum. Pro erratum: One goal of MPI-3.0 mpi_f08 was to allow a fully standardized implementation, i.e., without compiler dependent software (e.g., name mangling). MPI-3.0 was supposed to do this. The Option 3 will keep this quality. It is a change of the PMPI Fortran interface, but MPI-3.0 allows such changes (See removal of C++). It is an erratum that requires at least double-vote, if we allow this also for the mpi module and mpif.h. Con erratum: It is a real change for the user-interface of Fortran's PMPI.
Therefore: - Option 3a: allowing this only for the mpi_f08 module - Option 3b: allowing it for all three Fortran support methods. - Option 3c: To allow this Option 3 independently for each routine group.
Option 4: (needed for 2c, for all others it is independent) - Substituting in mpi_f08 all CHARACTER(LEN=xxx) by CHARACTER(LEN=1),DIMENSION(xxx) with xxx = a fixed constant.
As far as I understand, this cannot be done for CHARACTER(LEN=*) because otherwise the MPI routine cannot learn about the length of the string, see, e.g., argv array in MPI_COMM_SPAWN.
Pro: As far as I understand, this is needed for BIND(C) for the MPI routines that have CHARACTER(LEN=xxx) dummy arguments. Con: Non-consistent solution for a bug in TS29113 that should be fixed in TS29113. The mpi and the mpi_f08 moudlues will be inconsistent to each other. We do not want to change the old mpi module and mpif.h Fortran interface. With current CHARACTER(LEN=xxx), the MPI routine can check at runtime whether the provided actual argument is long enough. With CHARACTER(LEN=1),DIMENSION(xxx), such a check is impossible.
Bill and Craig, please correct me if I'm wrong. Is this bug fixable in TS29113, or is there a reason, why CHARACTER(LEN=xxx) is not interoperable with C?
Open questions:
- Is anybody for Option 1? - If we are doing Option 2, which version is your favorite 2a, 2b, 2c, 2d? - For each Option 2a-d do you say - absolute necessary - is okay for me - is not okay for me - Do you want Option 3? - If yes, do you want 3a, or 3b? - Can we do 3a as erratum (instead of postponing to MPI-3.1)? - Can we do 3b as erratum (instead of postponing to MPI-3.1)? - If yes, do you additionally want 3c? - Do you want Option 4?
-----------
My personal answers: - Is anybody for Option 1? ** No, it is to few. - If we are doing Option 2, which version is your favorite 2a, 2b, 2c, 2d? ** I would prefer 2d. - For each Option 2a-d do you say - absolute necessary - is okay for me - is not okay for me ** 2a: absolute necessary ** 2b: absolute necessary ** 2c: is okay ** 2d: is okay - Do you want Option 3? ** Yes - If yes, do you want 3a, or 3b? ** 3b - Can we do 3a as erratum (instead of postponing to MPI-3.1)? ** Yes - Can we do 3b as erratum (instead of postponing to MPI-3.1)? ** Yes - If yes to 3a or b, do you additionally want 3c? ** Yes - Do you want Option 4? ** No, should be done by the Fortran standardization as correction to TS29113.
Best regards Rolf
----- Original Message -----
From: "Bill Long" <[email protected]> To: "Jeff Squyres (jsquyres)" <[email protected]> Cc: "Craig Rasmussen" <[email protected]>, "Rolf Rabenseifner" <[email protected]>, "MPI-3 Fortran working group" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Wednesday, March 27, 2013 12:03:27 AM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind On 3/26/13 11:26 AM, Jeff Squyres (jsquyres) wrote:
On Mar 26, 2013, at 12:16 PM, Bill Long <[email protected]> wrote:
I don't think there is a problem. To accommodate -i8 you have to recompile the MPI module with the -i8 flag set. We've been shipping two mpi modules as long as I can remember; the compiler scripts pick the right one based on how the user code is compiled. Same as how we select the right version of other libraries that also have Fortran default type arguments. Within the MPI module source the values of MPI_INTEGER and the similar data types for other Fortran default types are set based on detection of the actual sizes of the default types (which change with the compiler options). If you are using the module compiled with -i8, MPI_INTEGER will encode the information for an 8-byte integer.
But not all Fortran types have corresponding C types (from a standards/strict perspective), right? (e.g., logical -- and I'm not sure about the various complex types...?) So the MPI implementation needs to know that it has to create some synthetic / "good enough" MPI datatypes based on actual Fortran types.
...at least, that's what OMPI does...
What I did was less sophisticated (at least in terms of mpich internals). The values for the various handles at page 666 (how appropriate) lines 5-10, and 34, and page 667 lines 15-17 are not new. I took the values out of the mpich mpif.h files generated for 32-bit defaults and 64-bit defaults. Those values are assigned to internal constants in the module. The definition of MPI_INTEGER selects between the two options depending on KIND(0), and similarly for the others. The goal is to produce the same values in the module as in the mpif.h files, only automatically selected based on the default type size specified by the compiler options used to compile the module.
Cheers, Bill
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
On 3/27/13 5:42 PM, Craig Rasmussen wrote:
Rolf, I'm sorry to say this but it all seems way too complicated to me. There are too many subgroups, options, special cases, ... When things are too complicated, usually this means that something is wrong.
To carry the "too complicated" argument (much) farther, and only partly tongue-in-cheek, the MPI Fortran forum should identify the 20 or so MPI functions that are adequate to link 99.9% of all Fortran programs using MPI, and completely delete the Fortran interface for all the rest. The 0.1% power users can easily figure out how to write their own wrappers and interface blocks to access the other C library routines directly. Had C interoperability in Fortran been well established at the time MPI was started, I would hope that there would have never been separate Fortran interfaces in the first place. This would be a big step in that direction, and would have minimal backward compatibility effects on the vast majority of users. Obviously, I'm not suggesting this as an immediate change to solve the current problems, but I think it is worth pondering. Cheers, Bill -- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
I beleive we can really fix all with one erratum. You all convinced me that the Option 3 strategy for PMPI through C is good. My total plan is without any long options. It is one vote for correcting the inconsistencies (LOGICAL and CHARACTER) with an appropriate partial rewriting of Section 17.1.5. Here the shortcut: - Only for optical: Removal of the BIND(C) from all mpi_f08 interface definitions. It has no real meaning in current MPI-3.0, because all about BIND(C) is defined in 17.1.5 and not in the 360 individual interface definitions. - The void-pointer and C-descriptor solution path only for the MPI_SEND and MPI_ISEND routine group. Without BIND(C). - Additionally, we allow that an MPI implementation may change some routine groups into BIND(C). (Of course, this is not possible for the routines with callback arguments, and also not for the predefined callbacks.) (With current Fortran 2008 + TS29113, it is also not possible for the routines with LOGICAL arguments and it requires a change in the interface syntax (not in the semantics) for the CHARACTER(NEN=xxx) arguments. But this is also trivial as obias showed us.) - For PMPI we allow to choose one of two solutions for each routine group: - Well-defined linker names on the MPI API level. (This is needed, if an implementation wants to continue the current PMPI Fortran service.) - Using thin Fortran wrappers to the MPI C interface and profiling is done only on the C interface. This includes special C interfaces for the routines with buffer C-descriptors. (Of course, this is not possible for the routines with callback arguments, because Fortran callbacks are not C interoperable, but this is as it was the last 15 years, i.e., every implementer has a solution.) Except for the callbacks, the whole Fortran MPI lib can be then implemented with portable wrapper software, i.e., without any name mangling problem. This is also true for MPI_STATUS_F2F08 and vice versa, but we put these two routines in an extra group. This is the goal, and it will happen only if we prepare it perfectly. And there is no need to wait until MPI-3.1/4.0. If the MPI is inconsistent, you are allowed to fix it through an errata. All important errata in the past changed the MPI API! The WG will decide, whether it needs - one vote in one meeting, or - two votes plus reading in three meetings. (Because it is mainlyonly in 17.1.5, I would tend to one vote to save the forum time for more important stuff in other areas.) You see, I'm convinced by nearly all of the arguments in all the emails. This is Craig's option 2x (which is identical to Bill's wish) with still allowing that an implementer may want to provide the full Fortran PMPI interface. It also solves the CHARACTER problems listed by Tobias, and of course all LOGICAL problems. If everybody switches to the wrapper solutions, the tools people can also be happy. I hope that all can be happy with proposal-draft. Okay? Best regards Rolf ----- Original Message -----
From: "Craig Rasmussen" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "MPI-3 Fortran working group" <[email protected]>, [email protected], "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]>, "Jeff Squyres (jsquyres)" <[email protected]> Sent: Wednesday, March 27, 2013 11:42:05 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Rolf, I'm sorry to say this but it all seems way too complicated to me. There are too many subgroups, options, special cases, ... When things are too complicated, usually this means that something is wrong.
Therefore I vote for new option 2x:
There is a problem with the PMPI interfaces for MPI-3 in Fortran. Therefore as an erratum, we pull them from the standard and fix things up in MPI 3.1/4.0. This means that the BIND(C) names are removed AND they are not replaced by anything in the erratum.
I'm of the strong opinion that while you CAN remove something that is broken in an erratum, you CAN'T replace the old names with a complete hash of NEW names that will be placed in the MPI namespace forever. Furthermore, these proposed NEW names are completely outside of any language standard and will place too many restrictions on how the MPI 3.0 standard must be implemented. For example, you have to be very careful in how you implement the new names, otherwise an aggressive compiler will optimize the calls away and the proposed NEW, NON-STANDARD symbol names, will disappear (i.e., they will never be called).
I plan on attending the next MPI meeting in June to discuss these issues in front of the full MPI Forum.
Craig Rasmussen CAS Scientific Programmer [email protected]
On Mar 27, 2013, at 11:57 AM, Rolf Rabenseifner wrote:
After all the replies, I tried to update the list of options. Please answer at the end the list of "straw votes", may be also with "more clarification needed".
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem. #364 removes BIND(C) only for routines with LOGICAL arguments and for all predefined callback routines and on all ABSTRACT INTERFACE definitions for user written callbacks, and all MPI routines that have such callback PROCEDURE dummy arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL and PROCEDURE arguments, and for all predefined callbacks.
Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2a - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.) The real linker name is defined through the compiler's name mangling for these names.
This means, we remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines.
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 2b - Same as Option 2a, but allow the four options MPI_Send_f08d (mpi_f08 interface with array descriptor for buffers) MPI_Send_fd (mpi and mpif.h with array descriptor for buffers) MPI_Send_f08 (mpi_f08 interface with void *, via IGNORE_TKR etc.) MPI_Send (mpi and mpif.h with void *, via IGNORE_TKR etc.) only for routines in the two routine groups MPI_SEND and MPI_ISEND. For all other routine groups, we provide only the options MPI_Send_f08 (mpi_f08 interface) MPI_Send (mpi and mpif.h) because there are no DIMENSION(..),TYPE(*) buffers.
Additional Pro: Reduces significantly the routine sets that must be handled with the PMPI interface. No additional Con.
Option 2c - Same as 2b, but we allow that all routine groups are done with BIND(C), except the callback routine group MPI_COMM_DUP_FN (and also MPI_TEST (with the LOGICAL args), MPI_OP_CREATE, MPI_REGISTER_DATAREP, and MPI_COMM_CREATE_KEYVAL as long as the required Fortran-C support is missing). For this, the following linker names are used, but without name mangling: MPI_Send_f08d (mpi_f08 interface with array descriptor for buffers) MPI_Send_fd (mpi and mpif.h with array descriptor for buffers) MPI_Send_f08 (mpi_f08 interface with void *, via IGNORE_TKR etc.) MPI_Send_f (mpi and mpif.h with void *, via IGNORE_TKR etc.)
Pro: In long term, when also LOGICAL and Fortran-callback wiil have become interoperable, an implementation can choose to internally implement all in C based on BIND(C), except these 9 predefined callbacks. This is an ideal basis for option 2d. Con: The tools people have more work.
Option 2d - Same as 2c, but we define one additional small routine group:
Routine group MPI_STATUS_F2F08 contains MPI_STATUS_F2F08 and MPI_STATUS_F082F.
Pro: An implementor may want to choose that this routine group is implemented in Fortran with BIND(C) and the C MPI_Status... routines internally call these BIND(C) Fortran routines. Con: One more routine group (a very weak argument).
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well.
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
Open question: Does Option 3 require MPI-3.1 or can it be done as an MPI-3.0 erratum. Pro erratum: One goal of MPI-3.0 mpi_f08 was to allow a fully standardized implementation, i.e., without compiler dependent software (e.g., name mangling). MPI-3.0 was supposed to do this. The Option 3 will keep this quality. It is a change of the PMPI Fortran interface, but MPI-3.0 allows such changes (See removal of C++). It is an erratum that requires at least double-vote, if we allow this also for the mpi module and mpif.h. Con erratum: It is a real change for the user-interface of Fortran's PMPI.
Therefore: - Option 3a: allowing this only for the mpi_f08 module - Option 3b: allowing it for all three Fortran support methods. - Option 3c: To allow this Option 3 independently for each routine group.
Option 4: (needed for 2c, for all others it is independent) - Substituting in mpi_f08 all CHARACTER(LEN=xxx) by CHARACTER(LEN=1),DIMENSION(xxx) with xxx = a fixed constant.
As far as I understand, this cannot be done for CHARACTER(LEN=*) because otherwise the MPI routine cannot learn about the length of the string, see, e.g., argv array in MPI_COMM_SPAWN.
Pro: As far as I understand, this is needed for BIND(C) for the MPI routines that have CHARACTER(LEN=xxx) dummy arguments. Con: Non-consistent solution for a bug in TS29113 that should be fixed in TS29113. The mpi and the mpi_f08 moudlues will be inconsistent to each other. We do not want to change the old mpi module and mpif.h Fortran interface. With current CHARACTER(LEN=xxx), the MPI routine can check at runtime whether the provided actual argument is long enough. With CHARACTER(LEN=1),DIMENSION(xxx), such a check is impossible.
Bill and Craig, please correct me if I'm wrong. Is this bug fixable in TS29113, or is there a reason, why CHARACTER(LEN=xxx) is not interoperable with C?
Open questions:
- Is anybody for Option 1? - If we are doing Option 2, which version is your favorite 2a, 2b, 2c, 2d? - For each Option 2a-d do you say - absolute necessary - is okay for me - is not okay for me - Do you want Option 3? - If yes, do you want 3a, or 3b? - Can we do 3a as erratum (instead of postponing to MPI-3.1)? - Can we do 3b as erratum (instead of postponing to MPI-3.1)? - If yes, do you additionally want 3c? - Do you want Option 4?
-----------
My personal answers: - Is anybody for Option 1? ** No, it is to few. - If we are doing Option 2, which version is your favorite 2a, 2b, 2c, 2d? ** I would prefer 2d. - For each Option 2a-d do you say - absolute necessary - is okay for me - is not okay for me ** 2a: absolute necessary ** 2b: absolute necessary ** 2c: is okay ** 2d: is okay - Do you want Option 3? ** Yes - If yes, do you want 3a, or 3b? ** 3b - Can we do 3a as erratum (instead of postponing to MPI-3.1)? ** Yes - Can we do 3b as erratum (instead of postponing to MPI-3.1)? ** Yes - If yes to 3a or b, do you additionally want 3c? ** Yes - Do you want Option 4? ** No, should be done by the Fortran standardization as correction to TS29113.
Best regards Rolf
----- Original Message -----
From: "Bill Long" < [email protected] >
To: "Jeff Squyres (jsquyres)" < [email protected] >
Cc: "Craig Rasmussen" < [email protected] >, "Rolf Rabenseifner" < [email protected] >, "MPI-3 Fortran working
group" < [email protected] >, "Hubert Ritzdorf" < [email protected] >, "Martin Schulz"
< [email protected] >, "Tobias Burnus" < [email protected] >
Sent: Wednesday, March 27, 2013 12:03:27 AM
Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
On 3/26/13 11:26 AM, Jeff Squyres (jsquyres) wrote:
On Mar 26, 2013, at 12:16 PM, Bill Long < [email protected] > wrote:
I don't think there is a problem. To accommodate -i8 you have to
recompile the MPI module with the -i8 flag set. We've been shipping
two mpi modules as long as I can remember; the compiler scripts
pick the right one based on how the user code is compiled. Same as
how we select the right version of other libraries that also have
Fortran default type arguments. Within the MPI module source the
values of MPI_INTEGER and the similar data types for other Fortran
default types are set based on detection of the actual sizes of the
default types (which change with the compiler options). If you are
using the module compiled with -i8, MPI_INTEGER will encode the
information for an 8-byte integer.
But not all Fortran types have corresponding C types (from a
standards/strict perspective), right? (e.g., logical -- and I'm not
sure about the various complex types...?) So the MPI implementation
needs to know that it has to create some synthetic / "good enough"
MPI datatypes based on actual Fortran types.
...at least, that's what OMPI does...
What I did was less sophisticated (at least in terms of mpich
internals). The values for the various handles at page 666 (how
appropriate) lines 5-10, and 34, and page 667 lines 15-17 are not new.
I took the values out of the mpich mpif.h files generated for 32-bit
defaults and 64-bit defaults. Those values are assigned to internal
constants in the module. The definition of MPI_INTEGER selects between
the two options depending on KIND(0), and similarly for the others.
The
goal is to produce the same values in the module as in the mpif.h
files,
only automatically selected based on the default type size specified
by
the compiler options used to compile the module.
Cheers,
Bill
--
Bill Long [email protected]
Fortran Technical Support & voice: 651-605-9024
Bioinformatics Software Development fax: 651-605-9142
Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
On 3/29/13 10:51 AM, Rolf Rabenseifner wrote:
Here the shortcut:
- Only for optical: Removal of the BIND(C) from all mpi_f08 interface definitions. It has no real meaning in current MPI-3.0, because all about BIND(C) is defined in 17.1.5 and not in the 360 individual interface definitions.
Yes, I agree with this. This means fixes in A.3, for example. To the extent that the document is a specification of the API for users, the BIND(C) is irrelevant. The user CALL statement is unaffected by whether the implementation has BIND(C) or not. BIND(C) is really only relevant for implementers. In just one place (17.1.5 seems popular) there should be the rules for what is required of an implementation if it specifies BIND(C) on a particular routine, mainly the requirements for the external name, based on how the implementation uses the routine. Cheers, Bill -- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
Yes, exactly this. The BIND(C) in Annex A.3 had no real meaning. All rules had been in 17.1.5. I hope you can also agree with the rest of the plan. E.g., your wishes (and also Craig's, Tobias' ...) are included. Best regards Rolf ----- Original Message -----
From: "Bill Long" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "MPI-3 Fortran working group" <[email protected]>, "Craig Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]>, "Jeff Squyres (jsquyres)" <[email protected]> Sent: Friday, March 29, 2013 5:19:15 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind On 3/29/13 10:51 AM, Rolf Rabenseifner wrote:
Here the shortcut:
- Only for optical: Removal of the BIND(C) from all mpi_f08 interface definitions. It has no real meaning in current MPI-3.0, because all about BIND(C) is defined in 17.1.5 and not in the 360 individual interface definitions.
Yes, I agree with this. This means fixes in A.3, for example. To the extent that the document is a specification of the API for users, the BIND(C) is irrelevant. The user CALL statement is unaffected by whether the implementation has BIND(C) or not.
BIND(C) is really only relevant for implementers. In just one place (17.1.5 seems popular) there should be the rules for what is required of an implementation if it specifies BIND(C) on a particular routine, mainly the requirements for the external name, based on how the implementation uses the routine.
Cheers, Bill
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
I restrict myself to replying to Option 4, only, in this email. Rolf Rabenseifner wrote:
Option 4: (needed for 2c, for all others it is independent) - Substituting in mpi_f08 all CHARACTER(LEN=xxx) by CHARACTER(LEN=1),DIMENSION(xxx) with xxx = a fixed constant.
As far as I understand, this cannot be done for CHARACTER(LEN=*) because otherwise the MPI routine cannot learn about the length of the string, see, e.g., argv array in MPI_COMM_SPAWN.
For C interoperability, "character(len=*)" doesn't make sense as with Bind(C) just a pointer is passed without any length information (and the string is also not zero-terminated). With Fortran semantics, one either passes the length as hidden argument or does not pass the pointer to the data but some descriptor with the length information. In order to make "len=*" work with Bind(C), the information has to be passed in some way. However, using subroutine sub(str, len) Bind(C) integer :: len character, dimension(len) :: str would be one valid replacement - albeit slightly inconvenient. (Without BIND(C), character(len=len) is also valid but has no advantage to len=*.)
Pro: As far as I understand, this is needed for BIND(C) for the MPI routines that have CHARACTER(LEN=xxx) dummy arguments. Con: Non-consistent solution for a bug in TS29113 that should be fixed in TS29113. The mpi and the mpi_f08 moudlues will be inconsistent to each other. We do not want to change the old mpi module and mpif.h Fortran interface.
Note: For users, both interfaces act exactly in the same way and they shouldn't be able to detect the difference. (Unless their compiler doesn't support that part of Fortran 2003. But (nearly) all current compilers should be able to do so.)
With current CHARACTER(LEN=xxx), the MPI routine can check at runtime whether the provided actual argument is long enough. With CHARACTER(LEN=1),DIMENSION(xxx), such a check is impossible.
I fail to see why the diagnostic should be impossible. Compilers are doing array bounds checks since ages. If the explicit interface is known, compilers can simply insert a check before the function call. If the interface is not known, it gets more cumbersome but it is also possible (e.g. some compiler store in the caller the size of the actual arguments in a global variable and compare in the callee the values with the expected sizes). The only advantage of len=... is that Fortran semantic implies that the string length of the actual argument is available to the callee, which makes checking simpler. But of one passes a simple "char *" with BIND(C), this information is also not available. In addition, if the the array sizes are known at compile time and an explicit interface is used (or, as here, both are in the same file), a compiler can detect the mismatch even at compile time: Warning: Actual argument contains too few elements for dummy argument 'x' (3/5) subroutine foo(x) character :: x(5) end call foo("abc") end
Bill and Craig, please correct me if I'm wrong. Is this bug fixable in TS29113, or is there a reason, why CHARACTER(LEN=xxx) is not interoperable with C?
Well, TS29113 cannot be changed as it is not a bug (solvable via an interpretation request) and as the TS is published; it would be rather something for the next Fortran standard. At least if one does not allow assumed-length, there is no real fundamental reason to not allow other lengths. However, one runs into many inconsistencies and has to deal with a lot of special cases. For instance, while "character(len=1), value" and "char" are passed in the same way, "character(len=2), value" does not match any C type. Assuming, one could use "char[]". But then, passing len=1 by value ("char") and len=2 by reference ("char[]") doesn't look clean. A "character(len=1)" would be "char *" and "character(len=2)" then "char**". But what would be "character(len=n)" - what happens if one changes an "integer :: n = 1" to "integer, paramter :: n=1", would that change the ABI? I think there are too many issues like those to make it very unlikely to be changed in the next standard. In particular, as using character(len=1), dimension(xxx) :: str works sufficiently well and it is already implemented in most currently used compilers. Hence, I believe the standard will only be augmented if a very strong case can be made that allowing len=1 has large advantages. (I do not see this happening.) Tobias
Tobias made a nice summary of character issue. In particular, I'm of the opinion that the Fortran standards body does not think there is a bug in character string interoperability. That was fixed in TS29113 by allowing character strings to be passed by descriptor, depending on the interface description. However, the whole issue become moot if we drop the BIND(C) names from the MPI namespace. Craig Rasmussen CAS Scientific Programmer [email protected] On Mar 27, 2013, at 3:44 PM, Tobias Burnus wrote:
I restrict myself to replying to Option 4, only, in this email.
Rolf Rabenseifner wrote:
Option 4: (needed for 2c, for all others it is independent) - Substituting in mpi_f08 all CHARACTER(LEN=xxx) by CHARACTER(LEN=1),DIMENSION(xxx) with xxx = a fixed constant.
As far as I understand, this cannot be done for CHARACTER(LEN=*) because otherwise the MPI routine cannot learn about the length of the string, see, e.g., argv array in MPI_COMM_SPAWN.
For C interoperability, "character(len=*)" doesn't make sense as with Bind(C) just a pointer is passed without any length information (and the string is also not zero-terminated). With Fortran semantics, one either passes the length as hidden argument or does not pass the pointer to the data but some descriptor with the length information. In order to make "len=*" work with Bind(C), the information has to be passed in some way.
However, using subroutine sub(str, len) Bind(C) integer :: len character, dimension(len) :: str would be one valid replacement - albeit slightly inconvenient. (Without BIND(C), character(len=len) is also valid but has no advantage to len=*.)
Pro: As far as I understand, this is needed for BIND(C) for the MPI routines that have CHARACTER(LEN=xxx) dummy arguments. Con: Non-consistent solution for a bug in TS29113 that should be fixed in TS29113. The mpi and the mpi_f08 moudlues will be inconsistent to each other. We do not want to change the old mpi module and mpif.h Fortran interface.
Note: For users, both interfaces act exactly in the same way and they shouldn't be able to detect the difference. (Unless their compiler doesn't support that part of Fortran 2003. But (nearly) all current compilers should be able to do so.)
With current CHARACTER(LEN=xxx), the MPI routine can check at runtime whether the provided actual argument is long enough. With CHARACTER(LEN=1),DIMENSION(xxx), such a check is impossible.
I fail to see why the diagnostic should be impossible. Compilers are doing array bounds checks since ages. If the explicit interface is known, compilers can simply insert a check before the function call. If the interface is not known, it gets more cumbersome but it is also possible (e.g. some compiler store in the caller the size of the actual arguments in a global variable and compare in the callee the values with the expected sizes).
The only advantage of len=... is that Fortran semantic implies that the string length of the actual argument is available to the callee, which makes checking simpler. But of one passes a simple "char *" with BIND(C), this information is also not available.
In addition, if the the array sizes are known at compile time and an explicit interface is used (or, as here, both are in the same file), a compiler can detect the mismatch even at compile time: Warning: Actual argument contains too few elements for dummy argument 'x' (3/5)
subroutine foo(x) character :: x(5) end
call foo("abc") end
Bill and Craig, please correct me if I'm wrong. Is this bug fixable in TS29113, or is there a reason, why CHARACTER(LEN=xxx) is not interoperable with C?
Well, TS29113 cannot be changed as it is not a bug (solvable via an interpretation request) and as the TS is published; it would be rather something for the next Fortran standard.
At least if one does not allow assumed-length, there is no real fundamental reason to not allow other lengths.
However, one runs into many inconsistencies and has to deal with a lot of special cases. For instance, while "character(len=1), value" and "char" are passed in the same way, "character(len=2), value" does not match any C type. Assuming, one could use "char[]". But then, passing len=1 by value ("char") and len=2 by reference ("char[]") doesn't look clean. A "character(len=1)" would be "char *" and "character(len=2)" then "char**". But what would be "character(len=n)" - what happens if one changes an "integer :: n = 1" to "integer, paramter :: n=1", would that change the ABI?
I think there are too many issues like those to make it very unlikely to be changed in the next standard. In particular, as using character(len=1), dimension(xxx) :: str works sufficiently well and it is already implemented in most currently used compilers. Hence, I believe the standard will only be augmented if a very strong case can be made that allowing len=1 has large advantages. (I do not see this happening.)
Tobias
On 3/27/13 5:55 PM, Craig Rasmussen wrote:
Tobias made a nice summary of character issue. In particular, I'm of
I agree. The example of the argv argument to MPI_COMM_SPAWN is certainly broken. The C version is an array of pointers to character strings. Each of the strings could be different length. Without BIND(C), the Fortran version [character(*) argv(*)] is very different. In that case, every element of the argv array has the same length. Clearly this is not "interoperable" in any normal sense. With BIND(C), the argument becomes a pointer to a descriptor that contains the length element of argv(*) [still all elements have the same length], which is still not the same as *argv[], but an even bigger disruption that before. The corresponding C function needs to have the argument specified as a pointer to a descriptor, and not *argv[], for this to be interoperable. In practice, you can write the Fortran wrapper (without BIND(C)) that internally constructs an array of C pointers and rigs up the right values to be passed to C. BIND(C) definitely needs to be dropped for the Fortran version.
the opinion that the Fortran standards body does not think there is a bug in character string interoperability. That was fixed in TS29113 by allowing character strings to be passed by descriptor, depending on the interface description.
However, the whole issue become moot if we drop the BIND(C) names from the MPI namespace.
Right. Cheers, Bill
Craig Rasmussen CAS Scientific Programmer [email protected] <mailto:[email protected]>
On Mar 27, 2013, at 3:44 PM, Tobias Burnus wrote:
I restrict myself to replying to Option 4, only, in this email.
Rolf Rabenseifner wrote:
Option 4: (needed for 2c, for all others it is independent) - Substituting in mpi_f08 all CHARACTER(LEN=xxx) by CHARACTER(LEN=1),DIMENSION(xxx) with xxx = a fixed constant.
As far as I understand, this cannot be done for CHARACTER(LEN=*) because otherwise the MPI routine cannot learn about the length of the string, see, e.g., argv array in MPI_COMM_SPAWN.
For C interoperability, "character(len=*)" doesn't make sense as with Bind(C) just a pointer is passed without any length information (and the string is also not zero-terminated). With Fortran semantics, one either passes the length as hidden argument or does not pass the pointer to the data but some descriptor with the length information. In order to make "len=*" work with Bind(C), the information has to be passed in some way.
However, using subroutine sub(str, len) Bind(C) integer :: len character, dimension(len) :: str would be one valid replacement - albeit slightly inconvenient. (Without BIND(C), character(len=len) is also valid but has no advantage to len=*.)
Pro: As far as I understand, this is needed for BIND(C) for the MPI routines that have CHARACTER(LEN=xxx) dummy arguments. Con: Non-consistent solution for a bug in TS29113 that should be fixed in TS29113. The mpi and the mpi_f08 moudlues will be inconsistent to each other. We do not want to change the old mpi module and mpif.h Fortran interface.
Note: For users, both interfaces act exactly in the same way and they shouldn't be able to detect the difference. (Unless their compiler doesn't support that part of Fortran 2003. But (nearly) all current compilers should be able to do so.)
With current CHARACTER(LEN=xxx), the MPI routine can check at runtime whether the provided actual argument is long enough. With CHARACTER(LEN=1),DIMENSION(xxx), such a check is impossible.
I fail to see why the diagnostic should be impossible. Compilers are doing array bounds checks since ages. If the explicit interface is known, compilers can simply insert a check before the function call. If the interface is not known, it gets more cumbersome but it is also possible (e.g. some compiler store in the caller the size of the actual arguments in a global variable and compare in the callee the values with the expected sizes).
The only advantage of len=... is that Fortran semantic implies that the string length of the actual argument is available to the callee, which makes checking simpler. But of one passes a simple "char *" with BIND(C), this information is also not available.
In addition, if the the array sizes are known at compile time and an explicit interface is used (or, as here, both are in the same file), a compiler can detect the mismatch even at compile time: Warning: Actual argument contains too few elements for dummy argument 'x' (3/5)
subroutine foo(x) character :: x(5) end
call foo("abc") end
Bill and Craig, please correct me if I'm wrong. Is this bug fixable in TS29113, or is there a reason, why CHARACTER(LEN=xxx) is not interoperable with C?
Well, TS29113 cannot be changed as it is not a bug (solvable via an interpretation request) and as the TS is published; it would be rather something for the next Fortran standard.
At least if one does not allow assumed-length, there is no real fundamental reason to not allow other lengths.
However, one runs into many inconsistencies and has to deal with a lot of special cases. For instance, while "character(len=1), value" and "char" are passed in the same way, "character(len=2), value" does not match any C type. Assuming, one could use "char[]". But then, passing len=1 by value ("char") and len=2 by reference ("char[]") doesn't look clean. A "character(len=1)" would be "char *" and "character(len=2)" then "char**". But what would be "character(len=n)" - what happens if one changes an "integer :: n = 1" to "integer, paramter :: n=1", would that change the ABI?
I think there are too many issues like those to make it very unlikely to be changed in the next standard. In particular, as using character(len=1), dimension(xxx) :: str works sufficiently well and it is already implemented in most currently used compilers. Hence, I believe the standard will only be augmented if a very strong case can be made that allowing len=1 has large advantages. (I do not see this happening.)
Tobias
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/364#comment:7 shows all 32 routines with CHARACTER arguments. Can I summarize your analysis to: - 21 routines CHARACTER(LEN=*),INTENT(IN) arguments All these routines need the length information at the callee. The non-BIND(C) interface works fine (as in mpif.h and the mpi module) A BIND(C) interface would need to be changed, because the length information must be passed in an additional argument (or as in C by a special terminator character). - 2 routines with CHARACTER(LEN=*),INTENT(OUT) arguments These routines works with non-BIND(C) and BIND(C), because the application must provide a string-buffer that is large enough. - 0 routines with CHARACTER(LEN=SOME_CONST),INTENT(IN) - 7 routines with (only) CHARACTER(LEN=SOME_CONST),INTENT(OUT) These routines work with non-BIND(C). For BIND(C), we need to change the declaration into CHARACTER(LEN=1),DIMENSION(xxx) With non-BIND(C), this alternative declaration would also work correctly with any arguments passed in the same way as to CHARACTER(LEN=xxx). This interface change requires Fortran 2003 compilers. - and following special cases: MPI_Lookup_name MPI_Info_get MPI_Comm_spawn MPI_Comm_spawn_multiple A shortcut of this summary: Due to the 22 routines with LEN=*, BIND(C) on the MPI API level should be fully removed. BIND(C) makes sense only on this internal interface, which should be optional. Based on this, the possible options dramatically shrink. Best regards Rolf ----- Original Message -----
From: "Tobias Burnus" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "MPI-3 Fortran working group" <[email protected]>, [email protected], "Craig Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Jeff Squyres (jsquyres)" <[email protected]> Sent: Wednesday, March 27, 2013 11:44:56 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind I restrict myself to replying to Option 4, only, in this email.
Rolf Rabenseifner wrote:
Option 4: (needed for 2c, for all others it is independent) - Substituting in mpi_f08 all CHARACTER(LEN=xxx) by CHARACTER(LEN=1),DIMENSION(xxx) with xxx = a fixed constant.
As far as I understand, this cannot be done for CHARACTER(LEN=*) because otherwise the MPI routine cannot learn about the length of the string, see, e.g., argv array in MPI_COMM_SPAWN.
For C interoperability, "character(len=*)" doesn't make sense as with Bind(C) just a pointer is passed without any length information (and the string is also not zero-terminated). With Fortran semantics, one either passes the length as hidden argument or does not pass the pointer to the data but some descriptor with the length information. In order to make "len=*" work with Bind(C), the information has to be passed in some way.
However, using subroutine sub(str, len) Bind(C) integer :: len character, dimension(len) :: str would be one valid replacement - albeit slightly inconvenient. (Without BIND(C), character(len=len) is also valid but has no advantage to len=*.)
Pro: As far as I understand, this is needed for BIND(C) for the MPI routines that have CHARACTER(LEN=xxx) dummy arguments. Con: Non-consistent solution for a bug in TS29113 that should be fixed in TS29113. The mpi and the mpi_f08 moudlues will be inconsistent to each other. We do not want to change the old mpi module and mpif.h Fortran interface.
Note: For users, both interfaces act exactly in the same way and they shouldn't be able to detect the difference. (Unless their compiler doesn't support that part of Fortran 2003. But (nearly) all current compilers should be able to do so.)
With current CHARACTER(LEN=xxx), the MPI routine can check at runtime whether the provided actual argument is long enough. With CHARACTER(LEN=1),DIMENSION(xxx), such a check is impossible.
I fail to see why the diagnostic should be impossible. Compilers are doing array bounds checks since ages. If the explicit interface is known, compilers can simply insert a check before the function call. If the interface is not known, it gets more cumbersome but it is also possible (e.g. some compiler store in the caller the size of the actual arguments in a global variable and compare in the callee the values with the expected sizes).
The only advantage of len=... is that Fortran semantic implies that the string length of the actual argument is available to the callee, which makes checking simpler. But of one passes a simple "char *" with BIND(C), this information is also not available.
In addition, if the the array sizes are known at compile time and an explicit interface is used (or, as here, both are in the same file), a compiler can detect the mismatch even at compile time: Warning: Actual argument contains too few elements for dummy argument 'x' (3/5)
subroutine foo(x) character :: x(5) end
call foo("abc") end
Bill and Craig, please correct me if I'm wrong. Is this bug fixable in TS29113, or is there a reason, why CHARACTER(LEN=xxx) is not interoperable with C?
Well, TS29113 cannot be changed as it is not a bug (solvable via an interpretation request) and as the TS is published; it would be rather something for the next Fortran standard.
At least if one does not allow assumed-length, there is no real fundamental reason to not allow other lengths.
However, one runs into many inconsistencies and has to deal with a lot of special cases. For instance, while "character(len=1), value" and "char" are passed in the same way, "character(len=2), value" does not match any C type. Assuming, one could use "char[]". But then, passing len=1 by value ("char") and len=2 by reference ("char[]") doesn't look clean. A "character(len=1)" would be "char *" and "character(len=2)" then "char**". But what would be "character(len=n)" - what happens if one changes an "integer :: n = 1" to "integer, paramter :: n=1", would that change the ABI?
I think there are too many issues like those to make it very unlikely to be changed in the next standard. In particular, as using character(len=1), dimension(xxx) :: str works sufficiently well and it is already implemented in most currently used compilers. Hence, I believe the standard will only be augmented if a very strong case can be made that allowing len=1 has large advantages. (I do not see this happening.)
Tobias
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
Rolf Rabenseifner wrote:
Can I summarize your analysis to: - 2 routines with CHARACTER(LEN=*),INTENT(OUT) arguments These routines works with non-BIND(C) and BIND(C), because the application must provide a string-buffer that is large enough.
Except that LEN=* is not permitted with Bind(C), either. For Bind(C), using CHARACTER(len=1),dimension(*) would be fine, though. Tobias
Good. Then the very short summary should be okay: Due to the 22 routines with LEN=*, BIND(C) on the MPI API level should be fully removed. BIND(C) makes sense only on this internal interface, which should be optional. Best regards and happy easter Rolf ----- Original Message -----
From: "Tobias Burnus" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "MPI-3 Fortran working group" <[email protected]>, [email protected], "Craig Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Jeff Squyres (jsquyres)" <[email protected]> Sent: Thursday, March 28, 2013 8:15:45 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Rolf Rabenseifner wrote:
Can I summarize your analysis to: - 2 routines with CHARACTER(LEN=*),INTENT(OUT) arguments These routines works with non-BIND(C) and BIND(C), because the application must provide a string-buffer that is large enough.
Except that LEN=* is not permitted with Bind(C), either. For Bind(C), using CHARACTER(len=1),dimension(*) would be fine, though.
Tobias
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
Except that LEN=* is not permitted with Bind(C), either. For Bind(C), using CHARACTER(len=1),dimension(*) would be fine, though.
Tobias, If I understand TS29113 correctly, then your sentence - is valid for Fortran 2003 and 2008 - is not correct for TS29113: TS29113 allows CHARACTER(LEN=*). As you wrote on March 18:
TS 29113 allows in addtion len=* and len=: for dummy arguments of BIND(C) procedures. However, that doesn't help for LEN=MPI_MAX_OBJECT_NAME, either. (For len=*/len=:, TS 29113 requires that an array descriptor is used; cf. "8.7 Interoperability of procedures and procedure interfaces", p.31, (6)(b) and the paragraph below (6)(d).).
BIND(C) was never expected to be usable with MPI-3.0 before TS 29113 is available (in any of the three Fortran support methods). Best regards Rolf ----- Original Message -----
From: "Rolf Rabenseifner" <[email protected]> To: "Tobias Burnus" <[email protected]> Cc: "MPI-3 Fortran working group" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]> Sent: Thursday, March 28, 2013 8:19:38 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Good.
Then the very short summary should be okay:
Due to the 22 routines with LEN=*, BIND(C) on the MPI API level should be fully removed. BIND(C) makes sense only on this internal interface, which should be optional.
Best regards and happy easter Rolf
----- Original Message -----
From: "Tobias Burnus" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "MPI-3 Fortran working group" <[email protected]>, [email protected], "Craig Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Jeff Squyres (jsquyres)" <[email protected]> Sent: Thursday, March 28, 2013 8:15:45 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Rolf Rabenseifner wrote:
Can I summarize your analysis to: - 2 routines with CHARACTER(LEN=*),INTENT(OUT) arguments These routines works with non-BIND(C) and BIND(C), because the application must provide a string-buffer that is large enough.
Except that LEN=* is not permitted with Bind(C), either. For Bind(C), using CHARACTER(len=1),dimension(*) would be fine, though.
Tobias
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) _______________________________________________ mpi3-fortran mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
Am 29.03.2013 12:47, schrieb Rolf Rabenseifner: >>> Except that LEN=* is not permitted with Bind(C), either. For >>> Bind(C), using CHARACTER(len=1),dimension(*) would be fine, though. > If I understand TS29113 correctly, then your sentence > - is valid for Fortran 2003 and 2008 > - is not correct for TS29113: TS29113 allows CHARACTER(LEN=*). You are right, of course. I was too much thinking of passing string argument by pointer address while TS29113 allows character(len=:) and character(len=*), which is passed using the descriptor. Sorry for adding to the confusion and happy Easter to everyone. Tobias
On Mar 26, 2013, at 6:38 AM, Jeff Squyres (jsquyres) wrote:
On Mar 25, 2013, at 2:35 PM, Bill Long <[email protected]> wrote:
I'm strongly in favor of option 3. It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API.
Remember that Open MPI can never do this approach since our C handles != Fortran handles, and Fortran wrappers will not be able to reach back into the back-end C MPI objects (at least, not without a truckload of extra interop code -- the Fortran bindings layer would have to gain an understanding of the back-end MPI C objects, and that seems like a terrible idea).
I'm a hacker at heart and so I'm not as pessimistic as Jeff is. I'll have a conversation with Jeff see if I can convince him. This may be beyond the scope of an errata but I don't want to close off this possibility. Otherwise the MPI standard will still REQUIRE implementations to go OUTSIDE of the Fortran standard. This to me is a big deal and was one of the two big reasons in my opinion why we did MPI-3 for Fortran in the first place (the other was type safety). I really hate to go to all the work we did and then give up on an MPI standard that is out of compliance with language standards. Craig Rasmussen CAS Scientific Programmer [email protected]
Is this a fair summary?
There is only one small correction to your summary necessary:
1. Absolute minimal change, remove BIND(C) only for routines with LOGICAL arguments and all predefined callback routines (because some of them have LOGICAL arguments, e.g., MPI_COMM_DUP_FN, and all predefined callbacks should be handled in the same way) and all ABSTRACT INTERFACE definitions for user written callbacks, (because they must fit to the predefined callbacks, and again the whole routine group should be handles in the same way), and all MPI routines that have such callback PROCEDURE dummy arguments (because ABSTRACT INTERFACEs without BIND(C) are also not interoperable). Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL and PROCEDURE arguments, all predefined callbacks.
This makes the routine list in https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/364#comment:5 plus all ABSTRACT INTERFACEs on MPI-3.0 pages 678-680. Besides this correction, your summary is correct. Best regards Rolf ----- Original Message -----
From: "Craig Rasmussen" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "MPI-3 Fortran working group" <[email protected]>, [email protected], "Jeff Squyres (jsquyres)" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Monday, March 25, 2013 6:50:46 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind My summary of the three options:
1. Absolute minimal change, remove BIND(C) only for routines with LOGICAL arguments. Tools will need to support underscore name mangling forever (even when TS29113 is implemented by compilers) but only on routines with LOGICAL arguments.
2. Remove BIND(C) from all routines. Tools will need to support underscore name mangling for all routines.
3. Remove BIND(C) from all routines. Plus allow Fortran implementations to substitute the C PMPI interface standard.
Is this a fair summary?
My vote:
I'm strongly in favor of option 3. It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API. There are several advantages with this implementation: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 4. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well.
Craig Rasmussen CAS Scientific Programmer [email protected]
On Mar 25, 2013, at 10:11 AM, Rolf Rabenseifner wrote:
We (Jeff, Bill L., Craig, Hubert, Martin, Tobias, and me (Rolf)) had a telecon today and the outcome are following options:
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem: Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2 - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.)
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: I'm not sure whether there is a real pro?
Bill, can you add the pros, it is your option.
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
My favorite in the moment, i.e. at the starting of the pro-con-discussion: Option 2.
Best regards Rolf
----- Original Message -----
From: "Rolf Rabenseifner" < [email protected] >
Cc: "Jeff Squyres (jsquyres)" < [email protected] >, "Craig E Rasmussen" < [email protected] >, "Hubert Ritzdorf"
< [email protected] >, "Martin Schulz" < [email protected]
, "Tobias Burnus" < [email protected] >
Sent: Monday, March 25, 2013 12:21:39 PM
Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
Yes, this freedom (1)-(4) in Bill's email is important.
In our discussion, we forgot one additional reason why we defined
it as it is written in MPI-3.0 Section 17.1.5, page 605:
- It should be possible to implement the interfaces with
these non-standard Fortran-void-* directives,
and later on,
to substitute this implementation by new ones
that are based on TS 29113.
- When doing this substitution, it should be save for
applications when the application is not re-compiled.
- More than that, it should be possible to combine older
.o files with newly TS 29113 based .o files.
- For this, we decided, that the implementation has to switch
from non-BIND(C) to BIND(C) at least in the MPI_ISEND
routine group when switching to MPI_SUBARRAYS_SUPPORTED==.TRUE.
- The real trick is that the linker name must be changed.
- This all is valid for both, the new mpi_f08 module,
and the old mpi module.
A second issue in our discussion is always, which changes
we want to do to the mpi_f08 MPI-3.0 user interface.
I want to remember that this user interface is defined
according following rules.
If a user modifies his source to use everywhere mpi_f08,
he/she has to modify:
- He/she has to substitute the "include 'mpif.h'" or "use mpi"
statements by "use mpi_f08".
- He/she has to modify all handle and status declarations.
- In the executable statements, he/she has to substitute all
status(MPI_TAG), status(MPI_SOURCE), status(MPI_ERROR)
accesses into
status%MPI_TAG, status%MPI_SOURCE, status%MPI_ERROR.
- Callback routines need more effort due to the
abstract interface definitions.
This is touched by ticket #364, i.e.,
the MPI-3.0 BIND(C) of callbacks had to be removed.
- Changes in the interfaces are only
-- all handles use the new handle types
and in
-- MPI_User_function,
-- MPI_Datarep_conversion_function,
the buffers are done with TYPE(C_TR), VALUE.
This should be all.
All other user correct code should directly compile with the new
mpi_f08 module.
We should not change these things, except the necessary
errata in #364.
I see that we have some freedom only in the area,
what we do internally (together with linker names) and with PMPI,
but not in the user interface.
If we add additional user interfaces (e.g. overloading the LOGICAL
with INTEGER arguments)
we would only make it more complicate.
Best regards
Rolf
----- Original Message -----
From: "Bill Long" < [email protected] >
To: "Rolf Rabenseifner" < [email protected] >
Cc: "Jeff Squyres (jsquyres)" < [email protected] >, "Craig E
Rasmussen" < [email protected] >, "Hubert Ritzdorf"
< [email protected] >, "Martin Schulz" < [email protected]
,
"Tobias Burnus" < [email protected] >
Sent: Sunday, March 24, 2013 5:59:38 PM
Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008
interface - issue with the LOGICAL kind
This time is OK for me. I'll look for the webex email.
I would like to note that the various options discussed are not
necessarily "all or nothing" propositions. An implementation could
use
the thin wrappers in a module approach to eliminate the bad Fortran
types problems for most of the MPI routines, but also have some
routines
implemented in a different way, such as (1) all in Fortran that is
callable from C, (2) all in C that is called through an interface
directly from Fortran, (3) as a fat wrapper in Fortran, or (4) a
completely replicated routine for the Fortran implementation. There
is
no reason that all of the routines have to be the same method - in
fact
probably good arguments against that in some cases. We just need to
have
the rules such that the tools have some known name to intercept.
Along
the lines of "If implementation method X is chosen for routine
MPI_xxx,
then these are the linker name requirements:.....".
Cheers,
Bill
On 3/23/13 2:11 AM, Rolf Rabenseifner wrote:
Dear Jeff, Craig, Bill, Hubert, Martin, and Tobias,
I expect that this is the total group that discusses the
Fortran LOGICAL etc. problem.
We learnt that the solutions also touch the PMPI concept.
There was a telecon between Craig, Jeff and Bill with the proposal
at the end of this email.
I like to understand this proposal and therefore I asked Jeff
for explanations. I would like to open this for the whole group.
I have meetings at 9am and 1pm is eastern on Monday. I can talk
any
other time.
Craig prob can't attend because he's 3 hours earlier, but that's
prob ok.
Monday March 25,
- 4 PM Central European Time CET
= 11 AM Eastern Daylight Time EST/EDT
= 8 AM Pacific Time
I hope these relations are correct. Would this work for you?
At least for Jeff and me, because I want to understand
what they discussed with their proposal.
Jeff, can you set up a webex?
Best regards
Rolf
----- Original Message -----
From: "Jeff Squyres (jsquyres)" < [email protected] >
To: "Rolf Rabenseifner" < [email protected] >
Cc: "Craig E Rasmussen" < [email protected] >
Sent: Friday, March 22, 2013 8:01:23 PM
Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008
interface - issue with the LOGICAL kind
Btw, I'm not sure how this is not the least surprise.
We can't change the params to not be logical, yes we must still
allow
for calling to c. So the overloaded solution is the only one.
And the new Integer param signature must have a new bind name so
as
to
did ambiguity it from the one we published in MPI3. See my
previous
mail about this.
I have meetings at 9am and 1pm is eastern on Monday. I can talk
any
other time.
Craig prob can't attend because he's 3 hours earlier, but that's
prob
ok.
Sent from my phone. No type good.
On Mar 22, 2013, at 8:51 AM, "Rolf Rabenseifner"
< [email protected] > wrote:
Jeff,
can I call you by phone?
I would like to talk to you as the binding responsible and MPI
chair.
2. Replace the current binding names for MPI functions with
LOGICAL
dummy arguments with a different name and replacing LOGICAL
type
with
default INTEGER arguments. The name must be changed so as to
not
...
This is a hard step and far away from "last surprise".
I would like to learn how it came out as a result of your
telecon,
before I try to go the next step (e.g. email answer).
Best regards
Rolf
----- Original Message -----
From: "Craig Rasmussen" < [email protected] >
To: "MPI-3 Fortran working group"
Cc: "Hubert Ritzdorf" < [email protected] >, "Martin
Schulz" < [email protected] >
Sent: Friday, March 22, 2013 4:29:36 PM
Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran
2008
interface - issue with the LOGICAL kind
Bill Long, Jeff Squyres, and I had a phone conversation
yesterday
and came up with a straw man proposal. We believe it fixes the
LOGICAL argument problem while satisfying three major goals:
1. Backwards compatibility.
2. Flexibility so that efficient implementations can be
produced
on
a variety of hardware.
3. Not dramatically increase the complexity for the tools
community.
4. Ensure that the changes won't be seen by ordinary Fortran
MPI
users.
The straw man proposal is:
1. Remove the current binding names for MPI functions with
LOGICAL
dummy arguments (. This must be done because these interfaces
break
the Fortran standard.
2. Replace the current binding names for MPI functions with
LOGICAL
dummy arguments with a different name and replacing LOGICAL
type
with
default INTEGER arguments. The name must be changed so as to
not
conflict with the current name with LOGICAL arguments. For
example,
the binding name "MPI_Test_f08" will be replaced by
"MPI_Test_int_f08"
(exact replacement name open for discussion). The Fortran
implementation of the function MPI_Test must call
MPI_Test_int_f08
as soon as possible for the tools implementers.
3. Implementation of the Fortran PMPI calling convention will
be
made optional (we think it already it). If an MPI implementer
doesn't
provide the MPI_F08 binding names, then it must call the C MPI
implementation in a thin Fortran wrapper so that the tools
implementers can intercept the call in C. An MPI implementation
shall not provide/call both the MPI_F08 binding name and the C
name.
Option 2 allows some MPI vendors to implement MPI functions
entirely within Fortran (using the MPI_f08 binding names),
if this is more efficient.
Option 3 allows some MPI implementations to place Fortran
wrappers
inside modules where they may be inlined by the compiler for
efficiency.
Craig Rasmussen
CAS Scientific Programmer
--
Bill Long [email protected]
Fortran Technical Support & voice: 651-605-9024
Bioinformatics Software Development fax: 651-605-9142
Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
--
Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected]
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
On 3/25/13 12:11 PM, Rolf Rabenseifner wrote:
We (Jeff, Bill L., Craig, Hubert, Martin, Tobias, and me (Rolf)) had a telecon today and the outcome are following options:
Option 1 - #364 as it currently is, as a minimal solution for the LOGICAL problem: Pro: It is consistent between Fortran and MPI. I.e., it is the minimal solution that we must do. Con: It has a mix of BIND(C) and non-BIND(C) routine groups which is ugly for the future. Remark: non-BIND(C) will stay there because the callbacks need non-BIND(C) in the moment and this cannot be changed in the future by nice new concepts from the Fortran language standardization.
Option 2 - #364 plus changing MPI-3.0 page 605 lines 24-30 and page 605 line 47 - page 606 line 6 by also using non-BIND(C), but other linker names: MPI_Send_f08d (in mpi_f08 with TS29113 quality, i.e., buffers are done with C-descriptors and not with void* pointers.) MPI_Send_fd (in the mpi module or mpif.h if MPI_SUBARRAYS_SUPPORTED=.TRUE. is also done there, i.e., with the descriptors.)
Pro: It is consistent between Fortran and MPI and it is consistent within MPI, i.e., all functionality is done with non-BIND(C) routines. BIND(C) can be still used internally to call MPI_Send_cdesc and MPI_Isend_cdesc. To be fair, implementatiosn will be based mainly on this TS 29113 feature, but only internally. Con: No BIND(C) on the official MPI interface. We urged Fortran to put some features into TS 29113, but we will not use these features, because they are not needed on this internal interface.
Option 3: - Option 2 plus the following rule: - The Fortran PMPI interface can be substituted by a wrapper based sulution that internally calls the full MPI C interface. - In this case, the Fortran interface should be hidden to the tools. - For the buffer routines with the new cdesc descriptor, new interfaces are added for the tools people, e.g., MPI_Send_cdesc and MPI_Isend_cdesc.
Pro: I'm not sure whether there is a real pro?
Bill, can you add the pros, it is your option.
I believe that Craig and I made comments on this in separate emails, but I can reproduce here: It is the only option that allows the possibility of underscore name mangling disappearing in the future. I think most (if not all) MPI Fortran implementations will eventually evolve to thin wrappers calling the C API. There are several advantages with this implementation: 1. The thin wrappers can be put in a module. 2. Compilers may be able to inline the wrappers for very efficient implementations. 3. Fortran implementations are much simpler because they won't have to directly support PMPI in Fortran. 4. Since the Fortran implementation is substantially less complex, the costs associated with testing, supporting, and maintaining a Fortran implementation will also be reduced. 5. If all MPI implementations eventually choose to call the C API, then the tools community eventually will not have to support underscore name mangling, thus reducing their costs as well. and 6. You can isolate the issues of Fortran types (compiled with -i8, meaning of LOGICAL, character arguments with len > 1) into the Fortran wrappers so that the C library writers do not have to know about these issues. Simplifies the maintenance and testing issues for the C library code. I would modify option 3 slightly. The implementer might want to write some routine ONLY in Fortran. The routines that convert the status formats between the Fortran and the C forms comes to mind. In that case, BIND(C) would be required, and it would specify the C binding name as the external linker name. This would be needed so that C codes could call the routine. and, due to the fact that the mpif.h method has to be supported as well, For that mode, the corresponding text from the mpi module (with INTEGER handles and ( void *) buffers) is extracted to a file without the module wrapping. Compiling that file will result in a routine with an external symbol of mpi_xxx_ (or something similar, depending on the conventions of the compiler). That version would be called only in the case that the user included mpif.h. A particularly aggressive compiler that defers inlining until the link step might even inline this case. If the mpif.h mode is deleted in the future, this file would just no longer be generated. Cheers, Bill
Con: An additional new interface for the tools folk. It is not backward compatible for users who really intercept the Fortran MPI interface.
My favorite in the moment, i.e. at the starting of the pro-con-discussion: Option 2.
Best regards Rolf
----- Original Message -----
From: "Rolf Rabenseifner" <[email protected]> To: [email protected] Cc: "Jeff Squyres (jsquyres)" <[email protected]>, "Craig E Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Monday, March 25, 2013 12:21:39 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Yes, this freedom (1)-(4) in Bill's email is important.
In our discussion, we forgot one additional reason why we defined it as it is written in MPI-3.0 Section 17.1.5, page 605:
- It should be possible to implement the interfaces with these non-standard Fortran-void-* directives, and later on, to substitute this implementation by new ones that are based on TS 29113. - When doing this substitution, it should be save for applications when the application is not re-compiled. - More than that, it should be possible to combine older .o files with newly TS 29113 based .o files. - For this, we decided, that the implementation has to switch from non-BIND(C) to BIND(C) at least in the MPI_ISEND routine group when switching to MPI_SUBARRAYS_SUPPORTED==.TRUE. - The real trick is that the linker name must be changed. - This all is valid for both, the new mpi_f08 module, and the old mpi module.
A second issue in our discussion is always, which changes we want to do to the mpi_f08 MPI-3.0 user interface. I want to remember that this user interface is defined according following rules. If a user modifies his source to use everywhere mpi_f08, he/she has to modify: - He/she has to substitute the "include 'mpif.h'" or "use mpi" statements by "use mpi_f08". - He/she has to modify all handle and status declarations. - In the executable statements, he/she has to substitute all status(MPI_TAG), status(MPI_SOURCE), status(MPI_ERROR) accesses into status%MPI_TAG, status%MPI_SOURCE, status%MPI_ERROR. - Callback routines need more effort due to the abstract interface definitions. This is touched by ticket #364, i.e., the MPI-3.0 BIND(C) of callbacks had to be removed. - Changes in the interfaces are only -- all handles use the new handle types and in -- MPI_User_function, -- MPI_Datarep_conversion_function, the buffers are done with TYPE(C_TR), VALUE.
This should be all. All other user correct code should directly compile with the new mpi_f08 module. We should not change these things, except the necessary errata in #364.
I see that we have some freedom only in the area, what we do internally (together with linker names) and with PMPI, but not in the user interface. If we add additional user interfaces (e.g. overloading the LOGICAL with INTEGER arguments) we would only make it more complicate.
Best regards Rolf
----- Original Message -----
From: "Bill Long" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "Jeff Squyres (jsquyres)" <[email protected]>, "Craig E Rasmussen" <[email protected]>, "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]>, "Tobias Burnus" <[email protected]> Sent: Sunday, March 24, 2013 5:59:38 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind This time is OK for me. I'll look for the webex email.
I would like to note that the various options discussed are not necessarily "all or nothing" propositions. An implementation could use the thin wrappers in a module approach to eliminate the bad Fortran types problems for most of the MPI routines, but also have some routines implemented in a different way, such as (1) all in Fortran that is callable from C, (2) all in C that is called through an interface directly from Fortran, (3) as a fat wrapper in Fortran, or (4) a completely replicated routine for the Fortran implementation. There is no reason that all of the routines have to be the same method - in fact probably good arguments against that in some cases. We just need to have the rules such that the tools have some known name to intercept. Along the lines of "If implementation method X is chosen for routine MPI_xxx, then these are the linker name requirements:.....".
Cheers, Bill
On 3/23/13 2:11 AM, Rolf Rabenseifner wrote:
Dear Jeff, Craig, Bill, Hubert, Martin, and Tobias,
I expect that this is the total group that discusses the Fortran LOGICAL etc. problem. We learnt that the solutions also touch the PMPI concept. There was a telecon between Craig, Jeff and Bill with the proposal at the end of this email. I like to understand this proposal and therefore I asked Jeff for explanations. I would like to open this for the whole group.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time. Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Monday March 25, - 4 PM Central European Time CET = 11 AM Eastern Daylight Time EST/EDT = 8 AM Pacific Time
I hope these relations are correct. Would this work for you?
At least for Jeff and me, because I want to understand what they discussed with their proposal.
Jeff, can you set up a webex?
Best regards Rolf
----- Original Message -----
From: "Jeff Squyres (jsquyres)" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "Craig E Rasmussen" <[email protected]> Sent: Friday, March 22, 2013 8:01:23 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind Btw, I'm not sure how this is not the least surprise.
We can't change the params to not be logical, yes we must still allow for calling to c. So the overloaded solution is the only one.
And the new Integer param signature must have a new bind name so as to did ambiguity it from the one we published in MPI3. See my previous mail about this.
I have meetings at 9am and 1pm is eastern on Monday. I can talk any other time.
Craig prob can't attend because he's 3 hours earlier, but that's prob ok.
Sent from my phone. No type good.
On Mar 22, 2013, at 8:51 AM, "Rolf Rabenseifner" <[email protected]> wrote:
Jeff,
can I call you by phone? I would like to talk to you as the binding responsible and MPI chair.
2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not ... This is a hard step and far away from "last surprise".
I would like to learn how it came out as a result of your telecon, before I try to go the next step (e.g. email answer).
Best regards Rolf
----- Original Message -----
From: "Craig Rasmussen" <[email protected]> To: "MPI-3 Fortran working group" <[email protected]> Cc: "Hubert Ritzdorf" <[email protected]>, "Martin Schulz" <[email protected]> Sent: Friday, March 22, 2013 4:29:36 PM Subject: Re: [MPI3 Fortran] [Mpi-comments] MPI 3.0: Fortran 2008 interface - issue with the LOGICAL kind
Bill Long, Jeff Squyres, and I had a phone conversation yesterday and came up with a straw man proposal. We believe it fixes the LOGICAL argument problem while satisfying three major goals:
1. Backwards compatibility. 2. Flexibility so that efficient implementations can be produced on a variety of hardware. 3. Not dramatically increase the complexity for the tools community. 4. Ensure that the changes won't be seen by ordinary Fortran MPI users.
The straw man proposal is:
1. Remove the current binding names for MPI functions with LOGICAL dummy arguments (. This must be done because these interfaces break the Fortran standard. 2. Replace the current binding names for MPI functions with LOGICAL dummy arguments with a different name and replacing LOGICAL type with default INTEGER arguments. The name must be changed so as to not conflict with the current name with LOGICAL arguments. For example, the binding name "MPI_Test_f08" will be replaced by "MPI_Test_int_f08" (exact replacement name open for discussion). The Fortran implementation of the function MPI_Test must call MPI_Test_int_f08 as soon as possible for the tools implementers. 3. Implementation of the Fortran PMPI calling convention will be made optional (we think it already it). If an MPI implementer doesn't provide the MPI_F08 binding names, then it must call the C MPI implementation in a thin Fortran wrapper so that the tools implementers can intercept the call in C. An MPI implementation shall not provide/call both the MPI_F08 binding name and the C name.
Option 2 allows some MPI vendors to implement MPI functions entirely within Fortran (using the MPI_f08 binding names), if this is more efficient.
Option 3 allows some MPI implementations to place Fortran wrappers inside modules where they may be inlined by the compiler for efficiency.
Craig Rasmussen CAS Scientific Programmer [email protected]
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
-- Bill Long [email protected] Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
participants (7)
-
Bill Long -
Craig Rasmussen -
Hubert Ritzdorf -
Jeff Squyres (jsquyres) -
Rolf Rabenseifner -
Schulz, Martin -
Tobias Burnus