Hi all, Sorry about the delay in sending out the BG/P patches. I've placed them here: http://www.mcs.anl.gov/~balaji/projects/mpich2/bgp/r2. I've assigned a few patches to each person below. Can the below folks please confirm that you can go over them and make sure there are no problems integrating them into the trunk? The plan is to go over these and have a telecon around mid next week. The IBM folks didn't do a very good job in breaking up the diffs, so some of the patches are large; you'll probably need to break them up and try different parts separately. build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev test-build.patch -- Dave test-source.patch -- Dave A few things I noticed on a quick glance through the patches: 1. MPI_Aint vs. (void *). I remember that there was some discussion, but what was the conclusion? Can MPI_Aint be larger than (void *)? A large chunk of the patches assumes that these two can be different, so they introduce MPI_AINT_TO_VOID_PTR and friends to deal with such cases. 2. Some of the patches (e.g., debugger.patch, test-build.patch and test-source.patch) have a number of BG specific additions. These need to be pruned out in a way that IBM can still use it, but they will continue to be generic enough for other platforms. Thanks. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
I've dealt with (but not checked in) most of the build patches. Most of them I had to change to make them non-BG specific and/or to be consistent with the rules. The MPI_Aint vs void * is a hack to deal with a combination of limitations in the MPI spec and in the MPICH2 implementation. Specifically, the real issue is to make MPI_Aint the same as MPI_Offset, though that's not the way it looks in their patch. Some things that they did, particularly the MPI_Pint and MPI_Puint, need to be changed to meet our naming conventions and avoid potential conflicts with MPI 3. Bill On May 2, 2008, at 3:11 AM, Pavan Balaji wrote:
Hi all,
Sorry about the delay in sending out the BG/P patches. I've placed them here: http://www.mcs.anl.gov/~balaji/projects/mpich2/bgp/r2. I've assigned a few patches to each person below. Can the below folks please confirm that you can go over them and make sure there are no problems integrating them into the trunk? The plan is to go over these and have a telecon around mid next week.
The IBM folks didn't do a very good job in breaking up the diffs, so some of the patches are large; you'll probably need to break them up and try different parts separately.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev test-build.patch -- Dave test-source.patch -- Dave
A few things I noticed on a quick glance through the patches:
1. MPI_Aint vs. (void *). I remember that there was some discussion, but what was the conclusion? Can MPI_Aint be larger than (void *)? A large chunk of the patches assumes that these two can be different, so they introduce MPI_AINT_TO_VOID_PTR and friends to deal with such cases.
2. Some of the patches (e.g., debugger.patch, test-build.patch and test-source.patch) have a number of BG specific additions. These need to be pruned out in a way that IBM can still use it, but they will continue to be generic enough for other platforms.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
The MPI_Aint vs void * is a hack to deal with a combination of limitations in the MPI spec and in the MPICH2 implementation. Specifically, the real issue is to make MPI_Aint the same as MPI_Offset, though that's not the way it looks in their patch. Some
I tried to look through my older email, but couldn't find this discussion. Isn't the right way to use MPI_Offset where needed instead of forcing MPI_Aint and MPI_Offset to be of the same size? The code assumes in many places that (void *) and MPI_Aint are of the same size. If we change this, many things might break. We should probably decide on this first, as a major part of the patches is based on this change. Thanks. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
Maybe you haven't been in on the conversation related to this? The issue is that MPI_Aint is the size for extents, even in I/I space, and so we need it to be 64 bit to be able to do useful file I/I (think subarray access from many processes). -- Rob On May 2, 2008, at 7:41 PM, Pavan Balaji <[email protected]> wrote:
The MPI_Aint vs void * is a hack to deal with a combination of limitations in the MPI spec and in the MPICH2 implementation. Specifically, the real issue is to make MPI_Aint the same as MPI_Offset, though that's not the way it looks in their patch. Some
I tried to look through my older email, but couldn't find this discussion. Isn't the right way to use MPI_Offset where needed instead of forcing MPI_Aint and MPI_Offset to be of the same size? The code assumes in many places that (void *) and MPI_Aint are of the same size. If we change this, many things might break.
We should probably decide on this first, as a major part of the patches is based on this change.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
Maybe you haven't been in on the conversation related to this? The issue is that MPI_Aint is the size for extents, even in I/I space, and so we need it to be 64 bit to be able to do useful file I/I (think subarray access from many processes).
Ah, got it. Thanks. I saw this discussion earlier, but couldn't pin-point the reason. So, we do want to accept the MPI_AINT_TO_VOID_PTR and family related parts of the patches, correct? We'll probably also need to go through the entire code to make sure there are no random typecasts from MPI_Aint to (void *); the other way should be fine. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
Hey, Sorry for the "I/I" instead of "I/O"...iPhone is trying to be helpful. At a conceptual level these patches are something that I think that we want to integrate, but your take on the specifics obviously trumps this conceptual need in the short term :). Thanks! Rob On May 3, 2008, at 12:33 AM, Pavan Balaji wrote:
Maybe you haven't been in on the conversation related to this? The issue is that MPI_Aint is the size for extents, even in I/I space, and so we need it to be 64 bit to be able to do useful file I/I (think subarray access from many processes).
Ah, got it. Thanks. I saw this discussion earlier, but couldn't pin- point the reason. So, we do want to accept the MPI_AINT_TO_VOID_PTR and family related parts of the patches, correct?
We'll probably also need to go through the entire code to make sure there are no random typecasts from MPI_Aint to (void *); the other way should be fine.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
(For some reason, my copy of Pavan's message is blank, so I'm replying to this one). There are a number of things about this approach that make me uneasy. In the short term, its probably ok (modulo some of the naming issues, particularly avoiding reserved MPI names and using MPIR_Pint instead of MPI_Pint, etc.), but in the long run (MPI 3?) we need to do something else. An example is the changes to the attribute routines - that's the sort of unintended side effect that I expect to cause problems in some peoples codes. My feeling is this: We need to make sure that the macros/types meet the naming rules and do not appear in mpi.h , where users might see them (they can be in mpiimpl.h or in a new file that configure creates that mpiimpl.h includes). We should think about whether we really want to specify the size of MPI_Aint or simply say that MPI_Aint and MPI_Offset are the same size, and the size is the max of what is needed (address, file offset). Bill On May 3, 2008, at 10:28 AM, Rob Ross wrote:
Hey,
Sorry for the "I/I" instead of "I/O"...iPhone is trying to be helpful.
At a conceptual level these patches are something that I think that we want to integrate, but your take on the specifics obviously trumps this conceptual need in the short term :).
Thanks!
Rob
On May 3, 2008, at 12:33 AM, Pavan Balaji wrote:
Maybe you haven't been in on the conversation related to this? The issue is that MPI_Aint is the size for extents, even in I/I space, and so we need it to be 64 bit to be able to do useful file I/I (think subarray access from many processes).
Ah, got it. Thanks. I saw this discussion earlier, but couldn't pin- point the reason. So, we do want to accept the MPI_AINT_TO_VOID_PTR and family related parts of the patches, correct?
We'll probably also need to go through the entire code to make sure there are no random typecasts from MPI_Aint to (void *); the other way should be fine.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
Hi, I like the idea of making MPI_Aint and MPI_Offset the same size all the time. Rob On May 4, 2008, at 6:58 PM, William Gropp wrote:
(For some reason, my copy of Pavan's message is blank, so I'm replying to this one).
There are a number of things about this approach that make me uneasy. In the short term, its probably ok (modulo some of the naming issues, particularly avoiding reserved MPI names and using MPIR_Pint instead of MPI_Pint, etc.), but in the long run (MPI 3?) we need to do something else. An example is the changes to the attribute routines - that's the sort of unintended side effect that I expect to cause problems in some peoples codes.
My feeling is this:
We need to make sure that the macros/types meet the naming rules and do not appear in mpi.h , where users might see them (they can be in mpiimpl.h or in a new file that configure creates that mpiimpl.h includes).
We should think about whether we really want to specify the size of MPI_Aint or simply say that MPI_Aint and MPI_Offset are the same size, and the size is the max of what is needed (address, file offset).
Bill
On May 3, 2008, at 10:28 AM, Rob Ross wrote:
Hey,
Sorry for the "I/I" instead of "I/O"...iPhone is trying to be helpful.
At a conceptual level these patches are something that I think that we want to integrate, but your take on the specifics obviously trumps this conceptual need in the short term :).
Thanks!
Rob
On May 3, 2008, at 12:33 AM, Pavan Balaji wrote:
Maybe you haven't been in on the conversation related to this? The issue is that MPI_Aint is the size for extents, even in I/I space, and so we need it to be 64 bit to be able to do useful file I/I (think subarray access from many processes).
Ah, got it. Thanks. I saw this discussion earlier, but couldn't pin- point the reason. So, we do want to accept the MPI_AINT_TO_VOID_PTR and family related parts of the patches, correct?
We'll probably also need to go through the entire code to make sure there are no random typecasts from MPI_Aint to (void *); the other way should be fine.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
We should think about whether we really want to specify the size of MPI_Aint or simply say that MPI_Aint and MPI_Offset are the same size, and the size is the max of what is needed (address, file offset).
Do we want to give an option where they are not equal? That would be incorrect, right? If you meant keeping MPI_Aint and MPI_Offset the same size *all* the time, I agree. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
They can (and possibly should) be different if the processors address space is 32 bits but the file system supports > 2GB files and has 64 bit offsets. This is the case for BG. I can't think of a realistic case where the size of MPI_Offset would be less than the sizeof MPI_Aint, however. Bill On May 4, 2008, at 9:48 PM, Pavan Balaji wrote:
We should think about whether we really want to specify the size of MPI_Aint or simply say that MPI_Aint and MPI_Offset are the same size, and the size is the max of what is needed (address, file offset).
Do we want to give an option where they are not equal? That would be incorrect, right?
If you meant keeping MPI_Aint and MPI_Offset the same size *all* the time, I agree.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
As per Rob's email, it looks like MPI_Aint is used for extents even for I/O. So, even if the processor address space is 32-bit, if the file-system supports > 2GB files, won't we still need to use a 64-bit MPI_Aint? I agree that having an MPI_Offset smaller than MPI_Aint doesn't make sense. -- Pavan On 05/05/2008 06:30 AM, William Gropp wrote:
They can (and possibly should) be different if the processors address space is 32 bits but the file system supports > 2GB files and has 64 bit offsets. This is the case for BG. I can't think of a realistic case where the size of MPI_Offset would be less than the sizeof MPI_Aint, however.
Bill
On May 4, 2008, at 9:48 PM, Pavan Balaji wrote:
We should think about whether we really want to specify the size of MPI_Aint or simply say that MPI_Aint and MPI_Offset are the same size, and the size is the max of what is needed (address, file offset).
Do we want to give an option where they are not equal? That would be incorrect, right?
If you meant keeping MPI_Aint and MPI_Offset the same size *all* the time, I agree.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
This is exactly the problem. MPI_Aint is defined by the MPI standard as an "address-sized int". Users will expect to be able to use a pointer to hold the contents of an address-sized int, particularly when using the attribute routines, and this will break in the BG fixes. Internally, MPICH uses MPI_Aint for datatype extents, in part because the MPI definition of the datatype routines does so. However, because the total data transferred is determined by (count,datatype), you can describe more than an MPI_Aint's worth of data even without having a single datatype with an extent greater than an MPI_Aint. One way out of this would be to add MPI datatype routines for extent that returned an MPI_Offset instead of an MPI_Aint; that would be a new routine (MPI 2.2 anyone?). Another would be to define an MPI_Size_t that was the largest of MPI_Aint, MPI_Offset, sizeof(void*), sizeof(void *()), .... and change all routines to use that single type (MPI 3?). Internally, MPICH2 could use that type now, performing the necessary casts to interface to the MPI routines. However, there's a small but real cost associated with using the largest type - it may require more time to perform operations and it increases the memory requirements. Bill On May 5, 2008, at 6:41 AM, Pavan Balaji wrote:
As per Rob's email, it looks like MPI_Aint is used for extents even for I/O. So, even if the processor address space is 32-bit, if the file-system supports > 2GB files, won't we still need to use a 64-bit MPI_Aint?
I agree that having an MPI_Offset smaller than MPI_Aint doesn't make sense.
-- Pavan
On 05/05/2008 06:30 AM, William Gropp wrote:
They can (and possibly should) be different if the processors address space is 32 bits but the file system supports > 2GB files and has 64 bit offsets. This is the case for BG. I can't think of a realistic case where the size of MPI_Offset would be less than the sizeof MPI_Aint, however.
Bill
On May 4, 2008, at 9:48 PM, Pavan Balaji wrote:
We should think about whether we really want to specify the size of MPI_Aint or simply say that MPI_Aint and MPI_Offset are the same size, and the size is the max of what is needed (address, file offset).
Do we want to give an option where they are not equal? That would be incorrect, right?
If you meant keeping MPI_Aint and MPI_Offset the same size *all* the time, I agree.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
This is exactly the problem. MPI_Aint is defined by the MPI standard as an "address-sized int". Users will expect to be able to use a pointer to hold the contents of an address-sized int, particularly when using the attribute routines, and this will break in the BG fixes. Internally,
Does the MPI standard say that MPI_Aint is exactly an "address-sized int" or does it say that it is large enough to contain an address? Here's a snippet from the MPI standard, which is a little vague IMO: ------------------------------------------------------------------------ Some MPI procedures use address arguments that represent an absolute address in the calling program. The datatype of such an argument is MPI Aint in C, MPI::Aint in C++ and INTEGER (KIND=MPI ADDRESS KIND) in Fortran. ------------------------------------------------------------------------ The following link seems to say that MPI_Aint should just be large enough to contain an address: http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi?coll=linux&db=man&fna... My point is, if the standard requires MPI_Aint to be exactly the size of an address, then as you pointed out, the BG fixes will break this.
MPICH uses MPI_Aint for datatype extents, in part because the MPI definition of the datatype routines does so. However, because the total data transferred is determined by (count,datatype), you can describe more than an MPI_Aint's worth of data even without having a single datatype with an extent greater than an MPI_Aint. One way out of this would be to add MPI datatype routines for extent that returned an MPI_Offset instead of an MPI_Aint; that would be a new routine (MPI 2.2 anyone?). Another would be to define an MPI_Size_t that was the largest of MPI_Aint, MPI_Offset, sizeof(void*), sizeof(void *()), .... and change all routines to use that single type (MPI 3?). Internally, MPICH2 could use that type now, performing the necessary casts to interface to the MPI routines. However, there's a small but real cost associated with using the largest type - it may require more time to perform operations and it increases the memory requirements.
I agree that whatever is broken in the MPI Standard (e.g., MPI_Type_extent which returns MPI_Aint as the extent) is out of our hands (at least for now). However, for whatever types are used internally, we can probably change them to use a larger type instead of MPI_Aint, as needed. As you pointed out, blindly increasing all types to the maximum size wouldn't be correct either. Also, why not use MPI_Offset internally for datatype and I/O operations, instead of defining a separate MPI_Size_t? Is there a case where MPI_Size_t would be larger or smaller than MPI_Offset? -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
The problem is that, as you note, the standard is ambiguous here. The problem is with established practice - users may have, understandably, assumed that MPI_Aint is exactly sizeof(void*). So while the letter of the standard may permit this, I fully expect it to generate bug reports from users, who will have suffered stack corruption because they didn't provide a longer-than-sizeof(void*) value for attribute values. Bill On May 5, 2008, at 8:37 AM, Pavan Balaji wrote:
Does the MPI standard say that MPI_Aint is exactly an "address-sized int" or does it say that it is large enough to contain an address?
Here's a snippet from the MPI standard, which is a little vague IMO:
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
In that case, more than 50% of the changes sent by IBM are invalid. So, I'm back to my first question :-). Are these changes (in principle) acceptable to be included in MPICH2? If not, we should tell them what to do about this. -- Pavan On 05/05/2008 08:54 AM, William Gropp wrote:
The problem is that, as you note, the standard is ambiguous here. The problem is with established practice - users may have, understandably, assumed that MPI_Aint is exactly sizeof(void*). So while the letter of the standard may permit this, I fully expect it to generate bug reports from users, who will have suffered stack corruption because they didn't provide a longer-than-sizeof(void*) value for attribute values.
Bill
On May 5, 2008, at 8:37 AM, Pavan Balaji wrote:
Does the MPI standard say that MPI_Aint is exactly an "address-sized
int" or does it say that it is large enough to contain an address?
Here's a snippet from the MPI standard, which is a little vague IMO:
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
I think that we want to allow for this tradeoff, but make sure that in the case where sizeof(MPI_Aint) == sizeof(void *) != sizeof (MPI_Offset), everything still works. That's why I'd like to think carefully about what we call the datatype size type internally to MPICH2, and how we export it. Because of limitations in the MPI spec itself, we'll have to make some tradeoffs for systems with 32 bit pointers and 64 bit file offsets. Bill On May 5, 2008, at 9:01 AM, Pavan Balaji wrote:
In that case, more than 50% of the changes sent by IBM are invalid. So, I'm back to my first question :-). Are these changes (in principle) acceptable to be included in MPICH2? If not, we should tell them what to do about this.
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
On Mon, May 05, 2008 at 08:37:34AM -0500, Pavan Balaji wrote:
This is exactly the problem. MPI_Aint is defined by the MPI standard as an "address-sized int". Users will expect to be able to use a pointer to hold the contents of an address-sized int, particularly when using the attribute routines, and this will break in the BG fixes. Internally,
Does the MPI standard say that MPI_Aint is exactly an "address-sized int" or does it say that it is large enough to contain an address?
Here's a snippet from the MPI standard, which is a little vague IMO:
------------------------------------------------------------------------ Some MPI procedures use address arguments that represent an absolute address in the calling program. The datatype of such an argument is MPI Aint in C, MPI::Aint in C++ and INTEGER (KIND=MPI ADDRESS KIND) in Fortran. ------------------------------------------------------------------------
The following link seems to say that MPI_Aint should just be large enough to contain an address: http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi?coll=linux&db=man&fna...
My point is, if the standard requires MPI_Aint to be exactly the size of an address, then as you pointed out, the BG fixes will break this.
Here's a slightly less vague passage (sec 2.6.3 of the 2.1 draft): ======================================================================== Address arguments are of MPI defined type MPI Aint. File displacements are of type MPI Offset. MPI Aint is defined to be an integer of the size needed to hold any valid address on the target architecture. MPI Offset is defined to be an integer of the size needed to hold any valid file size on the target architecture. ======================================================================== My inclination is to read it as only placing a minimum size constraint on MPI_Aint. Sadly, it could also be interpreted as constraining the maximum size as well. -Dave
The history behind all this is that on the BG/P, important I/O codes fail on
1,000 processes because MPI_Aint is 32-bit, and those codes access large files with derived datatypes that use MPI_Aint. We have been after IBM to fix it somehow, otherwise the machine would be unusable for such codes.
The easiest fix we and IBM could come up with was to bump up the size of MPI_Aint. It turned out to be less trivial than I had thought because the code assumes sizeof(void *) = sizeof(MPI_Aint) in many places, which required going through the code carefully and type casting etc. We had also volunteered to provide IBM with a test suite to test the correctness of these changes. IBM has been bugging us for these tests, and Rob Latham has provided them with a few, but they would like more. So, while we need not use IBM's exact fixes, we should fix it in some way -- whatever way we think is right. We can then try to propagate those fixes back to IBM. We should also get it fixed in the MPI Standard -- in 3.0 or 2.2, wherever it belongs. What exactly is the issue with the attribute routines? Rajeev
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Pavan Balaji Sent: Saturday, May 03, 2008 12:34 AM To: [email protected] Subject: Re: [mpich2-core] BG/P patches
Maybe you haven't been in on the conversation related to this? The issue is that MPI_Aint is the size for extents, even in I/I space, and so we need it to be 64 bit to be able to do useful file I/I (think subarray access from many processes).
Ah, got it. Thanks. I saw this discussion earlier, but couldn't pin-point the reason. So, we do want to accept the MPI_AINT_TO_VOID_PTR and family related parts of the patches, correct?
We'll probably also need to go through the entire code to make sure there are no random typecasts from MPI_Aint to (void *); the other way should be fine.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
The issue is that they use a void * argument as a void ** (using void * is a C idiom to avoid casting) but since this is a void *, if all you need is an int's worth of data, rather than allocating an int and setting the attribute to a pointer to that int, you might decide just to use the pointer as an int. That is, if I want to store the integer "3" on the communicator, rather than p = (int *)malloc(sizeof(int)); *p = 3; MPI_Comm_set_attr( ..., p ); I might do MPI_Aint pv=3; MPI_Comm_set_attr( ..., (void *)pv ); Later, I can do MPI_Aint v; MPI_Comm_get_attr( ..., (void *)&v ); But now I have a problem, because MPI_Comm_get_attr only sets a pointer's worth of data, and MPI_Aint is longer than a pointer's worth of data, so I don't get my data (actually, I lost it in the Comm_set_attr step for the same reason). Yes, this is a little sketchy, but its the sort of thing that you expect to work when you have an integer type for pointers. Basically, if sizeof(MPI_Aint) != sizeof(void *), some assumptions break, even in user code (it was this assumption that Rajeev mentioned that we overlooked originally). Exactly this sort of code is in the test suite; see mpich2/test/mpi/attr/attrt.c . Note that IBM had to change the test suite (in ways that I don't think are correct) to handle this. In particular, think about what happens on a system with 32 bit ints and 64 bit pointers in mpich2/test/mpi/attr/ attrerr.c ... Bill On May 5, 2008, at 2:32 PM, Rajeev Thakur wrote:
What exactly is the issue with the attribute routines?
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
So, while we need not use IBM's exact fixes, we should fix it in some way -- whatever way we think is right. We can then try to propagate those fixes back to IBM.
Would it make sense to initiate a discussion with IBM about this, or is it too early? I personally feel that we should internally use MPI_Offset instead of MPI_Aint for datatypes and I/O to calculate the total size and leave the rest (parts that are broken in the MPI standard) as they currently are. Specifically, anything that returns an MPI_Aint to the user might overflow on 32-bit systems (technically even on 64-bit systems, but we are some time away for that). But in cases where the user doesn't use such functions but MPICH2 internally has to evaluate the size, it should work fine. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
I think we should come up with a plan ourselves before discussing with IBM. Something that involves minimal changes to their code would be best, so that it is less work for us. Let me digest all the various mails on this topic, look at what IBM has done, and then come back... Rajeev
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Pavan Balaji Sent: Tuesday, May 06, 2008 4:07 PM To: [email protected] Subject: Re: [mpich2-core] BG/P patches
So, while we need not use IBM's exact fixes, we should fix it in some way -- whatever way we think is right. We can then try to propagate those fixes back to IBM.
Would it make sense to initiate a discussion with IBM about this, or is it too early?
I personally feel that we should internally use MPI_Offset instead of MPI_Aint for datatypes and I/O to calculate the total size and leave the rest (parts that are broken in the MPI standard) as they currently are. Specifically, anything that returns an MPI_Aint to the user might overflow on 32-bit systems (technically even on 64-bit systems, but we are some time away for that). But in cases where the user doesn't use such functions but MPICH2 internally has to evaluate the size, it should work fine.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group). Bill On May 6, 2008, at 4:41 PM, Rajeev Thakur wrote:
I think we should come up with a plan ourselves before discussing with IBM. Something that involves minimal changes to their code would be best, so that it is less work for us. Let me digest all the various mails on this topic, look at what IBM has done, and then come back...
Rajeev
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Pavan Balaji Sent: Tuesday, May 06, 2008 4:07 PM To: [email protected] Subject: Re: [mpich2-core] BG/P patches
So, while we need not use IBM's exact fixes, we should fix it in some way -- whatever way we think is right. We can then try to propagate those fixes back to IBM.
Would it make sense to initiate a discussion with IBM about this, or is it too early?
I personally feel that we should internally use MPI_Offset instead of MPI_Aint for datatypes and I/O to calculate the total size and leave the rest (parts that are broken in the MPI standard) as they currently are. Specifically, anything that returns an MPI_Aint to the user might overflow on 32-bit systems (technically even on 64-bit systems, but we are some time away for that). But in cases where the user doesn't use such functions but MPICH2 internally has to evaluate the size, it should work fine.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group).
Just to clarify my original suggestion -- I think all internal extents and sizes should be stored as MPI_Offset. If the user requests for the extent, we can always return an MPI_Aint (ignoring the overflow). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
This is exactly the problem - the application might need an extent that won't fit in an MPI_Aint . Right now, we don't have a good solution for those applications. Bill On May 7, 2008, at 8:58 AM, Pavan Balaji wrote:
I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group).
Just to clarify my original suggestion -- I think all internal extents and sizes should be stored as MPI_Offset. If the user requests for the extent, we can always return an MPI_Aint (ignoring the overflow).
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
Right. We need to be able to return the correct extent. I/O libraries/ applications need this capability, as Bill says. -- Rob On May 7, 2008, at 9:27 AM, William Gropp wrote:
This is exactly the problem - the application might need an extent that won't fit in an MPI_Aint . Right now, we don't have a good solution for those applications.
Bill
On May 7, 2008, at 8:58 AM, Pavan Balaji wrote:
I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group).
Just to clarify my original suggestion -- I think all internal extents and sizes should be stored as MPI_Offset. If the user requests for the extent, we can always return an MPI_Aint (ignoring the overflow).
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
I understand the problem. What I'm suggesting is that whatever has to be returned to the user can be an overflown value. But internally we can store the correct value. If the user needs this returned value for allocating a buffer, etc., then that's a problem, but that part is really a problem in the MPI standard, and not in the MPICH2 implementation. -- Pavan On Wed, 7 May 2008, Rob Ross wrote:
Right. We need to be able to return the correct extent. I/O libraries/ applications need this capability, as Bill says. -- Rob
On May 7, 2008, at 9:27 AM, William Gropp wrote:
This is exactly the problem - the application might need an extent that won't fit in an MPI_Aint . Right now, we don't have a good solution for those applications.
Bill
On May 7, 2008, at 8:58 AM, Pavan Balaji wrote:
I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group).
Just to clarify my original suggestion -- I think all internal extents and sizes should be stored as MPI_Offset. If the user requests for the extent, we can always return an MPI_Aint (ignoring the overflow).
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
We're starting to go around in the same circle. Here's a summary: MPI Spec issues: The MPI spec defines type extents as MPI_Aint ; some other fields are ints. File offsets are in terms of MPI_Offset , which may be bigger (or smaller) than an MPI_Aint It is possible to create MPI datatypes bigger than what can be represented in an address-sized integer; this can be relevant to datatypes used with I/O. Users expect the size of a pointer to be exactly the size of an MPI_Aint. If this is not true, some operations with attributes will fail (cause memory/stack overwrites) MPICH2 Implementation issues: Internal implementation of datatypes uses MPI_Aint; there may be performance reasons for that on some platforms (instead of always using MPI_Offset) Blue Gene issues: MPI_Offset is larger than a pointer. Some I/O libraries (HDF5, as Rob L confirms) may need extents that are greater than 32 bits. The problem is that there is no solution that simultaneously satisfies every one of these requirements. This is key; *any* solution is going to have some warts. So, what can we do, and what principles should we follow? One principle that should have priority is the integrity of the test suites. For example, I would not make any changes to the attribute tests, unless they're clearly wrong (not just ambiguous). This limits the ability to replace MPI_Aint with an integer that is longer than a pointer in MPI (this is not the same as changing the use of MPI_Aint internally to MPICH2). Another principle is to avoid mandating something that might significantly impact performance. This is why I'm leery of mandating MPI_Offset in the datatype code, at least for now. Not all processors support 64 bit integer operations with the same speed as 32 bit operations, though that may change in the next few years. One thing that can be done for the datatype code *internals* is to use an MPIR_Size_t type; this can be typedef'ed appropriately. This would permit us to distinguish between what MPI defines as an Aint, what MPI defines as an Offset, and what MPICH2 wants to use internally for message lengths and datatype operations. The tradeoff might be that for some systems with 32 bit Aints and 64bit Offsets, some file operations could not be expressed with datatypes. For the HDF5 problem, the cleanest solution might be to propose an MPI 2.2 extension and then implement it. Bill On May 7, 2008, at 9:41 AM, Pavan Balaji wrote:
I understand the problem. What I'm suggesting is that whatever has to be returned to the user can be an overflown value. But internally we can store the correct value.
If the user needs this returned value for allocating a buffer, etc., then that's a problem, but that part is really a problem in the MPI standard, and not in the MPICH2 implementation.
-- Pavan
On Wed, 7 May 2008, Rob Ross wrote:
Right. We need to be able to return the correct extent. I/O libraries/ applications need this capability, as Bill says. -- Rob
On May 7, 2008, at 9:27 AM, William Gropp wrote:
This is exactly the problem - the application might need an extent that won't fit in an MPI_Aint . Right now, we don't have a good solution for those applications.
Bill
On May 7, 2008, at 8:58 AM, Pavan Balaji wrote:
I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group).
Just to clarify my original suggestion -- I think all internal extents and sizes should be stored as MPI_Offset. If the user requests for the extent, we can always return an MPI_Aint (ignoring the overflow).
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
Let's see if I have understood this right: 1. IBM promotes Aint to 64 bits and does the appropriate type casts. 2. Pavan suggests using MPI_Offset internally for datatypes and I/O instead of Aint. Return MPI_Aint to the user, as defined by MPI, possibly causing truncation. 3. Bill suggests using an MPIR_Size_t internally to represent message and I/O lengths in the datatype code. I think Pavan and Bill's suggestions are similar, except that Bill wants to use the bigger size in fewer places -- only where calculations occur. I am also attaching Rob's original proposal to IBM, where he outlined two approaches. The first one suggests using the dataloop code for I/O and using MPI_Offset to hold integer values. This is similar in spirit to Pavan and Bill's. The second one is to bump up MPI_Aint size. We originally recommended to IBM that they do the first solution, but then changed our mind after a month thinking it would be too much work, and recommended the Aint option. IBM also independently evaluated the two approaches and was in favor of Aint. What would be the least amount of work and also do the job right? Suppose we went with the MPI_Aint approach, but not made it the default. Instead it has to be selected explicitly at configure time by those who need it. (These are not those who would use MPI_Aint i instead of int *i in their attribute code.) Most users will not even see the change, and on 64-bit systems it is never an issue. Those users on 32-bit systems needing large files can be told to configure it. (Last I heard, IBM was also thinking of providing two versions of their library.) Would that work? Rajeev ________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of William Gropp Sent: Wednesday, May 07, 2008 11:41 AM To: [email protected] Subject: Re: [mpich2-core] BG/P patches We're starting to go around in the same circle. Here's a summary: MPI Spec issues: The MPI spec defines type extents as MPI_Aint ; some other fields are ints. File offsets are in terms of MPI_Offset , which may be bigger (or smaller) than an MPI_Aint It is possible to create MPI datatypes bigger than what can be represented in an address-sized integer; this can be relevant to datatypes used with I/O. Users expect the size of a pointer to be exactly the size of an MPI_Aint. If this is not true, some operations with attributes will fail (cause memory/stack overwrites) MPICH2 Implementation issues: Internal implementation of datatypes uses MPI_Aint; there may be performance reasons for that on some platforms (instead of always using MPI_Offset) Blue Gene issues: MPI_Offset is larger than a pointer. Some I/O libraries (HDF5, as Rob L confirms) may need extents that are greater than 32 bits. The problem is that there is no solution that simultaneously satisfies every one of these requirements. This is key; *any* solution is going to have some warts. So, what can we do, and what principles should we follow? One principle that should have priority is the integrity of the test suites. For example, I would not make any changes to the attribute tests, unless they're clearly wrong (not just ambiguous). This limits the ability to replace MPI_Aint with an integer that is longer than a pointer in MPI (this is not the same as changing the use of MPI_Aint internally to MPICH2). Another principle is to avoid mandating something that might significantly impact performance. This is why I'm leery of mandating MPI_Offset in the datatype code, at least for now. Not all processors support 64 bit integer operations with the same speed as 32 bit operations, though that may change in the next few years. One thing that can be done for the datatype code *internals* is to use an MPIR_Size_t type; this can be typedef'ed appropriately. This would permit us to distinguish between what MPI defines as an Aint, what MPI defines as an Offset, and what MPICH2 wants to use internally for message lengths and datatype operations. The tradeoff might be that for some systems with 32 bit Aints and 64bit Offsets, some file operations could not be expressed with datatypes. For the HDF5 problem, the cleanest solution might be to propose an MPI 2.2 extension and then implement it. Bill On May 7, 2008, at 9:41 AM, Pavan Balaji wrote: I understand the problem. What I'm suggesting is that whatever has to be returned to the user can be an overflown value. But internally we can store the correct value. If the user needs this returned value for allocating a buffer, etc., then that's a problem, but that part is really a problem in the MPI standard, and not in the MPICH2 implementation. -- Pavan On Wed, 7 May 2008, Rob Ross wrote: Right. We need to be able to return the correct extent. I/O libraries/ applications need this capability, as Bill says. -- Rob On May 7, 2008, at 9:27 AM, William Gropp wrote: This is exactly the problem - the application might need an extent that won't fit in an MPI_Aint . Right now, we don't have a good solution for those applications. Bill On May 7, 2008, at 8:58 AM, Pavan Balaji wrote: I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group). Just to clarify my original suggestion -- I think all internal extents and sizes should be stored as MPI_Offset. If the user requests for the extent, we can always return an MPI_Aint (ignoring the overflow). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign -- Pavan Balaji http://www.mcs.anl.gov/~balaji William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
I think that Rob's solution needs to be part of the overall solution - rather than rely on interfaces in the MPI spec that we know are broken, we should simply do the right thing in terms of using the internal spec. For MPI implementations that don't want to expose a lower layer, and that don't want to face this real issue, we can set it up so they can continue to use the MPI routines as ROMIO does now. We'll still need the option of making the dataloop code use a type that is large enough for both Aint and Offset. And we should provide an option to detect and report truncation (MPI_Address did this for Fortran, where a similar problem existed). Bill On May 7, 2008, at 6:23 PM, Rajeev Thakur wrote:
Let's see if I have understood this right:
1. IBM promotes Aint to 64 bits and does the appropriate type casts.
2. Pavan suggests using MPI_Offset internally for datatypes and I/O instead of Aint. Return MPI_Aint to the user, as defined by MPI, possibly causing truncation.
3. Bill suggests using an MPIR_Size_t internally to represent message and I/O lengths in the datatype code.
I think Pavan and Bill's suggestions are similar, except that Bill wants to use the bigger size in fewer places -- only where calculations occur.
I am also attaching Rob's original proposal to IBM, where he outlined two approaches. The first one suggests using the dataloop code for I/O and using MPI_Offset to hold integer values. This is similar in spirit to Pavan and Bill's. The second one is to bump up MPI_Aint size. We originally recommended to IBM that they do the first solution, but then changed our mind after a month thinking it would be too much work, and recommended the Aint option. IBM also independently evaluated the two approaches and was in favor of Aint.
What would be the least amount of work and also do the job right?
Suppose we went with the MPI_Aint approach, but not made it the default. Instead it has to be selected explicitly at configure time by those who need it. (These are not those who would use MPI_Aint i instead of int *i in their attribute code.) Most users will not even see the change, and on 64- bit systems it is never an issue. Those users on 32-bit systems needing large files can be told to configure it. (Last I heard, IBM was also thinking of providing two versions of their library.)
Would that work?
Rajeev
________________________________
From: [email protected] [mailto:[email protected]] On Behalf Of William Gropp Sent: Wednesday, May 07, 2008 11:41 AM To: [email protected] Subject: Re: [mpich2-core] BG/P patches
We're starting to go around in the same circle. Here's a summary:
MPI Spec issues:
The MPI spec defines type extents as MPI_Aint ; some other fields are ints.
File offsets are in terms of MPI_Offset , which may be bigger (or smaller) than an MPI_Aint
It is possible to create MPI datatypes bigger than what can be represented in an address-sized integer; this can be relevant to datatypes used with I/O.
Users expect the size of a pointer to be exactly the size of an MPI_Aint. If this is not true, some operations with attributes will fail (cause memory/stack overwrites)
MPICH2 Implementation issues:
Internal implementation of datatypes uses MPI_Aint; there may be performance reasons for that on some platforms (instead of always using MPI_Offset)
Blue Gene issues:
MPI_Offset is larger than a pointer. Some I/O libraries (HDF5, as Rob L confirms) may need extents that are greater than 32 bits.
The problem is that there is no solution that simultaneously satisfies every one of these requirements. This is key; *any* solution is going to have some warts. So, what can we do, and what principles should we follow?
One principle that should have priority is the integrity of the test suites. For example, I would not make any changes to the attribute tests, unless they're clearly wrong (not just ambiguous). This limits the ability to replace MPI_Aint with an integer that is longer than a pointer in MPI (this is not the same as changing the use of MPI_Aint internally to MPICH2).
Another principle is to avoid mandating something that might significantly impact performance. This is why I'm leery of mandating MPI_Offset in the datatype code, at least for now. Not all processors support 64 bit integer operations with the same speed as 32 bit operations, though that may change in the next few years.
One thing that can be done for the datatype code *internals* is to use an MPIR_Size_t type; this can be typedef'ed appropriately. This would permit us to distinguish between what MPI defines as an Aint, what MPI defines as an Offset, and what MPICH2 wants to use internally for message lengths and datatype operations. The tradeoff might be that for some systems with 32 bit Aints and 64bit Offsets, some file operations could not be expressed with datatypes.
For the HDF5 problem, the cleanest solution might be to propose an MPI 2.2 extension and then implement it.
Bill
On May 7, 2008, at 9:41 AM, Pavan Balaji wrote:
I understand the problem. What I'm suggesting is that whatever has to be returned to the user can be an overflown value. But internally we can store the correct value.
If the user needs this returned value for allocating a buffer, etc., then that's a problem, but that part is really a problem in the MPI standard, and not in the MPICH2 implementation.
-- Pavan
On Wed, 7 May 2008, Rob Ross wrote:
Right. We need to be able to return the correct extent. I/O libraries/ applications need this capability, as Bill says. -- Rob
On May 7, 2008, at 9:27 AM, William Gropp wrote:
This is exactly the problem - the application might need an extent that won't fit in an MPI_Aint . Right now, we don't have a good solution for those applications.
Bill
On May 7, 2008, at 8:58 AM, Pavan Balaji wrote:
I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group).
Just to clarify my original suggestion -- I think all internal extents and sizes should be stored as MPI_Offset. If the user requests for the extent, we can always return an MPI_Aint (ignoring the overflow).
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana- Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
<bgp-mpi-io-plan.txt>
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
So should we ignore IBM's changes and plan to fix it the other way? Given the amount of effort involved, it probably won't happen until the MPI Forum formalizes a fix at the interface level. Rajeev _____ From: [email protected] [mailto:[email protected]] On Behalf Of William Gropp Sent: Wednesday, May 07, 2008 7:14 PM To: [email protected] Subject: Re: [mpich2-core] BG/P patches I think that Rob's solution needs to be part of the overall solution - rather than rely on interfaces in the MPI spec that we know are broken, we should simply do the right thing in terms of using the internal spec. For MPI implementations that don't want to expose a lower layer, and that don't want to face this real issue, we can set it up so they can continue to use the MPI routines as ROMIO does now. We'll still need the option of making the dataloop code use a type that is large enough for both Aint and Offset. And we should provide an option to detect and report truncation (MPI_Address did this for Fortran, where a similar problem existed). Bill On May 7, 2008, at 6:23 PM, Rajeev Thakur wrote: Let's see if I have understood this right: 1. IBM promotes Aint to 64 bits and does the appropriate type casts. 2. Pavan suggests using MPI_Offset internally for datatypes and I/O instead of Aint. Return MPI_Aint to the user, as defined by MPI, possibly causing truncation. 3. Bill suggests using an MPIR_Size_t internally to represent message and I/O lengths in the datatype code. I think Pavan and Bill's suggestions are similar, except that Bill wants to use the bigger size in fewer places -- only where calculations occur. I am also attaching Rob's original proposal to IBM, where he outlined two approaches. The first one suggests using the dataloop code for I/O and using MPI_Offset to hold integer values. This is similar in spirit to Pavan and Bill's. The second one is to bump up MPI_Aint size. We originally recommended to IBM that they do the first solution, but then changed our mind after a month thinking it would be too much work, and recommended the Aint option. IBM also independently evaluated the two approaches and was in favor of Aint. What would be the least amount of work and also do the job right? Suppose we went with the MPI_Aint approach, but not made it the default. Instead it has to be selected explicitly at configure time by those who need it. (These are not those who would use MPI_Aint i instead of int *i in their attribute code.) Most users will not even see the change, and on 64-bit systems it is never an issue. Those users on 32-bit systems needing large files can be told to configure it. (Last I heard, IBM was also thinking of providing two versions of their library.) Would that work? Rajeev ________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of William Gropp Sent: Wednesday, May 07, 2008 11:41 AM To: [email protected] Subject: Re: [mpich2-core] BG/P patches We're starting to go around in the same circle. Here's a summary: MPI Spec issues: The MPI spec defines type extents as MPI_Aint ; some other fields are ints. File offsets are in terms of MPI_Offset , which may be bigger (or smaller) than an MPI_Aint It is possible to create MPI datatypes bigger than what can be represented in an address-sized integer; this can be relevant to datatypes used with I/O. Users expect the size of a pointer to be exactly the size of an MPI_Aint. If this is not true, some operations with attributes will fail (cause memory/stack overwrites) MPICH2 Implementation issues: Internal implementation of datatypes uses MPI_Aint; there may be performance reasons for that on some platforms (instead of always using MPI_Offset) Blue Gene issues: MPI_Offset is larger than a pointer. Some I/O libraries (HDF5, as Rob L confirms) may need extents that are greater than 32 bits. The problem is that there is no solution that simultaneously satisfies every one of these requirements. This is key; *any* solution is going to have some warts. So, what can we do, and what principles should we follow? One principle that should have priority is the integrity of the test suites. For example, I would not make any changes to the attribute tests, unless they're clearly wrong (not just ambiguous). This limits the ability to replace MPI_Aint with an integer that is longer than a pointer in MPI (this is not the same as changing the use of MPI_Aint internally to MPICH2). Another principle is to avoid mandating something that might significantly impact performance. This is why I'm leery of mandating MPI_Offset in the datatype code, at least for now. Not all processors support 64 bit integer operations with the same speed as 32 bit operations, though that may change in the next few years. One thing that can be done for the datatype code *internals* is to use an MPIR_Size_t type; this can be typedef'ed appropriately. This would permit us to distinguish between what MPI defines as an Aint, what MPI defines as an Offset, and what MPICH2 wants to use internally for message lengths and datatype operations. The tradeoff might be that for some systems with 32 bit Aints and 64bit Offsets, some file operations could not be expressed with datatypes. For the HDF5 problem, the cleanest solution might be to propose an MPI 2.2 extension and then implement it. Bill On May 7, 2008, at 9:41 AM, Pavan Balaji wrote: I understand the problem. What I'm suggesting is that whatever has to be returned to the user can be an overflown value. But internally we can store the correct value. If the user needs this returned value for allocating a buffer, etc., then that's a problem, but that part is really a problem in the MPI standard, and not in the MPICH2 implementation. -- Pavan On Wed, 7 May 2008, Rob Ross wrote: Right. We need to be able to return the correct extent. I/O libraries/ applications need this capability, as Bill says. -- Rob On May 7, 2008, at 9:27 AM, William Gropp wrote: This is exactly the problem - the application might need an extent that won't fit in an MPI_Aint . Right now, we don't have a good solution for those applications. Bill On May 7, 2008, at 8:58 AM, Pavan Balaji wrote: I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group). Just to clarify my original suggestion -- I think all internal extents and sizes should be stored as MPI_Offset. If the user requests for the extent, we can always return an MPI_Aint (ignoring the overflow). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign -- Pavan Balaji http://www.mcs.anl.gov/~balaji William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign <bgp-mpi-io-plan.txt> William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
I think that we can get a large part of the way there with a few changes. I'd do 1. extent (and related) types in dataloop code -> change from MPI_Aint to MPIR_Size_t . Change mpiimpl.h to define this to MPI_Offset by default, but permit MPI_Aint as the type. The reason to do this now is that we'll have to make *some* change, and this one gives us the most flexibility. 2. Change ROMIO internals to use internal functions for datatype information, and provide a fallback implementation that makes use of MPI routines (complete with the bugs that will create for the BG case). The reason to do this now is similar; we'll need to make some change (if only to the MPI 2.2 or 3.0 versions of the MPI_Offset- sized-extent functions), and this will either eliminate the need or make it easy to find/replace. Both of these changes affect only the the internals of the implementation and won't impact the test suites or user codes. Bill On May 8, 2008, at 1:33 PM, Rajeev Thakur wrote:
So should we ignore IBM's changes and plan to fix it the other way? Given the amount of effort involved, it probably won't happen until the MPI Forum formalizes a fix at the interface level.
Rajeev
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
I think this is doable. (1) should be relatively easy although it requires going through the code carefully. (2), as I understand it, is to just update the flattening code in ROMIO to use MPIR_Size_t instead of Aint and use new internal datatype functions that return extents as MPIR_Size_t. This may perhaps be less work than updating ROMIO to use the dataloop code, although 60% of that has already been done by Rob in a separate directory that is not being compiled. All this also means that the code diverges significantly from IBM's code. Would that be ok? Rajeev
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of William Gropp Sent: Thursday, May 08, 2008 1:49 PM To: [email protected] Subject: Re: [mpich2-core] BG/P patches
I think that we can get a large part of the way there with a few changes. I'd do
1. extent (and related) types in dataloop code -> change from MPI_Aint to MPIR_Size_t . Change mpiimpl.h to define this to MPI_Offset by default, but permit MPI_Aint as the type. The reason to do this now is that we'll have to make *some* change, and this one gives us the most flexibility.
2. Change ROMIO internals to use internal functions for datatype information, and provide a fallback implementation that makes use of MPI routines (complete with the bugs that will create for the BG case). The reason to do this now is similar; we'll need to make some change (if only to the MPI 2.2 or 3.0 versions of the MPI_Offset-sized-extent functions), and this will either eliminate the need or make it easy to find/replace.
Both of these changes affect only the the internals of the implementation and won't impact the test suites or user codes.
Bill
On May 8, 2008, at 1:33 PM, Rajeev Thakur wrote:
So should we ignore IBM's changes and plan to fix it the other way? Given the amount of effort involved, it probably won't happen until the MPI Forum formalizes a fix at the interface level. Rajeev
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
On Wed, May 07, 2008 at 07:44:45AM -0500, William Gropp wrote:
I agree with Rajeev's plan. One thing to bear in mind (its in the mails but might not jump out) is that some I/O applications may need an MPI_Offset sized datatype extent (HDF5 might be in this group).
HDF5 is indeed in this group. BobC from IBM reported that HDF5 tests were failing with 32 bit MPI_Aints but pass with 64 bit MPI_Aints. ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
I have reviewed my two patches and attached my comments. Lots of the code is BG-specific and generally unusable. Of the 47 files patched, 22 of them pass muster while the remaining 25 should either be thrown out or otherwise modified. IBM doesn't seem to have their MPI_Aint story straight internally, since there are several changes that they made where they assume that sizeof(MPI_Aint)==sizeof(int) or sizeof(MPI_Aint)==sizeof(long). Also, they seem to have omitted several new files, such as wtimef90.f90 and all of the IO-node tests. -Dave On Fri, May 02, 2008 at 03:11:11AM -0500, Pavan Balaji wrote:
Hi all,
Sorry about the delay in sending out the BG/P patches. I've placed them here: http://www.mcs.anl.gov/~balaji/projects/mpich2/bgp/r2. I've assigned a few patches to each person below. Can the below folks please confirm that you can go over them and make sure there are no problems integrating them into the trunk? The plan is to go over these and have a telecon around mid next week.
The IBM folks didn't do a very good job in breaking up the diffs, so some of the patches are large; you'll probably need to break them up and try different parts separately.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev test-build.patch -- Dave test-source.patch -- Dave
A few things I noticed on a quick glance through the patches:
1. MPI_Aint vs. (void *). I remember that there was some discussion, but what was the conclusion? Can MPI_Aint be larger than (void *)? A large chunk of the patches assumes that these two can be different, so they introduce MPI_AINT_TO_VOID_PTR and friends to deal with such cases.
2. Some of the patches (e.g., debugger.patch, test-build.patch and test-source.patch) have a number of BG specific additions. These need to be pruned out in a way that IBM can still use it, but they will continue to be generic enough for other platforms.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
Just a reminder that this telecon is on Monday at 1p. If you haven't already looked at the patches assigned to you, please do so. Thanks. -- Pavan On 05/02/2008 03:11 AM, Pavan Balaji wrote:
Hi all,
Sorry about the delay in sending out the BG/P patches. I've placed them here: http://www.mcs.anl.gov/~balaji/projects/mpich2/bgp/r2. I've assigned a few patches to each person below. Can the below folks please confirm that you can go over them and make sure there are no problems integrating them into the trunk? The plan is to go over these and have a telecon around mid next week.
The IBM folks didn't do a very good job in breaking up the diffs, so some of the patches are large; you'll probably need to break them up and try different parts separately.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev test-build.patch -- Dave test-source.patch -- Dave
A few things I noticed on a quick glance through the patches:
1. MPI_Aint vs. (void *). I remember that there was some discussion, but what was the conclusion? Can MPI_Aint be larger than (void *)? A large chunk of the patches assumes that these two can be different, so they introduce MPI_AINT_TO_VOID_PTR and friends to deal with such cases.
2. Some of the patches (e.g., debugger.patch, test-build.patch and test-source.patch) have a number of BG specific additions. These need to be pruned out in a way that IBM can still use it, but they will continue to be generic enough for other platforms.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
Hi all, Now that the discussion with IBM is done, we should work on integrating the BG patches into the MPICH2 trunk.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev romio-bgl.patch -- RobL romio.patch -- RobL test-build.patch -- Dave test-source.patch -- Dave
I've already integrated the dcmfd patch and am working on integrating the datatype and ignore patches. Whoever has looked at the above patches, can you spend some time to integrate these into the trunk? From the discussion with IBM, the debugger patch is delayed for now and a part of the test suite patches are not being integrated as well. Everything else should go in. Once the build patch is in, we can start testing the stack on BG/P, so that will probably be the critical requirement for this integration. Thanks. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
I wanted to comment on the test build patch, even though that's also in the deferred category. Some of the changes to the runtests script were intended to bypass the io, rma, and spawn tests. But there is already a way to do that; the testlist file that is created by the test configure has these as configure variables just so MPI implementations that don't handle these can be tested without funky edits like this. Bill On Jun 23, 2008, at 8:12 PM, Pavan Balaji wrote:
Hi all,
Now that the discussion with IBM is done, we should work on integrating the BG patches into the MPICH2 trunk.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev romio-bgl.patch -- RobL romio.patch -- RobL test-build.patch -- Dave test-source.patch -- Dave
I've already integrated the dcmfd patch and am working on integrating the datatype and ignore patches. Whoever has looked at the above patches, can you spend some time to integrate these into the trunk? From the discussion with IBM, the debugger patch is delayed for now and a part of the test suite patches are not being integrated as well. Everything else should go in.
Once the build patch is in, we can start testing the stack on BG/P, so that will probably be the critical requirement for this integration.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
On Tue, Jun 24, 2008 at 08:18:25AM -0500, William Gropp wrote:
I wanted to comment on the test build patch, even though that's also in the deferred category. Some of the changes to the runtests script were intended to bypass the io, rma, and spawn tests. But there is already a way to do that; the testlist file that is created by the test configure has these as configure variables just so MPI implementations that don't handle these can be tested without funky edits like this.
I took a look at top-level configure, runtests.in and the MPICH2 wiki but could not find it: how do I run just the I/O tests? ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
The capability is to skip tests - if configured with --enable- romio=no , then the testlist line that has @iodir@ is set to "#". There are similar replacements for spawndir and rmadir . The IBM patch to runtest includes code to check for and skip these directories; that isn't necessary and should have been done by having configure update the mpich2/test/mpi/testlist file appropriately. Many of the Makefiles have a testing target ; you can just do cd io && make testing after configuring the tests. Bill On Jun 24, 2008, at 10:28 AM, Robert Latham wrote:
On Tue, Jun 24, 2008 at 08:18:25AM -0500, William Gropp wrote:
I wanted to comment on the test build patch, even though that's also in the deferred category. Some of the changes to the runtests script were intended to bypass the io, rma, and spawn tests. But there is already a way to do that; the testlist file that is created by the test configure has these as configure variables just so MPI implementations that don't handle these can be tested without funky edits like this.
I took a look at top-level configure, runtests.in and the MPICH2 wiki but could not find it: how do I run just the I/O tests?
==rob
-- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
Here's the status of the BG/P patches:
build.patch -- Bill
[DONE] Committed to trunk.
datatype.patch -- Pavan
[ALMOST] Ready to merge; waiting on IBM's response on one issue.
dcmfd.patch -- Pavan
[DONE] Committed to trunk.
debugger.patch -- Darius
[IGNORE] Deferred for now. Darius and Bill will follow-up with IBM on this.
ignore.patch -- Pavan
[TODO] Not done yet.
mpe.patch -- Anthony
[DONE] Committed to trunk.
other.patch -- Rajeev
[ALMOST] Ready to merge; waiting on IBM's response on several issues.
romio-bgl.patch -- RobL romio.patch -- RobL
[TODO] RobL is working on this.
test-build.patch -- Dave test-source.patch -- Dave
[TODO] Dave will get to this once the Nemesis dynamic processes part is done. This patch is not critical, so can be delayed, but should try to have it in before the release. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
On Wed, Jul 02, 2008 at 11:29:45PM -0500, Pavan Balaji wrote:
romio-bgl.patch -- RobL romio.patch -- RobL
[TODO] RobL is working on this.
romio-bgl.patch is committed to my staging branch The easy half of the romio.patch is either committed to the staging branch or dropped. I think I'm just going to have to hold my nose and commit the type casts and code rearranging scattered through the remaining "32 files changed, 891 insertions(+), 503 deletions(-)" ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
what sort of code rearranging? -- rob On Jul 3, 2008, at 9:24 AM, Robert Latham wrote:
On Wed, Jul 02, 2008 at 11:29:45PM -0500, Pavan Balaji wrote:
romio-bgl.patch -- RobL romio.patch -- RobL
[TODO] RobL is working on this.
romio-bgl.patch is committed to my staging branch
The easy half of the romio.patch is either committed to the staging branch or dropped. I think I'm just going to have to hold my nose and commit the type casts and code rearranging scattered through the remaining "32 files changed, 891 insertions(+), 503 deletions(-)"
==rob
-- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
On Thu, Jul 03, 2008 at 09:34:17AM -0500, Rob Ross wrote:
what sort of code rearranging? -- rob
Aside from the casts, there are two major things the IBM patches do: - additional and slightly mpich2-specific debugging code DBG_FPRINTF and ADIOI_Assert added to many many places - In several places (ADIOI_GEN_ReadStrided, ADIOI_GEN_WriteStrided, ADIOI_Fill_user_buffer) they moved the macro definitions (things like ADIOI_BUF_INCR and ADIOI_BUFFERED_READ) inside the function. Maybe this makes xl compilers happier? ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
i'd ask them if there is a XL compiler reason for the macro moves. the ADIOI_Asserts are fine with me. don't care about the fprintfs. rob On Jul 3, 2008, at 9:46 AM, Robert Latham wrote:
On Thu, Jul 03, 2008 at 09:34:17AM -0500, Rob Ross wrote:
what sort of code rearranging? -- rob
Aside from the casts, there are two major things the IBM patches do:
- additional and slightly mpich2-specific debugging code DBG_FPRINTF and ADIOI_Assert added to many many places
- In several places (ADIOI_GEN_ReadStrided, ADIOI_GEN_WriteStrided, ADIOI_Fill_user_buffer) they moved the macro definitions (things like ADIOI_BUF_INCR and ADIOI_BUFFERED_READ) inside the function. Maybe this makes xl compilers happier?
==rob
-- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B
I'm in the final stages of testing the changes to build.patch . I've attached an item-by-item summary. Bill  On Jun 23, 2008, at 8:12 PM, Pavan Balaji wrote:
Hi all,
Now that the discussion with IBM is done, we should work on integrating the BG patches into the MPICH2 trunk.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev romio-bgl.patch -- RobL romio.patch -- RobL test-build.patch -- Dave test-source.patch -- Dave
I've already integrated the dcmfd patch and am working on integrating the datatype and ignore patches. Whoever has looked at the above patches, can you spend some time to integrate these into the trunk? From the discussion with IBM, the debugger patch is delayed for now and a part of the test suite patches are not being integrated as well. Everything else should go in.
Once the build patch is in, we can start testing the stack on BG/P, so that will probably be the critical requirement for this integration.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
Hi Bill, Did you already commit this patch into the trunk? If you are still working on it, can you send out your current (temporarily workable) version of the patch? We can use it to compile the patches we are working on and check for errors. Thanks. -- Pavan William Gropp wrote:
I'm in the final stages of testing the changes to build.patch . I've attached an item-by-item summary.
Bill
------------------------------------------------------------------------
On Jun 23, 2008, at 8:12 PM, Pavan Balaji wrote:
Hi all,
Now that the discussion with IBM is done, we should work on integrating the BG patches into the MPICH2 trunk.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev romio-bgl.patch -- RobL romio.patch -- RobL test-build.patch -- Dave test-source.patch -- Dave
I've already integrated the dcmfd patch and am working on integrating the datatype and ignore patches. Whoever has looked at the above patches, can you spend some time to integrate these into the trunk? From the discussion with IBM, the debugger patch is delayed for now and a part of the test suite patches are not being integrated as well. Everything else should go in.
Once the build patch is in, we can start testing the stack on BG/P, so that will probably be the critical requirement for this integration.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
Yes, I committed it to the trunk. Bill On Jun 29, 2008, at 12:04 AM, Pavan Balaji wrote:
Hi Bill,
Did you already commit this patch into the trunk? If you are still working on it, can you send out your current (temporarily workable) version of the patch? We can use it to compile the patches we are working on and check for errors.
Thanks.
-- Pavan
William Gropp wrote:
I'm in the final stages of testing the changes to build.patch . I've attached an item-by-item summary.
Bill
--------------------------------------------------------------------- ---
On Jun 23, 2008, at 8:12 PM, Pavan Balaji wrote:
Hi all,
Now that the discussion with IBM is done, we should work on integrating the BG patches into the MPICH2 trunk.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev romio-bgl.patch -- RobL romio.patch -- RobL test-build.patch -- Dave test-source.patch -- Dave
I've already integrated the dcmfd patch and am working on integrating the datatype and ignore patches. Whoever has looked at the above patches, can you spend some time to integrate these into the trunk? From the discussion with IBM, the debugger patch is delayed for now and a part of the test suite patches are not being integrated as well. Everything else should go in.
Once the build patch is in, we can start testing the stack on BG/ P, so that will probably be the critical requirement for this integration.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
Hi Bill, Some parts of the build.patch don't seem to have been committed. For example, MPI_AINT_CAST_TO_VOID_PTR doesn't seem to be in. Was there a change in this part? -- Pavan On 07/01/2008 03:50 PM, William Gropp wrote:
Yes, I committed it to the trunk.
Bill
On Jun 29, 2008, at 12:04 AM, Pavan Balaji wrote:
Hi Bill,
Did you already commit this patch into the trunk? If you are still working on it, can you send out your current (temporarily workable) version of the patch? We can use it to compile the patches we are working on and check for errors.
Thanks.
-- Pavan
William Gropp wrote:
I'm in the final stages of testing the changes to build.patch . I've attached an item-by-item summary.
Bill
------------------------------------------------------------------------
On Jun 23, 2008, at 8:12 PM, Pavan Balaji wrote:
Hi all,
Now that the discussion with IBM is done, we should work on integrating the BG patches into the MPICH2 trunk.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev romio-bgl.patch -- RobL romio.patch -- RobL test-build.patch -- Dave test-source.patch -- Dave
I've already integrated the dcmfd patch and am working on integrating the datatype and ignore patches. Whoever has looked at the above patches, can you spend some time to integrate these into the trunk? From the discussion with IBM, the debugger patch is delayed for now and a part of the test suite patches are not being integrated as well. Everything else should go in.
Once the build patch is in, we can start testing the stack on BG/P, so that will probably be the critical requirement for this integration.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
These are related to the Aint is not void* in size. With the introduction of MPIR_Pint, these should simply use the explicit cast rather than the now unnecessary macro (that macro can have only one expansion, so there's no need to define it). The rules are In the patch, replace MPI_AINT_CAST_TO_VOID_PTR with (void*)(MPIR_Puint) replace MPI_VOID_PTR_CAST_TO_MPI_AINT with (MPIR_Puint) And do not use MPI_AINT_CAST_TO_LONG_LONG (architecture-specific hack for printing). Bill On Jul 8, 2008, at 10:21 PM, Pavan Balaji wrote:
Hi Bill,
Some parts of the build.patch don't seem to have been committed. For example, MPI_AINT_CAST_TO_VOID_PTR doesn't seem to be in. Was there a change in this part?
-- Pavan
On 07/01/2008 03:50 PM, William Gropp wrote:
Yes, I committed it to the trunk.
Bill
On Jun 29, 2008, at 12:04 AM, Pavan Balaji wrote:
Hi Bill,
Did you already commit this patch into the trunk? If you are still working on it, can you send out your current (temporarily workable) version of the patch? We can use it to compile the patches we are working on and check for errors.
Thanks.
-- Pavan
William Gropp wrote:
I'm in the final stages of testing the changes to build.patch . I've attached an item-by-item summary.
Bill
------------------------------------------------------------------- -----
On Jun 23, 2008, at 8:12 PM, Pavan Balaji wrote:
Hi all,
Now that the discussion with IBM is done, we should work on integrating the BG patches into the MPICH2 trunk.
build.patch -- Bill datatype.patch -- Pavan dcmfd.patch -- Pavan debugger.patch -- Darius ignore.patch -- Pavan mpe.patch -- Anthony other.patch -- Rajeev romio-bgl.patch -- RobL romio.patch -- RobL test-build.patch -- Dave test-source.patch -- Dave
I've already integrated the dcmfd patch and am working on integrating the datatype and ignore patches. Whoever has looked at the above patches, can you spend some time to integrate these into the trunk? From the discussion with IBM, the debugger patch is delayed for now and a part of the test suite patches are not being integrated as well. Everything else should go in.
Once the build patch is in, we can start testing the stack on BG/P, so that will probably be the critical requirement for this integration.
Thanks.
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
Bill,
replace MPI_VOID_PTR_CAST_TO_MPI_AINT with (MPIR_Puint)
Shouldn't this be (MPI_Aint)(MPIR_Puint)? Otherwise, something like: x = MPI_VOID_PTR_CAST_TO_MPI_AINT(y) where x is an MPI_Aint would throw a warning because of a mismatch in size. Do we need to run this by IBM since this would essentially throw out 90% of their patches? -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
On 07/17/2008 04:57 PM, Pavan Balaji wrote:
Do we need to run this by IBM since this would essentially throw out 90% of their patches?
BTW, I was talking to Joe and Doug and they were very impressed with the way our team has gone over their patches. So I would say they wouldn't object to throwing out 90% of their patches if it means getting it right. -d
BTW, I was talking to Joe and Doug and they were very impressed with the way our team has gone over their patches.
That's great. I hope their patches work though; we never actually tried them out :-).
So I would say they wouldn't object to throwing out 90% of their patches if it means getting it right.
Ok. I'm going to point this out to them and see what they say. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji
These really depend on the context - it depends on whether the Aint in question is really a integer-as-pointer or if it really is MPI_Aint. Bill On Jul 17, 2008, at 4:57 PM, Pavan Balaji wrote:
Bill,
replace MPI_VOID_PTR_CAST_TO_MPI_AINT with (MPIR_Puint)
Shouldn't this be (MPI_Aint)(MPIR_Puint)? Otherwise, something like:
x = MPI_VOID_PTR_CAST_TO_MPI_AINT(y)
where x is an MPI_Aint would throw a warning because of a mismatch in size.
Do we need to run this by IBM since this would essentially throw out 90% of their patches?
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
participants (10)
-
Darius Buntinas -
Dave Goodell -
Pavan Balaji -
Rajeev Thakur -
Rob Ross -
Robert Latham -
robl＠mcs.anl.gov
-
William Gropp -
William Gropp -
William Gropp