Let's see if I have understood this right:
1. IBM promotes Aint to 64 bits and does the
appropriate type casts.
2. Pavan suggests using MPI_Offset internally for
datatypes and I/O instead
of Aint. Return MPI_Aint to the user, as defined by
MPI, possibly causing
truncation.
3. Bill suggests using an MPIR_Size_t internally to
represent message and
I/O lengths in the datatype code.
I think Pavan and Bill's suggestions are similar,
except that Bill wants to
use the bigger size in fewer places -- only where
calculations occur.
I am also attaching Rob's original proposal to IBM,
where he outlined two
approaches. The first one suggests using the
dataloop code for I/O and using
MPI_Offset to hold integer values. This is similar
in spirit to Pavan and
Bill's. The second one is to bump up MPI_Aint size.
We originally
recommended to IBM that they do the first solution,
but then changed our
mind after a month thinking it would be too much
work, and recommended the
Aint option. IBM also independently evaluated the
two approaches and was in
favor of Aint.
What would be the least amount of work and also do
the job right?
Suppose we went with the MPI_Aint approach, but not
made it the default.
Instead it has to be selected explicitly at
configure time by those who need
it. (These are not those who would use MPI_Aint i
instead of int *i in their
attribute code.) Most users will not even see the
change, and on 64-bit
systems it is never an issue. Those users on 32-bit
systems needing large
files can be told to configure it. (Last I heard,
IBM was also thinking of
providing two versions of their library.)
Would that work?
Rajeev
________________________________
Sent:
Wednesday, May 07, 2008 11:41 AM
Subject:
Re: [mpich2-core] BG/P patches
We're
starting to go around in the same circle. Here's a summary:
MPI Spec
issues:
The MPI
spec defines type extents as MPI_Aint ; some other fields
are ints.
File
offsets are in terms of MPI_Offset , which may be bigger (or
smaller) than an MPI_Aint
It is
possible to create MPI datatypes bigger than what can be
represented in an address-sized integer; this can
be relevant to datatypes
used with I/O.
Users
expect the size of a pointer to be exactly the size of an
MPI_Aint. If this is not true, some operations with
attributes will fail
(cause memory/stack overwrites)
MPICH2
Implementation issues:
Internal
implementation of datatypes uses MPI_Aint; there may be
performance reasons for that on some platforms
(instead of always using
MPI_Offset)
Blue Gene
issues:
MPI_Offset
is larger than a pointer. Some I/O libraries (HDF5, as
Rob L confirms) may need extents that are greater
than 32 bits.
The
problem is that there is no solution that simultaneously
satisfies every one of these requirements. This is
key; *any* solution is
going to have some warts. So, what can we do, and
what principles should we
follow?
One
principle that should have priority is the integrity of the test
suites. For example, I would not make any changes
to the attribute tests,
unless they're clearly wrong (not just ambiguous).
This limits the ability
to replace MPI_Aint with an integer that is longer
than a pointer in MPI
(this is not the same as changing the use of
MPI_Aint internally to MPICH2).
Another
principle is to avoid mandating something that might
significantly impact performance. This is why I'm
leery of mandating
MPI_Offset in the datatype code, at least for now.
Not all processors
support 64 bit integer operations with the same
speed as 32 bit operations,
though that may change in the next few years.
One thing
that can be done for the datatype code *internals* is to
use an MPIR_Size_t type; this can be typedef'ed
appropriately. This would
permit us to distinguish between what MPI defines
as an Aint, what MPI
defines as an Offset, and what MPICH2 wants to use
internally for message
lengths and datatype operations. The tradeoff might
be that for some systems
with 32 bit Aints and 64bit Offsets, some file
operations could not be
expressed with datatypes.
For the
HDF5 problem, the cleanest solution might be to propose an
MPI 2.2 extension and then implement it.
Bill
On May 7,
2008, at 9:41 AM, Pavan Balaji wrote:
I
understand the problem. What I'm suggesting is that
whatever has to be
returned
to the user can be an overflown value. But
internally we can
store the
correct value.
If the
user needs this returned value for allocating a
buffer, etc., then
that's a
problem, but that part is really a problem in the
MPI standard,
and not in
the MPICH2 implementation.
--
Pavan
On Wed, 7
May 2008, Rob Ross wrote:
Right. We
need to be able to return the correct
extent. I/O libraries/
applications need this capability, as
Bill says. --
Rob
On May 7,
2008, at 9:27 AM, William Gropp wrote:
This is
exactly the problem - the
application might need an extent
that won't
fit in an MPI_Aint . Right now,
we don't have a good
solution
for those applications.
Bill
On May 7,
2008, at 8:58 AM, Pavan Balaji
wrote:
I agree
with Rajeev's plan. One
thing to bear in mind (its in the
mails
but might
not jump out) is that some
I/O applications may need an
MPI_Offset
sized datatype extent
(HDF5 might be in this group).
Just to
clarify my original
suggestion -- I think all internal
extents
and
sizes
should be stored as
MPI_Offset. If the user requests for the
extent,
we can
always return an MPI_Aint
(ignoring the overflow).
--
Pavan
--
Pavan
Balaji
William
Gropp
Paul and
Cynthia Saylor Professor of
Computer Science
University
of Illinois Urbana-Champaign
--
Pavan
Balaji
William
Gropp
Paul and
Cynthia Saylor Professor of Computer Science
University
of Illinois Urbana-Champaign
<bgp-mpi-io-plan.txt>