March 2011 - Mpich2-core - mailman3.cels.anl.gov

1.4
by Rajeev Thakur 07 Apr '11

07 Apr '11

I noticed last night that the nightly tests with Intel compilers on vanquish and octagon were not completing properly (no new output on the web page). The tests were also running the 1.4.x branch. I switched to the 1.4 branch and restarted the tests, but there were still some problems with Intel compilers. I have again started the tests. Let's hold off on releasing 1.4 until we figure out what is happening. I think the tests were fine until Monday. Rajeev

2 6

Intel compiler too slow; license server issue?
by Rajeev Thakur 01 Apr '11

01 Apr '11

The MPICH2 nightly tests with the Intel compilers version 12 (on vanquish and octagon) are taking forever to build. I believe it is a problem with the license server. If I try to compile the simple program attached below with icc a few times, sometimes it compiles instantly, and sometimes it takes a few minutes. Rajeev #include <stdio.h> main() { printf("hello world\n"); }

2 1

1.4rc1 release
by Pavan Balaji 28 Mar '11

28 Mar '11

I've cut a branch for 1.4 and tagged 1.4rc1. I'm planning to push out the release on Monday. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

2 1

NNSA programming models meeting
by Pavan Balaji 23 Mar '11

23 Mar '11

I'm sitting in the NNSA exascale programming models meeting. There seems to be a lot of confusion on what role MPI has to play at Exascale. I'm clarifying many of the myths that are floating around for MPI and I think I have made sure that people take MPI seriously (that's really the only serious contender right now, apart from OpenMP). But it might be a good idea for us to put together a small talk "Debunking the MPI at Exascale Myths", that many of us can use while talking to program managers or even random talks that we give. I've attached some of the myths that came up in this meeting, and my comments on it. Would folks be able to contribute some slides for this? Please add other myths you are aware of as well. Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

5 11

Test suite update?
by William Gropp 23 Mar '11

23 Mar '11

I've added some tests to the test suite (both in the trunk and 1.4.x) but they're not showing up in the nightlies - one of them should fail with the current code, and I've searched the summary files and they are not being run. The file in /home/MPI/testing/mpich2/mpich2 claims to be version 1.4rc1 , but I've verified that this doesn't include the new tests . Until those tests run, I won't know whether the C++ changes are acceptable to the compilers in our test environment. Also, this doesn't appear to have the new C++ or Fortran90 binding code, so that isn't being testing either (and that may change the ABI for C++ and Fortran90, so tests are essential). Am I checking code into the wrong place? Bill William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

3 6

next release
by Jayesh Krishna 22 Mar '11

22 Mar '11

When is the next release of MPICH2 ? We need couple of fixes that are already in the trunk (1. Correct handling of user defined signal handlers 2. Occassional hydra UI failure with Intrepid test cases) for Parvis development. I don't want parvis developers to move to OpenMPI due to these bugs. -Jayesh

2 1

Re: [mpich2-core] Outstanding merges
by Pavan Balaji 21 Mar '11

21 Mar '11

[moving this thread to mpich2-core] Dave and I had some discussion about this. This breaks ABI, but it is also a required correctness change (we are not MPI-2.2 if we don't have this). Here are our thoughts: 1. Drop the 1.3.3rc2 release and cancel 1.3.3. 2. Start a branch called 1.4.x copied from 1.3.x. This will have a new ABI version and can incorporate these new changes. 1.3.x is revert back the ABI breaking changes and will have the ABI string "2:x:0". 1.4.x will have the ABI string "3:x:0". 3. Move trunk to track 1.5.x. 4. Release 1.4rc1 immediately in place for 1.3.3rc2. Does this work for everyone? -- Pavan On 03/09/2011 10:10 AM, Pavan Balaji wrote: > > Cc'ing Dave as well, as he knows the ABI compatibility issue better than me. > > -- Pavan > > On 03/09/2011 08:11 AM, William Gropp wrote: >> I don't follow. r7822 only corrects bugs; for example, without the >> definition of MPI_DIST_GRAPH, it is incorrect for MPI_SUBVERSION to be >> 2, and it will have to be changed back to 1. Ditto for the changes to >> correctly handle logical. >> >> Bill >> >> On Mar 9, 2011, at 1:47 AM, Anthony Chan wrote: >> >>> >>> Another option is to exclude the change of distributed topology >>> functions >>> in r8135 from 1.3.x, but this would involve more works. >>> >>> A.Chan >>> >>> ----- Original Message ----- >>>> Hi Bill, >>>> >>>> The reason that r7822 isn't in 1.3.x because it is a patch that >>>> changes 1.3 series's ABI compatibility. Since r8135 depends on >>>> r7822, should it not be checked into 1.3.x as well ? >>>> >>>> A.Chan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

7 40

Fwd: [Mpi-forum] Significant discussion point with Ticket 269
by Dave Goodell 17 Mar '11

17 Mar '11

What is the group's general feeling on ticket #269 (additional "w" collectives)? I really dislike them and feel that it is unlikely that anyone will substantially optimize their implementation or that users will ever use them. Plus they add more collectives with non-scalable argument lists. And then there's the displacement unit inconsistency that Adam raises in the email thread below, which will just forever irritate everyone who touches the new interface. We should vote "no" on this ticket, IMO. -Dave Begin forwarded message: > From: Fab Tillier <ftillier(a)microsoft.com> > Date: March 17, 2011 4:30:23 PM CDT > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] Significant discussion point with Ticket 269 > Reply-To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > > Adam T. Moody wrote on Thu, 17 Mar 2011 at 12:43:49 > >> Fab Tillier wrote: >> >>> Hi Adam, >>> >>> -----Original Message----- From: mpi-forum-bounces(a)lists.mpi-forum.org >>> [mailto:mpi-forum- bounces(a)lists.mpi-forum.org] On Behalf Of Adam T. >>> Moody Sent: Wednesday, March 16, 2011 5:47 PM >>> >>>> The displacements in the new w functions are not semantically the same >>>> as in Alltoallw. See p 146, lines 41-44 vs p 169, line 26. In >>>> alltoallw, the displacements are given in number of bytes from the >>>> start of the buffer, but for the new functions, they are given as an >>>> integer number of extents of the corresponding datatype. >>> >>> The displacement parameters are identical in definition to those of the 'V' >>> variants of those functions, only the datatype (recvtype and/or sendtype) >>> was turned to an array and pluralized. I had missed the difference between >>> MPI_Alltoallv and MPI_Alltoallw, thanks for catching that. >>> >>>> First, we should be consistent with alltoallw or otherwise use a >>>> suffix other than 'w'. >>> >>> I don't know that we can succeed in achieving our goals of allowing users to >>> use large datatypes if we change to byte counts without going to a different >>> (larger) datatype - an 'int' may not be enough displacement for a single >>> datatype. If we are to change the suffix and use different parameters, let's >>> not mess around with using integer counts and just go to MPI_Count too, not >>> just the displacements. If we consider going to MPI_Count for count >>> parameters, we might as well just stick with a MPI_Count version of the 'v' >>> functions and avoid the 'w' all together. >> >> For the sake of discussion, let's go with 'x', and then like you >> suggest, we could create an 'x' set by extending all of the 'v' >> collectives to take MPI_Count instead of int. I think that could work. >> >>>> Second, specifying the displacement in terms of datatype extents >>>> restricts some codes from using these functions. For example, in >>>> gatherw, what if the root needs to receive a 2-byte datatype starting >>>> at byte position 1 in its receive buffer? I don't think this can be >>>> done under the current definition. >>> >>> Can't a user resize the datatype and set a LB so that the received >>> datatype is offset by 1 byte? >>> >> You're right. That would work. The down side is that this may require >> a user to create one datatype for each process, where I'm guessing most >> apps can use the same datatype (or a small set of different datatypes) >> in many cases but only change the displacement. In this case, it would >> be possible, but it's ugly. > > Having the displacement in alltoallw in bytes but specified as an int, especially since it's always relative to sendbuf, is just terrible. If displacements are going to be integers, I strongly believe they cannot be in units of bytes and still be useful, especially since the main motivation for introducing these new collectives is to allow large counts to be encapsulated in derived datatypes. The bottom line is that the inconsistency between alltoallw and the new 'w' functions is a result of alltoallw being poorly defined. We shouldn't define the new 'w' functions as badly just for consistency's sake. > >>>> Third, if we decide to define the displacement in bytes, as alltoallw >>>> does, then we should really think about making the displacement >>>> datatype an MPI_Aint, or perhaps even MPI_Count. Using an int >>>> datatype will lead to other problems. >>> >>> If we change these new functions to use MPI_Aint, then we should really >>> create a new version of Alltoallw to match. There are other functions that >>> use the wrong 'type' for parameters so we need to be careful about 'the >>> slippery slope'. >> >> Yes, I agree. Of these ideas so far, I'm leaning toward the 'x' >> extension of the 'v' collectives, but I need to think about it more. > > My goal with this ticket was to get minimal "complete" functionality for large derived types. My hope was always that it would be replaced before MPI3 is finalized with something better, but driven by *somebody else*. If it didn't get replaced, at least the standard would provide full (though cumbersome) functionality. > > -Fab > >>> Oy, the baggage we have to deal with to preserve back compat... >>> >>> -Fab >>> >>>> -Adam >>>> _______________________________________________ >>>> mpi-forum mailing list >>>> mpi-forum(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

2 1

Re: [mpich2-core] Outstanding merges
by Anthony Chan 10 Mar '11

10 Mar '11

Should related lines be removed in /mcs/web/research/projects/mpich2/nightly/cron/random/ignore.txt A.Chan ----- Original Message ----- > ----- Original Message ----- > > > Anthony: can you merge your pending changes into this? > > I've just merged r7822-r7826 (the ones related to distributed > graph functions) as r8198 to 1.4.x. Will merge the rest tomorrow. > > A.Chan

1 0

Re: [mpich2-core] MPI3-RMA routines update
by William Gropp 09 Mar '11

09 Mar '11

Kalyana, My guess is that creating a branch is the best approach; then we can review it before merging it. I've cc'ed the core group for their suggestions. We have an update to the copyright, so it should be ok. Bill On Mar 3, 2011, at 4:19 PM, Kalyana Chadalavada wrote: > Bill, > > Before I start writing code for the MPI3-RMA routines, I wanted to > check with you about the right way to update the code. > > Should I do a code checkout, update locally and check in the code? > If yes, what is the correct repository? Darius gave me access to > https://svn.mcs.anl.gov/repos/mpi/mpich2/branches/dev/lapi/ > > I cant seem to be able to access any other location under "dev" > > Also, for the code I add, what needs to be added in the source files > as per UIUC/NCSA legal requirements? > > Ex: copyright, disclaimer, contact etc > > I am making some working changes to a local code copy of MPICH2 to > try out my thoughts, I will update the wiki page with the model > shortly. > > Regards, > Kalyana Chadalavada > William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

4 8