August 2007 - Mpich2-core - mailman3.cels.anl.gov

MPID_Atomic functions
by Dave Goodell 16 Sep '07

16 Sep '07

I'm currently working on the "Generic and portable atomics" bullet for the mpich2 1.1 release. Since this exposes a new API to our users, it seems best to run this change by all of you mpich-core@ folks who have quite a bit more mpich context than I do. Nemesis currently contains several routines for performing atomic operations on a limited set of platforms. They are defined in "src/mpid/ch3/channels/nemesis/nemesis/include/mpid_nem-atomics.h". Their function signatures are given below: static inline void *MPID_NEM_SWAP (volatile void *ptr, void *val) static inline void *MPID_NEM_CAS (volatile void *ptr, void *oldv, void *newv) static inline int MPID_NEM_CAS_INT (volatile int *ptr, int oldv, int newv) static inline int MPID_NEM_FETCH_AND_ADD (volatile int *ptr, int val) static inline void MPID_NEM_ATOMIC_ADD (int *ptr, int val) static inline int MPID_NEM_FETCH_AND_INC (volatile int *ptr) static inline int MPID_NEM_FETCH_AND_DEC (volatile int *ptr) static inline void MPID_NEM_ATOMIC_INC (volatile int *ptr) static inline void MPID_NEM_ATOMIC_DEC (volatile int *ptr) The following similarly named routines already exist in mpich2: % grep -ri mpi._atomic_ /sandbox/mpi/src/mpich2/src | perl -ne 's/^.*(mpi._atomic\w+).*$/$1/i and print' | sort | uniq MPID_Atomic_decr_flag MPID_Atomic_fetch_and_incr MPID_Atomic_incr MPID_ATOMIC_SET_IF_ZERO MPID_Atomic_write_barrier I propose that we extend the Nemesis functions to provide implementations for additional platforms as well as a slower fallback implemented via locking. We would then make these improved functions available to the rest of MPI as the following routines: static inline void *MPID_Atomic_swap (volatile void *ptr, void *val) static inline void *MPID_Atomic_cas (volatile void *ptr, void *oldv, void *newv) static inline int MPID_Atomic_cas_int (volatile int *ptr, int oldv, int newv) static inline int MPID_Atomic_fetch_and_add (volatile int *ptr, int val) static inline void MPID_Atomic_add (int *ptr, int val) + static inline int MPID_Atomic_fetch_and_incr (volatile int *ptr) static inline int MPID_Atomic_fetch_and_decr (volatile int *ptr) + static inline void MPID_Atomic_incr (volatile int *ptr) static inline void MPID_Atomic_decr (volatile int *ptr) The functions prepended with a '+' already exist as cpp macros and will be replaced by the Nemesis versions. The two existing uses of MPID_Atomic_fetch_and_decr will be altered to take the old value as a normal return value instead of as a "parameter return" value. Question: do we have any reason to believe that any MPICH derivatives might be using this macro as it currently exists? And if so, do we need to maintain backwards compatibility? These functions would all be defined in "src/include/mpiatomic.h" in order to make them available to the entirety of the mpich code. The original Nemesis versions would eventually be removed and all uses would be replaced by the new routines. I would appreciate any comments you might have, since I am not very familiar with the history of the existing MPID_Atomic_* routines. Thanks, -Dave

5 12

Known issues with the MPICH2-1.0.6-rc
by Pavan Balaji 15 Aug '07

15 Aug '07

It looks like the RC might finally be out tomorrow assuming that we don't see any surprises in the nightly tests tonight. I've compiled a list of issues which we are not dealing with for this RC and need to be fixed eventually. The list is attached with this email. If I've missed something, please let me know -- we don't want to lose track of anything! Thanks. -- Pavan -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji MPICH2-1.0.6-RC1 ---------------- 1. Abort test fails with Sock/MPD. 2. Gforker drops parts of stdout once in a while. 3. MPI_Ssend_Overtake tests in the Intel test suite fail on 2.4 kernels due to the poll overhead on these kernels. 4. SPARC SHM tests with more processes than processors (2 on atlantis) fail. This is likely because the OS is not able to schedule processes too well during over-subscription. 5. Datatype performance tests for packing arrays larger than 2 dimensions fail. It looks like this is because the compiler can optimize the application code more easily as compared to the MPICH2 code, where all packing is broken down into 1-D packs. 6. Threaded alltoall fails in SMPD.

3 7

Re: new 32-bit BB image
by Pavan Balaji 15 Aug '07

15 Aug '07

Thanks Dave. CC'ing others so that they can use it as well. -- Pavan On 08/15/2007 02:50 PM, Dave Goodell wrote: > I've finally got a 32-bit image working on the breadboard cluster. Go > ahead and use 'ubuntu-32-bit' for any 32-bit testing that you need to do. > > -Dave > -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji

1 0

mpich2-1.0.6-rc1
by Pavan Balaji 10 Aug '07

10 Aug '07

Hi all, The plan is to create the RC tarball on Monday, do some basic testing and get the RC out on Tuesday. Here are the outstanding bugs that need to be fixed and their current assignments: 1. Intel MPI_Ssend tests with SSM using the Intel compiler -- Darius. 2. Broadcast and sendcancel tests with SSM using shared libraries -- Pavan. 3. Performance tests on SHM with the Intel compiler -- Pavan. 4. Failed spawn tests with --enable-threads=multiple -- Darius. 5. SMPD thread tests -- Jayesh. 6. Failed F90 tests on Nemesis with the Intel compiler -- Unassigned. 7. gforker F90 tests on SPARC -- Unassigned. Bill -- will you be able to take a look at (6) and (7) above? These are probably related to the F90 fixes that you did earlier. Thanks. -- Pavan -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji

6 15

SunStudio 12
by Pavan Balaji 09 Aug '07

09 Aug '07

I've installed the Sun Studio 12 compiler on atlantis. You can use it by adding /sandbox/software/sun/studio12/install/bin to your path. The current Sun Studio series has deprecated -xarch=v9 and replaced it with -m64. -- Pavan -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji

1 0

Interns Farewell Party
by Pavan Balaji 09 Aug '07

09 Aug '07

All, Some of the interns in our group (Ganesh, Harish and Seung) are finishing up their internships in the next 2 weeks starting tomorrow. As per our early tradition, we are planning to go out for lunch tomorrow to some nice place (not decided yet, but suggestions are welcome). Hope you all can join us. -- Pavan -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji

1 0

Nightly tests
by Pavan Balaji 09 Aug '07

09 Aug '07

A large number of nightly tests had failed last night. I took at look at them; almost all of them have failed due to some other large application running on the same machine causing the test to take longer than usual and eventually timeout. The SMPD concurrent_spawns and alltoall tests and the MPICH aborttest are probably the only exceptions (I'm not considering the SPARC tests right now, at least till I get the newer compilers in place). We should try to shift to the breadboard machines using Dave's script for the release as soon as possible. It's getting impossible to track progress based on the current nightly test results. -- Pavan -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji

1 0

Dry run of the MPICH2 SVN
by Pavan Balaji 08 Aug '07

08 Aug '07

I've created a dry-run of the MPICH2 SVN to make sure the basic conversion errors are caught before we make the actual switch. The sample layout is available here: https://gingerchord.homeip.net/svn/projects/mpi. The login/password are mpich2-core/mpich2-core. Can you do a quick verification of the packages you are familiar with and make sure there are no big errors. The romio package is called "mpich2" because of a typo in my script; that's fixed now and will be correct in the final version. FYI, here are the packages I added in: mpich2-01 --> mpi/mpich2 mpe_thread/src --> mpi/mpich2/src/util/thread mpe_thread/test --> mpi/mpich2/test/util/thread confdb --> mpi/mpich2/confdb adi3 --> mpi/mpich2/doc/adi3 mpich2-coding --> mpi/mpich2/doc/mpich2 mpich2-notes --> mpi/mpich2/doc/notes romio --> mpi/mpich2/src/mpi/romio mpe2 --> mpi/mpich2/src/mpe2 slog2sdk --> mpi/mpich2/src/mpe2/src Thanks. -- Pavan -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji

1 0

Re: test script configurations
by Pavan Balaji 03 Aug '07

03 Aug '07

Thanks Dave. I'm CC'ing the others in the group for their information. > The test script is in good working order right now in SVN[1]. In its > current form it just tests an empty set of configure options on the > 'default' image, but it does run all four tests (make testing, Intel, OSU, > NPB). I need to assemble a list of options that you would like to test. > Here's what I am currently going to add: > > --enable-g=all > --enable-fast > --enable-debuginfo > --enable-nmpi-as-mpi > --enable-mpe > --enable-sharedlibs=gcc > --disable-sharedlibs > --enable-alloca > --enable-timer-type=clock_gettime > --enable-timer-type=gettimeofday > --enable-timer-type=linux86_cycle Rajeev/Bill, are there any other options that need to be tested? > --with-device=ch3 > --with-device=ch3:ssm > --with-device=ch3:shm > --with-device=ch3:nemesis --with-device=ch3:dllchan=sock --with-device=ch3:dllchan=ssm ... --with-device=ch3:dllchan=sock,ssm,shm > As for compilers, what other compilers do you usually use? On Linux I'm > really only directly familiar with gcc, so I'm not sure what else to use. > Is there an intel compiler, maybe something else as well? Where can I > find the other compilers? Other than gcc, Intel compilers are all we have on the breadboard machines. > I'm working on adding non-mpd process manager support right now. > Currently the tests only use mpd. You mentioned 'gforker' before, are > there any others that should be tested (maybe smpd)? gforker definitely. For windows, we need smpd (which probably the script doesn't handle right now). For linux, I was mentioning to Rajeev that we should probably drop it. Maybe he can confirm. > [1] https://svn.mcs.anl.gov/repos/radix/projects/mpi/release_testing/ Thanks. -- Pavan -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji

4 4

Removing irrelevant modules during the SVN switch
by Pavan Balaji 02 Aug '07

02 Aug '07

Should we remove some of the irrelevant (unsupported) modules once we shift to the SVN (removing directories is a lot cleaner in SVN)? I see the following modules that can be removed: src/mpid/mm src/mpid/rdma src/mpid/ch3/channels/gasnet src/mpid/ch3/channels/ib Note that when someone checks out an older version from the SVN repository, these will still be available. They'll just not be present at the head of the trunk. Btw, does shm provide something ssm doesn't? -- Pavan -- Pavan Balaji, Mathematics and Computer Science, Argonne National Laboratory Ph: 630.252.3017 http://www.mcs.anl.gov/~balaji

2 1