April 2013 - discuss - mailman3.cels.anl.gov

mpich - windows
by Richard Muddle 18 Apr '13

18 Apr '13

Hi, Windows installers for mpich haven't been release since 2.1.4.1p1 - is Windows supported for newer versions? Apart from Microsoft and Intel, are there any other alternatives for Windows? Thank you, Richard

1 0

mpich2.1.4.1p1: intercomm_create.c
by Richard Muddle 18 Apr '13

18 Apr '13

In src/mpi/comm/intercomm_create.c from lines 58, the calculation of maxi seems to be incorrect when maxlpid is a multiple of 32. /* Compute the max index and zero the pids array */ maxi = (maxlpid + 31) / 32; if (maxi >= MAX_LPID32_ARRAY) { MPIU_CHKLMEM_MALLOC(lpidmask,int32_t*,maxi*sizeof(int32_t), mpi_errno,"lpidmask"); } else { lpidmask = lpidmaskPrealloc; } for (i=0; i<maxi; i++) lpidmask[i] = 0;

1 0

Re: [mpich-discuss] Hydra torque support issues
by Bharath Ramesh 18 Apr '13

18 Apr '13

On 4/18/2013 7:04 PM, Pavan Balaji wrote: > Bharath, > > Can you send the output of "mpiexec -info"? mpiexec -info HYDRA build details: Version: 3.0.2 Release Date: Wed Feb 27 19:12:19 EST 2013 CC: gcc -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 CXX: g++ -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 F77: gfortran -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 F90: gfortran Configure options: '--disable-option-checking' '--prefix=/opt/apps/gcc4_5/mvapich2/1.9b' '--with-ib-libpath=/opt/ofed/ofed/lib64/' '--with-ib-include=/opt/ofed/ofed/include/' '--with-pbs=/opt/torque/torque' '--enable-fast=O3' '--enable-shared' 'CC=gcc' 'CFLAGS= -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 -O3' 'CXX=g++' 'CXXFLAGS= -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 -O3' 'F77=gfortran' 'FFLAGS=-L/opt/ofed/ofed/lib64/ -L/lib -L/lib -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 -O3' 'FC=gfortran' '--enable-embedded-mode' '--cache-file=/dev/null' '--srcdir=.' 'LDFLAGS=-L/opt/ofed/ofed/lib64/ -L/lib -L/lib -L/lib -Wl,-rpath,/lib -L/lib -Wl,-rpath,/lib -L/lib -L/lib' 'LIBS=-libmad -lrdmacm -libumad -libverbs -ldl -lrt -lm -lpthread ' 'CPPFLAGS=-I/opt/ofed/ofed/include/ -I/opt/build/BUILD/mvapich2-1.9b/src/mpl/include -I/opt/build/BUILD/mvapich2-1.9b/src/mpl/include -I/opt/build/BUILD/mvapich2-1.9b/src/openpa/src -I/opt/build/BUILD/mvapich2-1.9b/src/openpa/src -I/opt/build/BUILD/mvapich2-1.9b/src/mpi/romio/include -I/include -I/include -I/include -I/include' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge pbs manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs Checkpointing libraries available: Demux engines available: poll select > I'm particularly interested in whether mpiexec recognizes pbs as a > possible launcher or not. If it recognizes pbs as a launcher, it should > automatically select it where possible. > It is using pbs as the launcher. The problem is with the setting of the session id for the forked hydra_pmi_proxy process on mother superior. Since the hydra_pmi_proxy process has the session id set to the bash process instead mpi_exec process. As a result of this torque is unable to track the resources consumed by the process. If you look at the openmpi processes their session id is set correctly and torque is able to track the resources consumed. -- Bharath

1 0

Re: [mpich-discuss] Hydra torque support issues
by Pavan Balaji 18 Apr '13

18 Apr '13

Bharath, Can you send the output of "mpiexec -info"? I'm particularly interested in whether mpiexec recognizes pbs as a possible launcher or not. If it recognizes pbs as a launcher, it should automatically select it where possible. -- Pavan On 04/17/2013 02:21 PM US Central Time, Bharath Ramesh wrote: > On 4/17/2013 3:10 PM, Brock Palen wrote: >> Bharath, >> >> (sorry on road real quick) >> >> I do this with hydra+mvapich with Matlab and it does work. >> >> In my case I didn't find how to tell hydra when building mpich/mvapich >> to tell it where libtorque.so lives. >> >> To resolve this I download hydra alone and build it giving explicit >> options to it to find the torque library. Then you get the expected >> behavior. I don't have the directions for how I did this handy. > Thanks for the pointer, I will dig into this to see if I can get hydra > to work correctly. Currently I do pass the option > --with-pbs=/path/to/torque to the mvapich2 configure line. > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Hydra torque support issues
by Bharath Ramesh 17 Apr '13

17 Apr '13

On 4/17/2013 3:10 PM, Brock Palen wrote: > Bharath, > > (sorry on road real quick) > > I do this with hydra+mvapich with Matlab and it does work. > > In my case I didn't find how to tell hydra when building mpich/mvapich to tell it where libtorque.so lives. > > To resolve this I download hydra alone and build it giving explicit options to it to find the torque library. Then you get the expected behavior. I don't have the directions for how I did this handy. Thanks for the pointer, I will dig into this to see if I can get hydra to work correctly. Currently I do pass the option --with-pbs=/path/to/torque to the mvapich2 configure line. -- Bharath

1 0

Re: [mpich-discuss] Hydra torque support issues
by Brock Palen 17 Apr '13

17 Apr '13

Bharath, (sorry on road real quick) I do this with hydra+mvapich with Matlab and it does work. In my case I didn't find how to tell hydra when building mpich/mvapich to tell it where libtorque.so lives. To resolve this I download hydra alone and build it giving explicit options to it to find the torque library. Then you get the expected behavior. I don't have the directions for how I did this handy. Brock Palen www.umich.edu/~brockp CAEN Advanced Computing brockp(a)umich.edu (734)936-1985 On Apr 17, 2013, at 1:04 PM, Bharath Ramesh <bramesh(a)vt.edu> wrote: > On Thu, Mar 07, 2013 at 04:18:58PM -0600, Pavan Balaji wrote: >> >> This was already present in 1.5, though not enabled by default. >> However, there were some bug fixes in 3.0, so using the latest version >> is the best bet. >> >> -- Pavan >> >> On 03/07/2013 03:57 PM US Central Time, Dave Goodell wrote: >>> On Mar 7, 2013, at 3:51 PM CST, Bharath Ramesh <bramesh(a)vt.edu> wrote: >>> >>>> I am using mvapich2-1.9a2 which is based of mpich2-1.5. We have >>>> enabled Torque integration with hydra. We are noticing an issue >>>> where in Torque is not tracking the resource used by the MPI >>>> application when they are built with mvapich2. Further >>>> investigation revealed that mvapich2 hydra process launcher was >>>> not setting the correct session id to what torque used for >>>> forking the shell. I am wondering if this is a known issue or >>>> something that has already been fixed. If it has been fixed, what >>>> would be the best way to upgrade just hydra without affecting the >>>> rest of MPI stack. >>> >>> I can't speak to whether the issue is known or fixed (Pavan will know). But you can install a different version of Hydra from one of the release tarballs or nightly tarballs: > > Sorry about a delayed response. I installed mvapich2-1.9b based > on mpich-3.0.2 and I can say that the issue still exists. To > better understand I am attaching the output of ps axf with > relevant portions to differentiate between the behavior of hydra > launcher when compared to OpenMPI which does the correct thing. > This allows torque to track the resources used. > > -- > Bharath > <mpi_ps_axf.out>_______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Hydra torque support issues
by Bharath Ramesh 17 Apr '13

17 Apr '13

On Thu, Mar 07, 2013 at 04:18:58PM -0600, Pavan Balaji wrote: > > This was already present in 1.5, though not enabled by default. > However, there were some bug fixes in 3.0, so using the latest version > is the best bet. > > -- Pavan > > On 03/07/2013 03:57 PM US Central Time, Dave Goodell wrote: > > On Mar 7, 2013, at 3:51 PM CST, Bharath Ramesh <bramesh(a)vt.edu> wrote: > > > >> I am using mvapich2-1.9a2 which is based of mpich2-1.5. We have > >> enabled Torque integration with hydra. We are noticing an issue > >> where in Torque is not tracking the resource used by the MPI > >> application when they are built with mvapich2. Further > >> investigation revealed that mvapich2 hydra process launcher was > >> not setting the correct session id to what torque used for > >> forking the shell. I am wondering if this is a known issue or > >> something that has already been fixed. If it has been fixed, what > >> would be the best way to upgrade just hydra without affecting the > >> rest of MPI stack. > > > > I can't speak to whether the issue is known or fixed (Pavan will know). But you can install a different version of Hydra from one of the release tarballs or nightly tarballs: Sorry about a delayed response. I installed mvapich2-1.9b based on mpich-3.0.2 and I can say that the issue still exists. To better understand I am attaching the output of ps axf with relevant portions to differentiate between the behavior of hydra launcher when compared to OpenMPI which does the correct thing. This allows torque to track the resources used. -- Bharath

1 0

Re: [mpich-discuss] mpich-3.0.3 ARMCI compile warnings
by Jim Dinan 17 Apr '13

17 Apr '13

FYI, this was fixed in [89fb23ebb2]. ~Jim. On 4/9/13 8:43 PM, Jim Dinan wrote: > Hi Jeff, > > Thanks for the bug report. I created a ticket for this; should be an > easy fix. > > https://trac.mpich.org/projects/mpich/ticket/1807 > > ~Jim. > > On 04/09/2013 06:09 PM, Daily, Jeff A wrote: >> On NERSCï¿½s hopper I get the following warnings when compiling the ARMCI >> sources using the default module environment. >> >> >> >> CC parmci.lo >> >> PGC-W-0119-void function ARMCI_Barrier cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 52) >> >> PGC-W-0119-void function ARMCI_Fence cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 57) >> >> PGC-W-0119-void function ARMCI_AllFence cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 62) >> >> PGC-W-0119-void function ARMCI_Access_begin cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 67) >> >> PGC-W-0119-void function ARMCI_Access_end cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 72) >> >> PGC-W-0119-void function ARMCI_Lock cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 262) >> >> PGC-W-0119-void function ARMCI_Unlock cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 267) >> >> PGC-W-0119-void function armci_msg_barrier cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 277) >> >> PGC-W-0119-void function armci_msg_group_barrier cannot return value >> (../mpich-3.0.3/src/armci/src/parmci.c: 282) >> >> PGC/x86-64 Linux 12.9-0: compilation completed with warnings >> >> >> >> Looks like the when the profiling interface was generated, all functions >> were assumed to return a value. I hope itï¿½s an easy fix. >> > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] Problem in Building MPICH
by Pavan Balaji 14 Apr '13

14 Apr '13

Hi Ali, [please keep discuss(a)mpich.org cc'ed] >From the logs, it looks like configure is detecting blcr fine. My guess is that something is broken in your blcr installation. In any case, if you don't need it, disabling it should be sufficient. -- Pavan On 04/13/2013 03:56 PM US Central Time, Ali Behjatian wrote: > Hi Paven, > > Here is the config.log file. > > Thanks > Ali > > > On Sat, Apr 13, 2013 at 10:53 AM, Ali Behjatian <ali.behjatian(a)gmail.com > <mailto:[email protected]>> wrote: > > Oh sorry about that. I am out now for few hours. I will send it to > you once I get home. > > Thanks > Ali > > On Apr 13, 2013 10:44 AM, "Pavan Balaji" <balaji(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > Hi Ali, > > We need the config.log file from src/pm/hydra. > > Thanks, > > -- Pavan > > On 04/13/2013 10:40 AM US Central Time, Ali Behjatian wrote: > > First of all, thank you very much for your support. I was able > to build > > MPICH by adding "--without-hydra-ckpointlib" > > to the configure line. Thus I have attached 2 config.log files > for you. > > One without using the above option (config.log) > > and another one after using that. > > > > Ali > > > > > > On Sat, Apr 13, 2013 at 9:30 AM, Pavan Balaji > <balaji(a)mcs.anl.gov <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > > > Can you also send us the config.log from src/pm/hydra? > Hydra should be > > checking for blcr support in configure before enabling it. > > > > -- Pavan > > > > On 04/12/2013 05:44 PM US Central Time, Dave Goodell wrote: > > > Sorry, looks like I gave you the wrong configure option. > Try > > "--without-hydra-ckpointlib". > > > > > > -Dave > > > > > > On Apr 12, 2013, at 5:19 PM CDT, Ali Behjatian > > <ali.behjatian(a)gmail.com <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>> wrote: > > > > > >> I think I got the same error again. > > >> > > >> Thanks > > >> Ali > > >> > > >> > > >> On Fri, Apr 12, 2013 at 4:37 PM, Dave Goodell > > <goodell(a)mcs.anl.gov <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > >> On Apr 12, 2013, at 4:30 PM CDT, Ali Behjatian > > <ali.behjatian(a)gmail.com <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>> wrote: > > >> > > >>> I tried to follow the instructions in the MPICH2 > Installer's > > Guide but > > >>> I received an error after running "make 2>&1 | tee > m.txt". I > > appreciate > > >>> if you help me to fix this problem. > > >> > > >> Try adding "--without-blcr" on your configure line. It > looks > > like hydra, our configure tests, and your BLCR > installation don't > > agree on something. Ideally we would/will fix the > configure test, > > though this is the easiest way for you to get a working MPICH. > > >> > > >> -Dave > > >> > > >> _______________________________________________ > > >> discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > > >> To manage subscription options or unsubscribe: > > >> https://lists.mpich.org/mailman/listinfo/discuss > > >> > > >> > <c.txt><m.txt>_______________________________________________ > > >> discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > > >> To manage subscription options or unsubscribe: > > >> https://lists.mpich.org/mailman/listinfo/discuss > > > > > > _______________________________________________ > > > discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > > > To manage subscription options or unsubscribe: > > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > -- > > Pavan Balaji > > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Problem in Building MPICH
by Pavan Balaji 13 Apr '13

13 Apr '13

Hi Ali, We need the config.log file from src/pm/hydra. Thanks, -- Pavan On 04/13/2013 10:40 AM US Central Time, Ali Behjatian wrote: > First of all, thank you very much for your support. I was able to build > MPICH by adding "--without-hydra-ckpointlib" > to the configure line. Thus I have attached 2 config.log files for you. > One without using the above option (config.log) > and another one after using that. > > Ali > > > On Sat, Apr 13, 2013 at 9:30 AM, Pavan Balaji <balaji(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > > Can you also send us the config.log from src/pm/hydra? Hydra should be > checking for blcr support in configure before enabling it. > > -- Pavan > > On 04/12/2013 05:44 PM US Central Time, Dave Goodell wrote: > > Sorry, looks like I gave you the wrong configure option. Try > "--without-hydra-ckpointlib". > > > > -Dave > > > > On Apr 12, 2013, at 5:19 PM CDT, Ali Behjatian > <ali.behjatian(a)gmail.com <mailto:[email protected]>> wrote: > > > >> I think I got the same error again. > >> > >> Thanks > >> Ali > >> > >> > >> On Fri, Apr 12, 2013 at 4:37 PM, Dave Goodell > <goodell(a)mcs.anl.gov <mailto:[email protected]>> wrote: > >> On Apr 12, 2013, at 4:30 PM CDT, Ali Behjatian > <ali.behjatian(a)gmail.com <mailto:[email protected]>> wrote: > >> > >>> I tried to follow the instructions in the MPICH2 Installer's > Guide but > >>> I received an error after running "make 2>&1 | tee m.txt". I > appreciate > >>> if you help me to fix this problem. > >> > >> Try adding "--without-blcr" on your configure line. It looks > like hydra, our configure tests, and your BLCR installation don't > agree on something. Ideally we would/will fix the configure test, > though this is the easiest way for you to get a working MPICH. > >> > >> -Dave > >> > >> _______________________________________________ > >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > >> > >> <c.txt><m.txt>_______________________________________________ > >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > >> To manage subscription options or unsubscribe: > >> https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0