April 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] success and failure report for mpich-3.0.2
by Pavan Balaji 30 Apr '13

30 Apr '13

On 04/30/2013 05:55 AM US Central Time, Siegmar Gross wrote: > It seems, that I don't need a path, if the command has the same path > on both machines. It breaks, if the program has different pathnames. >From the launching logic, I don't know how that'll be true. I just tested this as well and it works fine for me. > sunpc1 fd1026 108 mpiexec -np 2 -host sunpc1,linpc1 hostname > sunpc1 > [proxy:0:1@linpc1] HYDU_create_process > (../../../../mpich-3.0.2/src/pm/hydra/utils/launch/launch.c:74): > execvp error on file hostname (No such file or directory) My guess is that "hostname" one of the machines is on your path and the other is not. > linpc1 fd1026 105 mpiexec -np 2 -host sunpc1,linpc1 hostname > linpc1 > sunpc1 Are all of /bin /usr/local/bin and /usr/bin in your path? -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] where is two-phase I/O implemented
by Pavan Balaji 30 Apr '13

30 Apr '13

Please don't call it mpich3; it's just "mpich" :-). -- Pavan On 04/30/2013 10:32 AM US Central Time, Matthieu Dorier wrote: > Hi, > > A question more for the mpich3 developers: > > If I were to implement an ADIO interface to a file system in mpich3, > would I have to implement optimizations like two-phase I/O myself or are > these optimizations implemented outside ADIO, independently of the file > system? > > Thanks, > > Matthieu Dorier > PhD student at ENS Cachan Brittany and IRISA > http://people.irisa.fr/Matthieu.Dorier > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] where is two-phase I/O implemented
by Dries Kimpe 30 Apr '13

30 Apr '13

* Matthieu Dorier <matthieu.dorier(a)irisa.fr> [2013-04-30 17:32:41]: > If I were to implement an ADIO interface to a file system in mpich3, > would I have to implement optimizations like two-phase I/O myself or are > these optimizations implemented outside ADIO, independently of the file > system? Hi Matthieu, The ADIO interface enables a filesystem driver to override this functionality, but typically it is 'inherited' by using the _generic_ variants. For example, for BGL: struct ADIOI_Fns_struct ADIO_BGL_operations = { ADIOI_BGL_Open, /* Open */ ADIOI_GEN_OpenColl, /* Collective open */ ADIOI_BGL_ReadContig, /* ReadContig */ ADIOI_BGL_WriteContig, /* WriteContig */ #if BGL_OPTIM_STEP1_2 ADIOI_BGL_ReadStridedColl, /* ReadStridedColl */ ADIOI_BGL_WriteStridedColl, /* WriteStridedColl */ #else ADIOI_GEN_ReadStridedColl, /* ReadStridedColl */ ADIOI_GEN_WriteStridedColl, /* WriteStridedColl */ #endif ADIOI_GEN_SeekIndividual, /* SeekIndividual */ ADIOI_BGL_Fcntl, /* Fcntl */ #if BGL_OPTIM_STEP1_1 ADIOI_BGL_SetInfo, /* SetInfo */ #else ADIOI_GEN_SetInfo, /* SetInfo */ #endif ADIOI_BGL_ReadStrided, /* ReadStrided */ ADIOI_BGL_WriteStrided, /* WriteStrided */ ADIOI_BGL_Close, /* Close */ #ifdef ROMIO_HAVE_WORKING_AIO #warning Consider BG support for NFS before enabling this. ADIOI_GEN_IreadContig, /* IreadContig */ ADIOI_GEN_IwriteContig, /* IwriteContig */ #else ADIOI_FAKE_IreadContig, /* IreadContig */ ADIOI_FAKE_IwriteContig, /* IwriteContig */ #endif ADIOI_GEN_IODone, /* ReadDone */ ADIOI_GEN_IODone, /* WriteDone */ ADIOI_GEN_IOComplete, /* ReadComplete */ ADIOI_GEN_IOComplete, /* WriteComplete */ ADIOI_GEN_IreadStrided, /* IreadStrided */ ADIOI_GEN_IwriteStrided, /* IwriteStrided */ ADIOI_BGL_Flush, /* Flush */ ADIOI_GEN_Resize, /* Resize */ ADIOI_GEN_Delete, /* Delete */ ADIOI_GEN_Feature, /* Features */ }; All of the _GEN_ functions are ADIO independent implementations. Hope this helps, Dries

1 0

Re: [mpich-discuss] where is two-phase I/O implemented
by Rajeev Thakur 30 Apr '13

30 Apr '13

You don't need to reimplement those optimizations unless you specifically want to change something in them. Rajeev On Apr 30, 2013, at 10:32 AM, Matthieu Dorier wrote: > Hi, > > A question more for the mpich3 developers: > > If I were to implement an ADIO interface to a file system in mpich3, would I have to implement optimizations like two-phase I/O myself or are these optimizations implemented outside ADIO, independently of the file system? > > Thanks, > > Matthieu Dorier > PhD student at ENS Cachan Brittany and IRISA > http://people.irisa.fr/Matthieu.Dorier > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

where is two-phase I/O implemented
by Matthieu Dorier 30 Apr '13

30 Apr '13

Hi, A question more for the mpich3 developers: If I were to implement an ADIO interface to a file system in mpich3, would I have to implement optimizations like two-phase I/O myself or are these optimizations implemented outside ADIO, independently of the file system? Thanks, Matthieu Dorier PhD student at ENS Cachan Brittany and IRISA http://people.irisa.fr/Matthieu.Dorier

1 0

Re: [mpich-discuss] get the process rank in the bash script
by Jeff Hammond 30 Apr '13

30 Apr '13

If you are on a Cray machine, why are you not using the appropriate launcher, aprun? You can find some info on parsing Cray physical topology on https://wiki.alcf.anl.gov/parts/index.php/Cray. For example, the physical node id, which may not correspond with what PMI gives you inside of a parallel job, can be read from /proc/cray_xt/nid on every node in the system, including both compute and service nodes. I can't tell exactly what you are doing, but if you are using CSCS, you might find CSCS support is able to better address your needs. In many cases, there is a "better way" to do things on a supercomputer than what one uses on traditional Linux clusters. Jeff On Mon, Apr 29, 2013 at 9:12 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > Hello, > > Adding discuss(a)mpich.org back to cc. > > There is a second alternative -- PMI_ID -- that you might want to try. > If that doesn't work, Cray might be using something different and you'll > need to check with them. > > -- Pavan > > On 04/29/2013 02:22 AM US Central Time, Kirill Lykov wrote: >> Thanks for the reply, I'm working on the Cray machine so the MPICH2 version for cray is defined by the following line: CRAY_MPICH2_VERSION=5.6.0 >> >> >> ________________________________________ >> From: Pavan Balaji [balaji(a)mcs.anl.gov] >> Sent: Monday, April 29, 2013 12:13 AM >> To: discuss(a)mpich.org >> Cc: Lykov Kirill >> Subject: Re: [mpich-discuss] get the process rank in the bash script >> >> You can use PMI_RANK. I just tried it and it works as expected. Which >> version of MPICH are you using? >> >> -- Pavan >> >> On 02/25/2013 10:06 AM US Central Time, Kirill Lykov wrote: >>> Dear mpich users, >>> >>> I'm looking for an environmental variable which can be used in a bash script to identify the process rank. >>> I launch a bash script using mpi: >>> mpirun ./my-script.sh >>> >>> In my-script I want to use the rank of the process to identify which input file to use. In OpenMPI, I used OMPI_COMM_WORLD_RANK and my-script.sh looked like the following: >>> >>> inputName="MyInput_$OMPI_COMM_WORLD_RANK"./myexe $inputName >>> >>> What can be used in MPICH instead of OMPI_COMM_WORLD_RANK? I found somewhere that $PMI_RANK is the same but I found that it is empty. >>> >>> Kirill Lykov, Mr. >>> >>> Doctoral Assistant >>> Institute of >>> Computational Science >>> Faculty of Informatics >>> Università della Svizzera italiana, USI >>> Via G. Buffi 13 >>> CH - 6904 Lugano >>> (e) kirill.lykov(a)usi.ch >>> (t) +41 (0) 788413392 >>> >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides

1 0

Re: [mpich-discuss] install + config on windows
by Jayesh Krishna 30 Apr '13

30 Apr '13

Hi, You can download MPICH2 binaries for Windows at http://www.mpich.org/downloads/ . You need to compile your MPI programs with MPICH2 to make it work. I would recommend recompiling your code after you install MPICH2 (If you have MPI program binaries pre-built with MPICH2 - instead of compiling them on your own - make sure that you install the same version of MPICH2 that was used to build the binaries). The wmpiregister program has a bug and you can ignore this error message ("...unable to query host"). Can you run your MPI program using mpiexec from a command prompt? Regards, Jayesh ----- Original Message ----- From: "spatiogis" <geo(a)spatiogis.fr> To: discuss(a)mpich.org Sent: Tuesday, April 30, 2013 9:26:35 AM Subject: [mpich-discuss] install + config on windows Hello, I'm not very good at computing, but I would like to install Mpich2 on windows 7 - 64 bits. There is only one pc, with one user plus the admin, and a simple core processor. I would like to know if it's mandatory to have compiling softwares with it to make it work, whereas it is asked in this case only to make run another software, and not for compiling (that would maybe save some disk space and simplify the installation) ? My second issue is that I must be missing something about the server configuration. I have installed Mpich from the .msi file, then configured the wmpiregister program with the Domain/user informations. There is this message displayed when trying to connect in the 'configurable settings' window : 'MPICH2 not installed or unable to query the host'. What is the host actually ? I know I am starting from very far, I am sorry for these very simple questions. Thanks if you can reply me, that would certainly save me some long hours of reading and testing ;) sincerely, Ben _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] install + config on windows
by Wesley Bland 30 Apr '13

30 Apr '13

Hi Ben, On Apr 30, 2013, at 9:26 AM, spatiogis <geo(a)spatiogis.fr> wrote: > Hello, > > I'm not very good at computing, but I would like to install Mpich2 on windows 7 - 64 bits. There is only one pc, with one user plus the admin, and a simple core processor. Which version of MPICH2 are you using? The last supported version of MPICH for windows was 1.4.1p1 available at http://www.mpich.org/downloads/. > > I would like to know if it's mandatory to have compiling softwares with it to make it work, whereas it is asked in this case only to make run another software, and not for compiling (that would maybe save some disk space and simplify the installation) ? I'm not the MPICH/Windows expert (actually, I don't think there is a Windows expert anymore), but I don't believe you need to compile anything to install the binary files. However, without a compiler, you most likely won't be able to do much with MPICH once you've installed it so you might end up needing to install one in the end. > > My second issue is that I must be missing something about the server configuration. I have installed Mpich from the .msi file, then configured the wmpiregister program with the Domain/user informations. > > There is this message displayed when trying to connect in the 'configurable settings' window : 'MPICH2 not installed or unable to query the host'. > > What is the host actually ? > > I know I am starting from very far, I am sorry for these very simple questions. Thanks if you can reply me, that would certainly save me some long hours of reading and testing ;) Unfortunately, you will probably have to do a bit of reading and testing to get things to work, but the Windows installation instructions are relatively short. If you look at the installation guide (http://www.mpich.org/static/downloads/1.4.1p1/mpich2-1.4.1p1-installguide.p…) the Windows instructions start on page 16. Try going through those and then let us know if you still have issues. Thanks, Wesley > > sincerely, > > Ben > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

install + config on windows
by spatiogis 30 Apr '13

30 Apr '13

Hello, I'm not very good at computing, but I would like to install Mpich2 on windows 7 - 64 bits. There is only one pc, with one user plus the admin, and a simple core processor. I would like to know if it's mandatory to have compiling softwares with it to make it work, whereas it is asked in this case only to make run another software, and not for compiling (that would maybe save some disk space and simplify the installation) ? My second issue is that I must be missing something about the server configuration. I have installed Mpich from the .msi file, then configured the wmpiregister program with the Domain/user informations. There is this message displayed when trying to connect in the 'configurable settings' window : 'MPICH2 not installed or unable to query the host'. What is the host actually ? I know I am starting from very far, I am sorry for these very simple questions. Thanks if you can reply me, that would certainly save me some long hours of reading and testing ;) sincerely, Ben

1 0

Re: [mpich-discuss] Bug in HYDT_dbg_setup_procdesc
by Pavan Balaji 30 Apr '13

30 Apr '13

Gah. Yes, good catch. I've fixed it here: http://git.mpich.org/mpich.git/commitdiff/a9640dc4 -- Pavan On 04/30/2013 04:59 AM US Central Time, Chris January wrote: > Hello, > > We (Allinea) have noticed a bug introduced in HYDT_dbg_setup_procdesc > between 3.0.2 and 3.0.3 caused by this commit: > > http://trac.mpich.org/projects/mpich/changeset/e04dd4b64ff618f2df58789265b7… > > When debugging a 4 process job on a 32-core machine using DDT we find > that the 4 entries in MPIR_Proctable all have the same pid. > > Here is how to reproduce the issue outside of DDT: > > jbray@mic3:31053% gdb --args mpirun -np 4 wave_f.exe > GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6) > Copyright (C) 2010 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-redhat-linux-gnu". > For bug reporting instructions, please see: > <http://www.gnu.org/software/gdb/bugs/>... > Reading symbols > from /home/jbray/prog/mpich/mpich-3.0.3/mic_gnu/install/bin/mpirun...done. > (gdb) break MPIR_Breakpoint > Breakpoint 1 at 0x428a70: file ./tools/debugger/debugger.c, line 25. > (gdb) r > Starting > program: /home/jbray/prog/mpich/mpich-3.0.3/mic_gnu/install/bin/mpirun > -np 4 wave_f.exe > [Thread debugging using libthread_db enabled] > Detaching after fork from child process 107606. > > Breakpoint 1, MPIR_Breakpoint () at ./tools/debugger/debugger.c:25 > 25 } > Missing separate debuginfos, use: debuginfo-install > glibc-2.12-1.107.el6.x86_64 libxml2-2.7.6-12.el6_4.1.x86_64 > zlib-1.2.3-29.el6.x86_64 > (gdb) print *MPIR_proctable@4 > $1 = {{host_name = 0x6723f0 "mic3", executable_name = 0x6723d0 > "./wave_f.exe", pid = 107847}, {host_name = 0x6723b0 "mic3", > executable_name = 0x672390 "./wave_f.exe", pid = 107847}, { > host_name = 0x672370 "mic3", executable_name = 0x672350 > "./wave_f.exe", pid = 107847}, {host_name = 0x672330 "mic3", > executable_name = 0x672310 "./wave_f.exe", pid = 107847}} > (gdb) > > As you can see MPIR_proctable claims each rank has the same pid, when in > reality they do not: > > -bash-4.1$ ps aux | grep 'wave_f.exe' > jbray 107841 0.3 0.0 99896 18772 pts/2 S+ 10:57 0:00 gdb > --args mpirun -np 4 ./wave_f.exe > jbray 107843 0.0 0.0 23144 1288 pts/2 T 10:57 > 0:00 /home/jbray/prog/mpich/mpich-3.0.3/mic_gnu/install/bin/mpirun -np > 4 ./wave_f.exe > jbray 107847 0.0 0.0 46488 1504 ? Ss 10:57 > 0:00 ./wave_f.exe > jbray 107848 0.0 0.0 29332 1472 ? Ss 10:57 > 0:00 ./wave_f.exe > jbray 107849 0.0 0.0 29332 1472 ? Ss 10:57 > 0:00 ./wave_f.exe > jbray 107850 0.0 0.0 29332 1472 ? Ss 10:57 > 0:00 ./wave_f.exe > cjanuary 107870 0.0 0.0 103244 864 pts/6 S+ 10:57 0:00 > grep ./wave_f.exe > > Regards, > Chris January - VP Engineering - Allinea Software Ltd. > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0