March 2013 - discuss - mailman3.cels.anl.gov

Semantics of asynchronous I/O operations in MPI3
by Matthieu Dorier 08 Mar '13

08 Mar '13

Hi, I can't manage to find a clear answer to this question in the MPI3 standard book; Assume we chain several asynchronous I/O operation on the same file, such as MPI_File_iwrite(...) MPI_File_set_view(...) MPI_File_iwrite(...) MPI_File_iwrite(...) MPI_Wait_all(all requests from previous calls) (equivalently with any other collective or independent asynchronous file accesses) Is an implementation of the MPI3 standard supposed to perform these operations in the same order than the functions are called, or can they be re-ordered if they are not in conflict (regions written are distinct)? Thanks! Matthieu Dorier PhD student at ENS Cachan Brittany and IRISA http://people.irisa.fr/Matthieu.Dorier

1 0

MPICH2-1.5 installation problems on Linux
by psj58285＠gmail 08 Mar '13

08 Mar '13

Dear Sir, I am trying to install MPICH2-1.5 on a Linux System. It succeeded at the configure step, but when I was building MPICH2, it failed. My information: baoys@altix-1:~> uname -a Linux altix-1 2.6.5-7.244-sn2 #1 SMP Mon Dec 12 18:32:25 UTC 2005 ia64 ia64 ia64 GNU/Linux baoys@altix-1:~> icc -v Version 11.1 baoys@altix-1:~> gcc -v Reading specs from /usr/lib/gcc-lib/ia64-suse-linux/3.3.3/specs Configured with: ../configure --enable-threads=posix --prefix=/usr --with-local-prefix=/usr/local --infodir=/usr/share/info --mandir=/usr/share/man --enable-languages=c,c++,f77,objc,java,ada --disable-checking --libdir=/usr/lib --enable-libgcj --with-gxx-include-dir=/usr/include/g++ --with-slibdir=/lib --with-system-zlib --enable-shared --enable-__cxa_atexit ia64-suse-linux Thread model: posix gcc version 3.3.3 (SuSE Linux) Here are the attachments, including the c.txt and the m.txt(generate by 'make VERBOSE=1 2>&1 | tee m.txt '). Looking forward to your help! Thanks, Sijie Pan psj58285@gmail

1 0

Re: [mpich-discuss] Hydra torque support issues
by Bharath Ramesh 07 Mar '13

07 Mar '13

On Thu, Mar 07, 2013 at 04:18:58PM -0600, Pavan Balaji wrote: > > This was already present in 1.5, though not enabled by default. > However, there were some bug fixes in 3.0, so using the latest version > is the best bet. Thanks. I will upgrade to latest 3.0 hydra launcher and see if it fixes the issue. [..] -- Bharath

1 0

Re: [mpich-discuss] Hydra torque support issues
by Pavan Balaji 07 Mar '13

07 Mar '13

This was already present in 1.5, though not enabled by default. However, there were some bug fixes in 3.0, so using the latest version is the best bet. -- Pavan On 03/07/2013 03:57 PM US Central Time, Dave Goodell wrote: > On Mar 7, 2013, at 3:51 PM CST, Bharath Ramesh <bramesh(a)vt.edu> wrote: > >> I am using mvapich2-1.9a2 which is based of mpich2-1.5. We have >> enabled Torque integration with hydra. We are noticing an issue >> where in Torque is not tracking the resource used by the MPI >> application when they are built with mvapich2. Further >> investigation revealed that mvapich2 hydra process launcher was >> not setting the correct session id to what torque used for >> forking the shell. I am wondering if this is a known issue or >> something that has already been fixed. If it has been fixed, what >> would be the best way to upgrade just hydra without affecting the >> rest of MPI stack. > > I can't speak to whether the issue is known or fixed (Pavan will know). But you can install a different version of Hydra from one of the release tarballs or nightly tarballs: > > http://www.mpich.org/downloads/ > http://www.mpich.org/static/tarballs/3.0.2/hydra-3.0.2.tar.gz > http://www.mpich.org/static/tarballs/nightly/hydra/ > > -Dave > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Hydra torque support issues
by Dave Goodell 07 Mar '13

07 Mar '13

On Mar 7, 2013, at 3:51 PM CST, Bharath Ramesh <bramesh(a)vt.edu> wrote: > I am using mvapich2-1.9a2 which is based of mpich2-1.5. We have > enabled Torque integration with hydra. We are noticing an issue > where in Torque is not tracking the resource used by the MPI > application when they are built with mvapich2. Further > investigation revealed that mvapich2 hydra process launcher was > not setting the correct session id to what torque used for > forking the shell. I am wondering if this is a known issue or > something that has already been fixed. If it has been fixed, what > would be the best way to upgrade just hydra without affecting the > rest of MPI stack. I can't speak to whether the issue is known or fixed (Pavan will know). But you can install a different version of Hydra from one of the release tarballs or nightly tarballs: http://www.mpich.org/downloads/ http://www.mpich.org/static/tarballs/3.0.2/hydra-3.0.2.tar.gz http://www.mpich.org/static/tarballs/nightly/hydra/ -Dave

1 0

Hydra torque support issues
by Bharath Ramesh 07 Mar '13

07 Mar '13

I am using mvapich2-1.9a2 which is based of mpich2-1.5. We have enabled Torque integration with hydra. We are noticing an issue where in Torque is not tracking the resource used by the MPI application when they are built with mvapich2. Further investigation revealed that mvapich2 hydra process launcher was not setting the correct session id to what torque used for forking the shell. I am wondering if this is a known issue or something that has already been fixed. If it has been fixed, what would be the best way to upgrade just hydra without affecting the rest of MPI stack. -- Bharath

1 0

Re: [mpich-discuss] MPI_Comm_Spawn causing zombies of hydra_pmi_proxy
by Pavan Balaji 07 Mar '13

07 Mar '13

I just tried this with the mpich master and it seems to work correctly, and there are no zombie processes (though I reduced the number of iterations to 10000, instead of 200000). This was a problem in mpich once upon a time, but that was a few years ago. Are you using the latest version of mpich (3.0.2)? -- Pavan On 03/07/2013 09:44 AM US Central Time, Silvan Brï¿½ndli wrote: > PS: The attached programs are a simplification of my code. They > reproduce the zombie problem. Waiting for the 32k zombies takes a > while... but I expect the same behaviour as with my original code. > > Am I missing something when finishing the called program? I just use > MPI_Comm_disconnect and MPI_Finalize. > > Best regards > Silvan > > main.cpp > > #include <mpi.h>; > > int main(int argc, char *argv[]) > { > int myrank; > int spawnerror; > int value = 123; > void *buf = &value; > MPI_Comm child_comm; > > if (MPI_Init(&argc,&argv)!=MPI_SUCCESS) > { > printf("MPI_Init failed"); > } > > MPI_Comm_rank(MPI_COMM_WORLD, &myrank); > > char* hiargv[] = {"23",NULL}; > for(int i = 1; i <= 200000; i++) > { > value = i; > printf("Main before spawn %d\n",i); > MPI_Comm_spawn("./hi",hiargv, 1, MPI_INFO_NULL, myrank, > MPI_COMM_SELF, &child_comm, &spawnerror); > MPI_Send(buf, 1, MPI_INTEGER, 0, 1, child_comm); > MPI_Comm_disconnect(&child_comm); > } > > MPI_Finalize(); > return 0; > } > > hi.cpp: > > #include <mpi.h>; > > int main(int argc, char** argv) { > MPI_Comm parent; > MPI_Status status; > int err; > int value = -1; > void* buf= &value; > > if (MPI_Init(&argc,&argv)!=MPI_SUCCESS) > { > printf("MPI_Init failed"); > } > MPI_Comm_get_parent(&parent); > if (parent == MPI_COMM_NULL) printf("No parent!"); > > MPI_Recv(buf, 1, MPI_INTEGER, 0, MPI_ANY_TAG, parent, &status); > MPI_Comm_disconnect(&parent); > err = MPI_Finalize(); > printf("hi finalized %d %d \n",err, value); > return 0; > } > > > > On 07.03.2013 12:38, Silvan Brï¿½ndli wrote: >> Dear all, >> >> again I have a question related to spawning processes. I understand the >> situation as follows: >> >> My program A spawns program B. Program B spawns program C1, C2 ... >> C10000 ... >> Program Cx terminates correctly before Cx+1 is called, however returning >> 1 to mpiexec. To handle this I use the workaround as described in >> http://lists.mpich.org/pipermail/discuss/2013-February/000429.html >> >> Now it looks like with every Spawn a "hydra_pmi_proxy" is started, the >> calling program is mpiexec. When the program Cx is finished this >> "hydra_pmi_proxy" remains as a zombie until the programs A, B and >> mpiexec are finished. When approx. 32k of those "hydra_pmi_proxy" exist >> I get some problems (too many processes or something similar). >> >> What can I do to finish "hydra_pmi_proxy" while my programs A, B and >> mpiexec are still running? >> >> I'm glad about every hint. >> >> Best regards >> Silvan >> > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] MPI_Comm_Spawn causing zombies of hydra_pmi_proxy
by Silvan Brändli 07 Mar '13

07 Mar '13

PS: The attached programs are a simplification of my code. They reproduce the zombie problem. Waiting for the 32k zombies takes a while... but I expect the same behaviour as with my original code. Am I missing something when finishing the called program? I just use MPI_Comm_disconnect and MPI_Finalize. Best regards Silvan main.cpp #include <mpi.h>; int main(int argc, char *argv[]) { int myrank; int spawnerror; int value = 123; void *buf = &value; MPI_Comm child_comm; if (MPI_Init(&argc,&argv)!=MPI_SUCCESS) { printf("MPI_Init failed"); } MPI_Comm_rank(MPI_COMM_WORLD, &myrank); char* hiargv[] = {"23",NULL}; for(int i = 1; i <= 200000; i++) { value = i; printf("Main before spawn %d\n",i); MPI_Comm_spawn("./hi",hiargv, 1, MPI_INFO_NULL, myrank, MPI_COMM_SELF, &child_comm, &spawnerror); MPI_Send(buf, 1, MPI_INTEGER, 0, 1, child_comm); MPI_Comm_disconnect(&child_comm); } MPI_Finalize(); return 0; } hi.cpp: #include <mpi.h>; int main(int argc, char** argv) { MPI_Comm parent; MPI_Status status; int err; int value = -1; void* buf= &value; if (MPI_Init(&argc,&argv)!=MPI_SUCCESS) { printf("MPI_Init failed"); } MPI_Comm_get_parent(&parent); if (parent == MPI_COMM_NULL) printf("No parent!"); MPI_Recv(buf, 1, MPI_INTEGER, 0, MPI_ANY_TAG, parent, &status); MPI_Comm_disconnect(&parent); err = MPI_Finalize(); printf("hi finalized %d %d \n",err, value); return 0; } On 07.03.2013 12:38, Silvan Brï¿½ndli wrote: > Dear all, > > again I have a question related to spawning processes. I understand the > situation as follows: > > My program A spawns program B. Program B spawns program C1, C2 ... > C10000 ... > Program Cx terminates correctly before Cx+1 is called, however returning > 1 to mpiexec. To handle this I use the workaround as described in > http://lists.mpich.org/pipermail/discuss/2013-February/000429.html > > Now it looks like with every Spawn a "hydra_pmi_proxy" is started, the > calling program is mpiexec. When the program Cx is finished this > "hydra_pmi_proxy" remains as a zombie until the programs A, B and > mpiexec are finished. When approx. 32k of those "hydra_pmi_proxy" exist > I get some problems (too many processes or something similar). > > What can I do to finish "hydra_pmi_proxy" while my programs A, B and > mpiexec are still running? > > I'm glad about every hint. > > Best regards > Silvan > -- Dipl.-Ing. Silvan Brï¿½ndli Numerische Strukturanalyse mit Anwendungen in der Schiffstechnik (M-10) Technische Universitï¿½t Hamburg-Harburg Schwarzenbergstraï¿½e 95c 21073 Hamburg Tel. : +49 (0)40 42878 - 6187 Fax. : +49 (0)40 42878 - 6090 e-mail: silvan.braendli(a)tuhh.de www : http://www.tuhh.de/skf 5th GACM Colloquium on Computational Mechanics http://www.tu-harburg.de/gacm2013

1 0

MPI_Comm_Spawn causing zombies of hydra_pmi_proxy
by Silvan Brändli 07 Mar '13

07 Mar '13

Dear all, again I have a question related to spawning processes. I understand the situation as follows: My program A spawns program B. Program B spawns program C1, C2 ... C10000 ... Program Cx terminates correctly before Cx+1 is called, however returning 1 to mpiexec. To handle this I use the workaround as described in http://lists.mpich.org/pipermail/discuss/2013-February/000429.html Now it looks like with every Spawn a "hydra_pmi_proxy" is started, the calling program is mpiexec. When the program Cx is finished this "hydra_pmi_proxy" remains as a zombie until the programs A, B and mpiexec are finished. When approx. 32k of those "hydra_pmi_proxy" exist I get some problems (too many processes or something similar). What can I do to finish "hydra_pmi_proxy" while my programs A, B and mpiexec are still running? I'm glad about every hint. Best regards Silvan -- Dipl.-Ing. Silvan Brï¿½ndli Numerische Strukturanalyse mit Anwendungen in der Schiffstechnik (M-10) Technische Universitï¿½t Hamburg-Harburg Schwarzenbergstraï¿½e 95c 21073 Hamburg Tel. : +49 (0)40 42878 - 6187 Fax. : +49 (0)40 42878 - 6090 e-mail: silvan.braendli(a)tuhh.de www : http://www.tuhh.de/skf 5th GACM Colloquium on Computational Mechanics http://www.tu-harburg.de/gacm2013

1 0

Help with eclipse juno and mpich2
by George Papageorgakis 06 Mar '13

06 Mar '13

Hello people! I am new in mpi. I have been trying to setup eclipse to play with mpich2 in windows 7. I have installed mpich2 and managed to make the eclipse editor identify mpi libs, functions etc but it wont compile correctly. I am using the example of mpi_heat2D and i would like to run it on a ubuntu cluster of my university. I have done so with a kinda slow and weary methodology using WinSCP to copy the .c file from my pc to a cluster pc, compile it and run it via console not being able to see any graphs etc, only the output file with the results. I would like to know what can i do to run and see everything from eclipse, including graphs for optimization etc. I didnt find much help on the internet and i am kinda frustrated. Is what i ask possible? Thanks in advance!

1 0