January 2014 - discuss - mailman3.cels.anl.gov

error recieving
by Soheil Hooshdaran 03 Jan '14

03 Jan '14

Hi. What's wrong with my code? 1. char send_buf[12] = "hello world"; 2. char send_buf2[12] = "hello iran"; 3. char recv_buf[10]; 4. int i; 5. for(i=0;i<10;++i) 6. if (world_rank == 0) { 7. MPI_Send(send_buf, 12, MPI_CHAR, 1, 0, MPI_COMM_WORLD); 8. //MPI_Send(send_buf2, 12, MPI_INT, 1, 1, MPI_COMM_WORLD); 9. } else if (world_rank == 1) { 10. MPI_Recv(recv_buf, 12, MPI_CHAR, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE); 11. printf("Process 1 received %s from process 0\n", recv_buf); 12. } Run-time errors are very hard to fix!Help me please Thanks in advance Soheil Hooshdaran

1 0

Re: [mpich-discuss] Potentially odd memory behavior in MPI_Waitall
by Kenneth Raffenetti 02 Jan '14

02 Jan '14

Can you provide a minimal demonstration of this behavior? That way we can work from the same reference point to identify possible causes. On 01/01/2014 03:05 PM, John Grime wrote: > Hi all, > > I was wondering if someone could help me out with trying to understand what’s gong on with an MPI_Waitall() call with regards to resident set memory usage. Platform information at the end of the email. > > If I run a 4 process MPI job on my desktop machine, I get a steady increase in memory usage over the course of the program. Apart from the places where I would reasonably expect expect memory usage to increase, I also see regular increases when I call MPI_Waitall(). > > I measure this behavior via: > > getmem( &rss0, &vs ); > > result = MPI_Waitall( n_buffer_reqs, buffer_reqs, MPI_STATUSES_IGNORE ); > if( result != MPI_SUCCESS ) ERROR_MACRO( "MPI_Waitall" ); > > getmem( &rss1, &vs ); > > … where getmem() queries the OS kernel to return the resident set size (“rss") and virtual memory size (“vs", ignored). The getmem() function does not allocate any memory itself, so I can hopefully see any difference in the resident set memory used before and after the MPI_Waitall() call via (rss1-rss0). > > After MPI_Waitall() returns, I very often see an increase in the resident set size ranging from 4096 bytes to 65536 bytes. > > Running an “identical” 4-process job on a single node of the Blue Waters compute cluster (Cray XE system, GCC 4.8.1, Cray mpich 6.0.1) does not seem to produce this behaviour. > > Switching the MPI_Waitall() for a loop over MPI_Waitany() calls also produces resident set increases. > > Does anyone have any comments or suggestions as to how I might understand whether this behavior is errant, and how I might track down the cause? It’s irritating me as I’d really like to produce code with as low a memory overhead as possible! > > Cheers, > > J. > > ------------------ > Platform info: > ------------------ > > OS: Apple OSX 10.9.1 > > mpicxx —version: > > Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn) > Target: x86_64-apple-darwin13.0.0 > Thread model: posix > > mpiexec —version > > HYDRA build details: > Version: 3.0.4 > Release Date: Wed Apr 24 10:08:10 CDT 2013 > CC: /usr/bin/clang -pipe -arch x86_64 -Wl,-headerpad_max_install_names -arch x86_64 > CXX: /usr/bin/clang++ -pipe -arch x86_64 -Wl,-headerpad_max_install_names -arch x86_64 > F77: /opt/local/bin/gfortran-mp-4.8 -pipe -m64 -Wl,-headerpad_max_install_names -arch x86_64 > F90: /opt/local/bin/gfortran-mp-4.8 -pipe -m64 -Wl,-headerpad_max_install_names -arch x86_64 > Configure options: '--disable-option-checking' '--prefix=/opt/local' '--disable-dependency-tracking' '--disable-silent-rules' '--enable-base-cache' '--enable-cache' '--enable-cxx' '--enable-fast=O2' '--enable-shared' '--enable-smpcoll' '--with-device=ch3:nemesis' '--with-pm=hydra' '--with-thread-package=posix' '--enable-versioning' 'F90FLAGS=' 'F90=' '--enable-timer-type=mach_absolute_time' '--libdir=/opt/local/lib/mpich-mp' '--sysconfdir=/opt/local/etc/mpich-mp' '--program-suffix=-mp' '--enable-f77' '--enable-fc' 'CC=/usr/bin/clang' 'CFLAGS=-pipe -arch x86_64 -O2' 'LDFLAGS=-Wl,-headerpad_max_install_names -arch x86_64 ' 'CXX=/usr/bin/clang++' 'CXXFLAGS=-pipe -arch x86_64 -O2' 'F77=/opt/local/bin/gfortran-mp-4.8' 'FFLAGS=-pipe -m64 -O2' 'FC=/opt/local/bin/gfortran-mp-4.8' 'FCFLAGS=-pipe -m64 -O2' '--cache-file=/dev/null' '--srcdir=.' 'LIBS=-lpthread ' 'CPPFLAGS= -I/opt/local/var/macports/build/_opt_mports_dports_science_mpich/mpich-default/work/mpich-3.0.4 /src/mpl/include -I/opt/local/var/macports/build/_opt_mports_dports_science_mpich/mpich-default/work/mpich-3.0.4/src/mpl/include -I/opt/local/var/macports/build/_opt_mports_dports_science_mpich/mpich-default/work/mpich-3.0.4/src/openpa/src -I/opt/local/var/macports/build/_opt_mports_dports_science_mpich/mpich-default/work/mpich-3.0.4/src/openpa/src -I/opt/local/var/macports/build/_opt_mports_dports_science_mpich/mpich-default/work/mpich-3.0.4/src/mpi/romio/include' > Process Manager: pmi > Launchers available: ssh rsh fork slurm ll lsf sge manual persist > Topology libraries available: hwloc > Resource management kernels available: user slurm ll lsf sge pbs cobalt > Checkpointing libraries available: > Demux engines available: poll select > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] Issue with mpiexec
by Lu, Huiwei 02 Jan '14

02 Jan '14

Hi, Konstantinos, Unfortunately we have stopped supporting Windows since a while ago due to the lack of developer resources. Currently, MPICH works mostly on Linux and Mac systems. http://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_Why_can… As an alternative, you can get help from Microsoft MPI. http://msdn.microsoft.com/en-us/library/bb524831%28v=vs.85%29.aspx — Huiwei Lu http://www.mcs.anl.gov/~huiweilu/ On Jan 2, 2014, at 1:53 PM, Konstantinos Karmas <karmkons(a)auth.gr> wrote: > > > Hello. My name is Konstantinos Karmas and i am an undergraduate student in the Electrical Engineering faculty of Aristotle Univercity of Thessaloniki. > I have an assignment to complete, regarding a program running in distributed systems and i need to install MPICH2 on my computer through cygwin, running windows 7 64bit. I followed the instructions in http://www.mpich.org/static/downloads/1.4.1/mpich2-1.4.1-README.txt , but when i try to use "mpiexec -n <number> ./examples/cpi" command, i am getting the following errors: > > mpiexec -n 2 ./examples/cpi > [mpiexec@Anachronoss] HYDT_topo_bind (./tools/topo/topo.c:557): no topology library available > [mpiexec@Anachronoss] HYDU_create_process (./utils/launch/launch.c:62): bind process failed > [mpiexec@Anachronoss] HYDT_bscd_common_launch_procs (./tools/bootstrap/external/external_common_launch.c:310): create process returned error > [mpiexec@Anachronoss] HYDT_bsci_launch_procs (./tools/bootstrap/src/bsci_launch.c:17): launcher returned error while launching processes > [mpiexec@Anachronoss] HYD_pmci_launch_procs (./pm/pmiserv/pmiserv_pmci.c:137): launcher cannot launch processes > [mpiexec@Anachronoss] main (./ui/mpich/mpiexec.c:401): process manager returned error launching processes > [mpiexec@Anachronoss] HYD_pmcd_pmiserv_proxy_init_cb (./pm/pmiserv/pmiserv_cb.c:520): assert (!closed) failed > [mpiexec@Anachronoss] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status > [mpiexec@Anachronoss] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:181): error waiting for event > [mpiexec@Anachronoss] main (./ui/mpich/mpiexec.c:405): process manager error waiting for completion > > I am including the c,config, m,mi and src/openpa/config files. Please can you help me solve this? > > Thanks in advance. > > Konstantinos Karmas > Aristotle Univercity of Thessaloniki > > > > > > <c.txt><config.log><m.txt><mi.txt><config.log>_______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Can't receive messages
by Jeff Hammond 02 Jan '14

02 Jan '14

> On Jan 2, 2014, at 1:19 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> Configure figures these out already and can write them to a file as part of installation. Are type sizes not encoded in datatypes #define anyways? > > You want me to detect where the mpich installation is? The process manager is separate from the mpich installation. It can be installed separately, and at a different location. It has no idea where mpich is installed, or even if it’s mpich that it is interacting with. Let us assume the common case where "which mpiexec" tells you where MPI is installed. > Also, you want me to write things to a file and read them from all over the place? Also, note that type sizes are not all. You need the datatype layout as well. Can this not be generated as part of the build process and thus be installed withe everything else. Does the MPICH ABI standardization effort not give you something to lean on w.r.t. datatype layout homogeneity? > Finally, suppose we run a program to generate this file. How do we ship it around? Scp? stdout? I'll say scp, but I don't have an opinion because I don't think that shipping a file between two hosts is that difficult compared to firing up MPI programs, which is what the user is ultimately trying to do. > This solution is gross beyond imagination. I’m not going to debate about it. I accept that you do not want to do it, but I don't accept the argument that it is impossible. Jeff -- Jeff Hammond jeff.science(a)gmail.com

1 0

Issue with mpiexec
by Konstantinos Karmas 02 Jan '14

02 Jan '14

Hello. My name is Konstantinos Karmas and i am an undergraduate student in the Electrical Engineering faculty of Aristotle Univercity of Thessaloniki. I have an assignment to complete, regarding a program running in distributed systems and i need to install MPICH2 on my computer through cygwin, running windows 7 64bit. I followed the instructions in http://www.mpich.org/static/downloads/1.4.1/mpich2-1.4.1-README.txt , but when i try to use "mpiexec -n <number> ./examples/cpi" command, i am getting the following errors: mpiexec -n 2 ./examples/cpi [mpiexec@Anachronoss] HYDT_topo_bind (./tools/topo/topo.c:557): no topology library available [mpiexec@Anachronoss] HYDU_create_process (./utils/launch/launch.c:62): bind process failed [mpiexec@Anachronoss] HYDT_bscd_common_launch_procs (./tools/bootstrap/external/external_common_launch.c:310): create process returned error [mpiexec@Anachronoss] HYDT_bsci_launch_procs (./tools/bootstrap/src/bsci_launch.c:17): launcher returned error while launching processes [mpiexec@Anachronoss] HYD_pmci_launch_procs (./pm/pmiserv/pmiserv_pmci.c:137): launcher cannot launch processes [mpiexec@Anachronoss] main (./ui/mpich/mpiexec.c:401): process manager returned error launching processes [mpiexec@Anachronoss] HYD_pmcd_pmiserv_proxy_init_cb (./pm/pmiserv/pmiserv_cb.c:520): assert (!closed) failed [mpiexec@Anachronoss] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status [mpiexec@Anachronoss] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:181): error waiting for event [mpiexec@Anachronoss] main (./ui/mpich/mpiexec.c:405): process manager error waiting for completion I am including the c,config, m,mi and src/openpa/config files. Please can you help me solve this? Thanks in advance. Konstantinos Karmas Aristotle Univercity of Thessaloniki

1 0

Re: [mpich-discuss] Can't receive messages
by Pavan Balaji 02 Jan '14

02 Jan '14

On Jan 2, 2014, at 1:19 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > Configure figures these out already and can write them to a file as part of installation. Are type sizes not encoded in datatypes #define anyways? You want me to detect where the mpich installation is? The process manager is separate from the mpich installation. It can be installed separately, and at a different location. It has no idea where mpich is installed, or even if it’s mpich that it is interacting with. Also, you want me to write things to a file and read them from all over the place? Also, note that type sizes are not all. You need the datatype layout as well. Finally, suppose we run a program to generate this file. How do we ship it around? Scp? stdout? This solution is gross beyond imagination. I’m not going to debate about it. — Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Can't receive messages
by Jeff Hammond 02 Jan '14

02 Jan '14

Configure figures these out already and can write them to a file as part of installation. Are type sizes not encoded in datatypes #define anyways? Jeff Sent from my iPhone > On Jan 2, 2014, at 1:15 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > > You need a C program to figure out the datatype layouts. And how do you pass the values upstream to the process that did the ssh? Through stdout? That’s not safe since ssh can throw out other spurious output. You’ll need to open your own connections and such. That is no longer a simple script. > > — Pavan > >> On Jan 2, 2014, at 7:32 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> >> What if the proxy app is just a shell script that parses the magic number file? >> >> Jeff >> >> Sent from my iPhone >> >>> On Jan 1, 2014, at 6:49 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >>> >>> >>> On Jan 1, 2014, at 6:30 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>>>> The only thing you can do is have mpiexec ssh to each node, and get the information needed. This is (1) not scalable, and (2) not viable unless we start distributing a program that gets the datatype information for all datatypes we care about and compare the bits inside mpiexec. >>>> >>>> Can't you do this in a tree rather than root-to-all? >>> >>> No. To do a tree, I need some proxy application that I need to run on the intermediate nodes, that will do the second-level ssh. This proxy application will need to be heterogeneous-safe if it needs to communicate with mpiexec. >>> >>> — Pavan >>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Can't receive messages
by Pavan Balaji 02 Jan '14

02 Jan '14

You need a C program to figure out the datatype layouts. And how do you pass the values upstream to the process that did the ssh? Through stdout? That’s not safe since ssh can throw out other spurious output. You’ll need to open your own connections and such. That is no longer a simple script. — Pavan On Jan 2, 2014, at 7:32 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > What if the proxy app is just a shell script that parses the magic number file? > > Jeff > > Sent from my iPhone > >> On Jan 1, 2014, at 6:49 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >> >> >> On Jan 1, 2014, at 6:30 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>>> The only thing you can do is have mpiexec ssh to each node, and get the information needed. This is (1) not scalable, and (2) not viable unless we start distributing a program that gets the datatype information for all datatypes we care about and compare the bits inside mpiexec. >>> >>> Can't you do this in a tree rather than root-to-all? >> >> No. To do a tree, I need some proxy application that I need to run on the intermediate nodes, that will do the second-level ssh. This proxy application will need to be heterogeneous-safe if it needs to communicate with mpiexec. >> >> — Pavan >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-discuss] Can't receive messages
by Jeff Hammond 02 Jan '14

02 Jan '14

What if the proxy app is just a shell script that parses the magic number file? Jeff Sent from my iPhone > On Jan 1, 2014, at 6:49 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > > On Jan 1, 2014, at 6:30 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>> The only thing you can do is have mpiexec ssh to each node, and get the information needed. This is (1) not scalable, and (2) not viable unless we start distributing a program that gets the datatype information for all datatypes we care about and compare the bits inside mpiexec. >> >> Can't you do this in a tree rather than root-to-all? > > No. To do a tree, I need some proxy application that I need to run on the intermediate nodes, that will do the second-level ssh. This proxy application will need to be heterogeneous-safe if it needs to communicate with mpiexec. > > — Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Can't receive messages
by Pavan Balaji 01 Jan '14

01 Jan '14

On Jan 1, 2014, at 6:30 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> The only thing you can do is have mpiexec ssh to each node, and get the information needed. This is (1) not scalable, and (2) not viable unless we start distributing a program that gets the datatype information for all datatypes we care about and compare the bits inside mpiexec. > > Can't you do this in a tree rather than root-to-all? No. To do a tree, I need some proxy application that I need to run on the intermediate nodes, that will do the second-level ssh. This proxy application will need to be heterogeneous-safe if it needs to communicate with mpiexec. — Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0