June 2015 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPICH issue when running jobs in background
by Kenneth Raffenetti 09 Jun '15

09 Jun '15

Hi, I created bug 2274 for this issue. http://trac.mpich.org/projects/mpich/ticket/2274 Ken On 06/08/2015 11:04 PM, Roy, Hirak wrote: > Dear MPICH team, > > Waiting for your response. > > Thanks, > > Hirak > > *From:*Roy, Hirak > *Sent:* Tuesday, June 02, 2015 1:08 PM > *To:* discuss(a)mpich.org > *Subject:* MPICH issue when running jobs in background > > Hi Team, > > I am using MPICH 3.0.4. > > My top level script invokes a ‘batch script’ in background and then runs > my mpi-program using mpiexec. > > I run this top-level script in background mode ( please check details > later). > > It seems once I invoke the mpiexec, my batch-script execution is suspended. > > I tried OpenMPI and they do not have the issue. > > I have attached 3 files. > > 1>run.sh : run script to run the top-level program > > 2>background.sh : invoked from run.sh as a background job. Just prints > something on a while loop > > 3>myprog.c : simple MPI program which just waits on a loop > > Compile step : mpicc myprog.c à a.out > > #! /bin/bash > > ./background.sh & > > mpiexec -n 1 ./a.out > > To reproduce : > > *Shell >> ./run.sh &* > > *Shell >> jobs* > > *[1] + Suspended (tty input) ./run.sh* > > Here the background.sh is suspended. However, a.out keeps running. > > If I change the script so that mpiexec takes input from /dev/null, then > the issue is resolved. > > #! /bin/bash > > ./background.sh & > > mpiexec -n 1 ./a.out *< /dev/null* > > It seems to be a MPICH specific issue. > > Could you please have a look and file a bug for this. > > Thanks, > > Hirak > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Efficiently handling both MPI and TCP comms?
by Dorier, Matthieu 09 Jun '15

09 Jun '15

Hi, I have a master/worker-type program where communications between the master and the worker go through MPI, while TCP is used for the user to communicate with the master. So I need the master to wait for the results of the workers using MPI_Recv (or post non-blocking recieves and wait on the request) and wait for commands from the user through the TCP. Every solutions I can come up with involve setting the TCP socket to non-bvlocking and enter a busy loop (either internal to MPI if I use MPI generalized requests, or external if I alternate MPI_Test_any and try reading on the socket). The only other solution I can imagine uses threads (1 thread blocking on MPI_Recv, 1 thread blocking on a TCP read). Any other potential solutions? Thanks, Matthieu ________________________________ Matthieu Dorier, PhD Postdoctoral Appointee Argonne National Laboratory, MCS 9700 S Cass Ave, Lemont, IL 60439 +1 630 252 1515 _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH issue when running jobs in background
by Roy, Hirak 08 Jun '15

08 Jun '15

Dear MPICH team, Waiting for your response. Thanks, Hirak From: Roy, Hirak Sent: Tuesday, June 02, 2015 1:08 PM To: discuss(a)mpich.org Subject: MPICH issue when running jobs in background Hi Team, I am using MPICH 3.0.4. My top level script invokes a 'batch script' in background and then runs my mpi-program using mpiexec. I run this top-level script in background mode ( please check details later). It seems once I invoke the mpiexec, my batch-script execution is suspended. I tried OpenMPI and they do not have the issue. I have attached 3 files. 1> run.sh : run script to run the top-level program 2> background.sh : invoked from run.sh as a background job. Just prints something on a while loop 3> myprog.c : simple MPI program which just waits on a loop Compile step : mpicc myprog.c --> a.out #! /bin/bash ./background.sh & mpiexec -n 1 ./a.out To reproduce : Shell >> ./run.sh & Shell >> jobs [1] + Suspended (tty input) ./run.sh Here the background.sh is suspended. However, a.out keeps running. If I change the script so that mpiexec takes input from /dev/null, then the issue is resolved. #! /bin/bash ./background.sh & mpiexec -n 1 ./a.out < /dev/null It seems to be a MPICH specific issue. Could you please have a look and file a bug for this. Thanks, Hirak _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Automake version in MPICH 3.1.4
by Balaji, Pavan 07 Jun '15

07 Jun '15

Hi, We needed some fixes in automake-1.15, so the upgrade was not arbitrary. Having said that, that only introduces a dependency for developers. Users who download the tarball shouldn't need automake (or any of the autotools). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji From: Aurélien Larcher Reply-To: "discuss(a)mpich.org<mailto:[email protected]>" Date: Sunday, June 7, 2015 at 11:46 AM To: "<discuss(a)mpich.org<mailto:[email protected]>>" Subject: [mpich-discuss] Automake version in MPICH 3.1.4 Hi, I just noticed that the following commit on 2015-02-09: http://git.mpich.org/mpich.git/commit/8ac6607db3a087cc8986d8124cab2b0930ae3… adds automake >= 1.15 requirement in autogen.sh Is this fix really necessary for general platforms or is it platform specific ? Does it not seem a bit too drastic to force a dependency on a core package released on 2015-01-05 (a month before or so) just to get a fix which is not mpich dependent ? Thank you in avance ! Best regards Aurélien -- ------------------------------------------------------------------------------- LARCHER Aurélien | KTH, School of Computer Science and Communication Work: +46 (0) 8 790 71 42 | Lindstedtsvägen 5, Plan 4, 100 44 Stockholm, SWEDEN ------------------------------------------------------------------------------- _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Automake version in MPICH 3.1.4
by Aurélien Larcher 07 Jun '15

07 Jun '15

Hi, I just noticed that the following commit on 2015-02-09: http://git.mpich.org/mpich.git/commit/8ac6607db3a087cc8986d8124cab2b0930ae3… adds automake >= 1.15 requirement in autogen.sh Is this fix really necessary for general platforms or is it platform specific ? Does it not seem a bit too drastic to force a dependency on a core package released on 2015-01-05 (a month before or so) just to get a fix which is not mpich dependent ? Thank you in avance ! Best regards Aurélien -- ------------------------------------------------------------------------------- LARCHER Aurélien | KTH, School of Computer Science and Communication Work: +46 (0) 8 790 71 42 | Lindstedtsvägen 5, Plan 4, 100 44 Stockholm, SWEDEN ------------------------------------------------------------------------------- _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPICH 3.2b3 packages for Ubuntu, Debian and Slackware (x86 and amd64)
by Marcelo Soares Souza 04 Jun '15

04 Jun '15

mpich-3.2b3 packages * Ubuntu 14.10 - amd64 http://www.cebacad.net/files/mpich/ubuntu/mpich-3.2b3/mpich_3.2b3-1ubuntu_a… - x86 http://www.cebacad.net/files/mpich/ubuntu/mpich-3.2b3/mpich_3.2b3-1ubuntu_i… * Debian 8.0 (Jessie) - amd64 http://www.cebacad.net/files/mpich/debian/mpich-3.2b3/mpich_3.2b3-1debian_a… - x86 http://www.cebacad.net/files/mpich/debian/mpich-3.2b3/mpich_3.2b3-1debian_i… * Slackware 14.1 - amd64 http://www.cebacad.net/files/mpich/slackware/mpich-3.2b3/mpich-3.2b3-x86_64… - x86 http://www.cebacad.net/files/mpich/slackware/mpich-3.2b3/mpich-3.2b3-i486-g… Enjoy! -- Marcelo Soares Souza https://marcelo.libertais.org/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Totalview and Mpich3
by Peter Thompson 04 Jun '15

04 Jun '15

This is a summary of my reply to the person who posted the original query. I'd fallen off the list when my email address changed and the old address was no longer forwarded. But I was in touch and got the user running so I thought I'd pass on what I found. The original post reported problems starting TotalView with mpirun -tv This method appears to be broken currently. Looking at the 3.1.4 manual it looks like the documentation suggests other ways of starting TotalView. It has totalview -a mpirun -a -n 3 ./foo but then suggests you can use an 'indirect' launch with totalview mpirun -a -n 3 ./foo The first method is just wrong and won't work. The -a indicates that anything after the -a is an argument for the target program being debugged. This example has two '-a' with the first one before mpirun, so we don't see a program to debug. I think the behavior is undefined. The second method should work, but it does not invoke the 'indirect method'. Instead it is the 'classic' or 'direct' method. It will start up TotalView on mpirun, and when the job starts, it will attach to the processes that are launched. This method is tried and true, and it is scalable. I prefer a variation of this method which is totalview -args mpirun -np 3 ./foo. It's the same functionality, but you don't have to remember to place the -a after mpirun, and it allows one to get the mpirun command set up correctly, and then prepend totalview -args to it. The indirect method works as well, but is not scalable as it launches debug servers for each process rather than for each node. To start that one could simply type totalview or maybe totalview foo The first method will bring up the session manager, and you would choose to start a new parallel session. It will show a screen where you can pick the parallel starter. For MPICH 3.1, you could pick either MPICH2 or MPICH3, and they should work. MPICH will not, as that goes back to the original MPICH of the 1.2 era. Then pick the number of tasks. You may have to add the additional arguments of -f hostfile to run across different nodes, but I don't think setting the number of nodes really makes a difference. If you use 'totalview foo' the method is similar, but you need to go to the parallel tab to pick which parallel system you are using. If you want some help with updating the docs, just let me know. ;-) PeterT -- Peter Thompson | Principal Technical Support Engineer | Rogue Wave Software, Inc. Accelerating Great Code | P 508-652-7734 | F 508-652-7701 | www.roguewave.com / peter.thompson(a)roguewave.com

1 0

Re: [mpich-discuss] MPICH and sudo mpiexec
by Kenneth Raffenetti 04 Jun '15

04 Jun '15

On 06/04/2015 08:44 AM, Thibault, Daniel wrote: > "Open RTE detected a parse error in the hostfile"; even more remarkably, it reports a non-sensical location for the error: line 37, token 5 "user", even though the file has two lines and "user" occurs as token 2). base/ras_base_allocate.c line 293 This is an error message from ORTE, the mpiexec/mpirun program included with OpenMPI, not MPICH. Your paths for MPICH 3.0.4 must not be correct. Ken _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH and sudo mpiexec
by Thibault, Daniel 04 Jun '15

04 Jun '15

-----Message d'origine----- De : "Antonio J. Peña" [mailto:[email protected]] Envoyé : 3 juin 2015 11:03 > On 06/03/2015 08:40 AM, Thibault, Daniel wrote: > > ------------------------------ > > It seems there are two problems here. The first is using a user-qualified machine file (e.g. <username>@<host>): this fails even before we get to the elevation problem. The failure is different under MPICH 3.1.4: > > Yeah, I don't think we support that kind of format. Our wiki page > (http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager) > is down now and I can't check for sure, but I seem to remember that the correct format to specify the user name in host files is: > > <host> user=<username> > > Antonio J. Peña > Postdoctoral Appointee > Mathematics and Computer Science Division Argonne National Laboratory > 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya ------------------------------ That's indeed the correct format! Remarkably, only for MPICH 3.1.4. I tried this format with 3.0.4 and it choked ("Open RTE detected a parse error in the hostfile"; even more remarkably, it reports a non-sensical location for the error: line 37, token 5 "user", even though the file has two lines and "user" occurs as token 2). base/ras_base_allocate.c line 293 Daniel U. Thibault RDDC - Centre de recherches de Valcartier | DRDC - Valcartier Research Centre NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ> Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/> _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] how do I configure a passwordless SSH for communication between nodes?
by Moshe Goldstein 04 Jun '15

04 Jun '15

_______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0