June 2015 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] how do I configure a passwordless SSH for communication between nodes?
by Jeff Hammond 04 Jun '15

04 Jun '15

https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2 covers this in a generic context. Jeff On Thu, Jun 4, 2015 at 4:44 AM, Moshe Goldstein <goldmosh(a)g.jct.ac.il> wrote: > Dear all, > > This is my first post to this list. > > I successfully installed MPICH2 in the two Ubuntu Linux 14.04 workstations I > have in my lab. I also successfully run an mpi program in everyone of the > two workstations alone, but when I tried to run the same program using the > "machinefile" option (in order to run it in both machines in parallel), a > password prompt is shown. > I found in the internet instructions how to allow passwordless communication > between nodes, but it continues to require a password. > Could you, please, tell me what I need to do in order to allow the two nodes > to communicate without the need of a password? > > Thank you in advance for your help. > -- Dr. Moshe Goldstein > Computer Science Dept. > Jerusalem College of Technology > Jerusalem, Israel > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

how do I configure a passwordless SSH for communication between nodes?
by Moshe Goldstein 04 Jun '15

04 Jun '15

_______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Totalview and Mpich3
by John DelSignore 03 Jun '15

03 Jun '15

Hi Joe, I forwarded you email to my colleagues in TotalView Support. I'm sure one of them will be able to help you. Cheers, John D. -- John V. DelSignore, Jr. John.DelSignore(a)roguewave.com Chief Technology Officer, TotalView Rogue Wave Software WWW: http://www.roguewave.com 24 Prime Parkway Phone: (508) 652-7730 Natick, MA 01760 FAX: (508) 652-7787 Joseph Kotulski wrote: > When I try to run totalview with a program compiled with: > > mpich-3-1.2 > > or > > mpich-3.1.4 > > using > > mpirun -tv -np 4 program , I get the error > > [mpiexec(a)s976392.sandia.gov] match_arg (utils/args/args.c:159): > unrecognized argument tv > [mpiexec(a)s976392.sandia.gov] HYDU_parse_array (utils/args/args.c:174): > argument matching returned error > [mpiexec(a)s976392.sandia.gov] parse_args (ui/mpich/utils.c:1596): error > parsing input array > [mpiexec(a)s976392.sandia.gov] HYD_uii_mpx_get_parameters > (ui/mpich/utils.c:1648): unable to parse user arguments > [mpiexec(a)s976392.sandia.gov] main (ui/mpich/mpiexec.c:153): error > parsing parameters > > > when I try to run it in totalview it appears totalview runs a similar > command > > PI launch string is: mpirun -tv -ksq -np 4 program > > and I get the same error as above. This is with totalview > 8.13.0-0. > > Is there a way to circumvent this issue? > > > Joe Kotulski > Sandia National Labs > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Totalview and Mpich3
by Joseph Kotulski 03 Jun '15

03 Jun '15

When I try to run totalview with a program compiled with: mpich-3-1.2 or mpich-3.1.4 using mpirun -tv -np 4 program , I get the error [mpiexec(a)s976392.sandia.gov] match_arg (utils/args/args.c:159): unrecognized argument tv [mpiexec(a)s976392.sandia.gov] HYDU_parse_array (utils/args/args.c:174): argument matching returned error [mpiexec(a)s976392.sandia.gov] parse_args (ui/mpich/utils.c:1596): error parsing input array [mpiexec(a)s976392.sandia.gov] HYD_uii_mpx_get_parameters (ui/mpich/utils.c:1648): unable to parse user arguments [mpiexec(a)s976392.sandia.gov] main (ui/mpich/mpiexec.c:153): error parsing parameters when I try to run it in totalview it appears totalview runs a similar command PI launch string is: mpirun -tv -ksq -np 4 program and I get the same error as above. This is with totalview 8.13.0-0. Is there a way to circumvent this issue? Joe Kotulski Sandia National Labs _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] (no subject)
by Kenneth Raffenetti 03 Jun '15

03 Jun '15

Hi Maxim, This issue is fixed in the MPICH master repository. You can try a 3.2 beta release or one of the recent nightly builds here: http://www.mpich.org/static/downloads/nightly/master/ Ken On 06/03/2015 04:44 AM, Maxim Abalenkov wrote: > Dear all, > > I believe I have encountered a minor bug in the MPICH 3.1.4 > implementation. Following the 'README' advice for configuration flags > for performance runs I have used the flag '--disable-error-checking'. > Configure works fine, but the compilation fails at the file > 'info_get.c', where the program tries to use the label 'fn_fail', that > is __not__ defined in case the error checks are __disabled__. I have > tried to configure and compile with both Intel and GNU Fortran > compilers. The problem persists in both cases. Please see the attached > TAR archive for 'c.txt' and 'm.txt' log files. Thank you. > > -- > Kind regards, > Maxim Abalenkov > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH and sudo mpiexec
by Thibault, Daniel 03 Jun '15

03 Jun '15

------------------------------ Date: Tue, 2 Jun 2015 17:31:21 -0500 From: "Antonio J. Peña" <apenya(a)mcs.anl.gov> To: <discuss(a)mpich.org> Your environment variables and password-less privileges are likely to not being propagated to applications executed by "sudo" (for example, can you do "sudo ssh" between nodes without being asked for a password?). If you really need that, consider logging in as root and preparing the environment there. Best, Antonio ------------------------------ Unfortunately, that is not an option. Ubuntu systems have disabled root accounts: you can't log into them. That would not be desirable anyway. It seems there are two problems here. The first is using a user-qualified machine file (e.g. <username>@<host>): this fails even before we get to the elevation problem. The failure is different under MPICH 3.1.4: $ mpiexec -n 4 -f ~/.machines.user.mpi /usr/src/mpich-3.1.4/examples/cpi Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(498)..............: MPID_Init(187).....................: channel initialization failed MPIDI_CH3_Init(89).................: MPID_nem_init(320).................: MPID_nem_tcp_init(171).............: MPID_nem_tcp_get_business_card(418): MPID_nem_tcp_init(377).............: gethostbyname failed, username(a)hostname.domainname (errno 1) [ a duplicate of the previous 8 lines occurs here ] =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 25764 RUNNING AT username(a)hostname.domainname = EXIT CODE: 1 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== Is this a bug or am I just writing the machine file wrong? The other is regarding the need for elevation in any case: I'm not convinced MPI has the facilities required to tell the hosted instances to request sudo elevation. Pondering my problem, I've now come to the conclusion that the Epiphany API should not be called from a sudoed MPI worker instance; rather, I'll write a sudoed daemon that will provide the Epiphany API services to the MPI workers through a socket or some such. That way, the unprivileged MPI workers will be able to do their thing. Daniel U. Thibault RDDC - Centre de recherches de Valcartier | DRDC - Valcartier Research Centre NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ> Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/> _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

(no subject)
by Maxim Abalenkov 03 Jun '15

03 Jun '15

Dear all, I believe I have encountered a minor bug in the MPICH 3.1.4 implementation. Following the 'README' advice for configuration flags for performance runs I have used the flag '--disable-error-checking'. Configure works fine, but the compilation fails at the file 'info_get.c', where the program tries to use the label 'fn_fail', that is __not__ defined in case the error checks are __disabled__. I have tried to configure and compile with both Intel and GNU Fortran compilers. The problem persists in both cases. Please see the attached TAR archive for 'c.txt' and 'm.txt' log files. Thank you. -- Kind regards, Maxim Abalenkov _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH and sudo mpiexec
by "Antonio J. Peña" 02 Jun '15

02 Jun '15

Hi Daniel, Your environment variables and password-less privileges are likely to not being propagated to applications executed by "sudo" (for example, can you do "sudo ssh" between nodes without being asked for a password?). If you really need that, consider logging in as root and preparing the environment there. Best, Antonio On 06/02/2015 11:21 AM, Thibault, Daniel wrote: > I have a small network of machines all running the same OS (Linaro > Ubuntu Linux); they were all cloned from the same disk image and > differ only in their machine names (UNIT1 through UNIT4). > I can ssh between them at will, trusty has been established and I > no longer get asked for a password upon connecting. MPICH is > installed from the Ubuntu repository (not quite the latest version: > mpiexec reports version OpenRTE 1.8.1, the mpich package is > 3.0.4-6ubuntu1), and I can run a demo like cpi with no issues, using a > little mpi-run.sh bash script (the default shell is tcsh, however) : > (begin script) > #!/bin/bash > set -e > ESDK=${EPIPHANY_HOME} > ELIBS=${ESDK}/tools/host/lib:${LD_LIBRARY_PATH} > EHDF=${EPIPHANY_HDF} > echo "Running cpi on machines.u2.mpi" > LD_LIBRARY_PATH=${ELIBS} mpiexec --allow-run-as-root -machinefile > /home/linaro/.machines.u2.mpi -n 1 /home/linaro/myMPI/cpi > echo "Done!" > (end script) > .machines.u2.mpi consists of the one line: > linaro@UNIT2 > From UNIT1, if I do: > $ ./mpi-run.sh > Running cpi on machines.u2.mpi > Process 0 of 1 is on UNIT2 > pi is approximately 3.1415926544231341, Error is 0.0000000008333410 > wall clock time = 0.001327 > Done! > If I edit the script to change the mpiexec line like this: > sudo -E LD_LIBRARY_PATH=${ELIBS} mpiexec --allow-run-as-root > -machinefile /home/linaro/.machines.u2.mpi -n 1 /home/linaro/myMPI/cpi > Now I get (edited for brevity): > $ ./mpi-run.sh > Running cpi on machines.u2.mpi > linaro@UNIT2’s password: > PATH=/usr/local/bin[…]: Command not found. > export: Command not found. > LD_LIBRARY_PATH=/usr/local/lib[…]: Command not found. > export: Command not found. > DYLD_LIBRARY_PATH: Undefined variable. > And it just stops there. Note that the LD_LIBRARY_PATH being reported > is **not** the one passed in by the script. I don’t think it’s > managing to reach the mpi execution stage itself. > If the machinefile lists more than one host, the password prompts > appear two at a time and interfere with each other such that no login > succeeds (although all machines have the same password). > Googling around, I’ve seen this series of error outputs in a wide > variety of other contexts, including Open MPI but also some completely > unrelated application suites and SDKs. > My problem is that the mpi binaries I need to run on the hosts > absolutely require sudo elevation. Is sudo mpiexec the way to go? > What is going on in my example case? > Daniel U. Thibault > RDDC - Centre de recherches de Valcartier | DRDC - Valcartier Research > Centre > NAC : _918V QSDJ_ > <http://www.travelgis.com/map.asp?addr=918V%20QSDJ><_http://www.travelgis.com/map.asp?addr=918V%20QSDJ_> > Gouvernement du Canada | Government of Canada > <_http://www.valcartier.drdc-rddc.gc.ca/_> > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI_Comm_idup hang with multiple nodes
by Huiwei Lu 02 Jun '15

02 Jun '15

Hi Daniel, Thanks for reporting the bug. I can reproduce it on my desktop. I will work on a fix. You can track the progress here. http://trac.mpich.org/projects/mpich/ticket/2269 -- Huiwei Lu Postdoc Appointee Mathematics and Computer Science Division Argonne National Laboratory http://www.mcs.anl.gov/~huiweilu/ On Tue, Jun 2, 2015 at 4:32 PM, Daniel Pou <daniel.pou(a)gmail.com> wrote: > Starting with the test comm_idup_mul.c, and increasing the value of > NUM_ITER 10, I am seeing a > fairly regular hang with a build from latest master (commit 25204 5/31). > > I am running with 2 ranks (1 per node) from SLURM (v2.6.5). I was able to > see the behavior with > > srun -N 2 -n 2 ./comm_idup_mul > (linking against SLURM PMI) and > > salloc -N 2 -n 2 mpiexec ./comm_idup_mul. > > I witnessed this with both the MXM and TCP netmods. I don't see any issues > on single node runs. > > Thank you, > -Dan > > For reference: > Modified code from http://trac.mpich.org/projects/mpich/ticket/1935 > > > /* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ > /* > * * (C) 2012 by Argonne National Laboratory. > * * See COPYRIGHT in top-level directory. > * */ > > /* > * * Test creating multiple communicators with MPI_Comm_idup. > * */ > > #include <stdio.h> > #include <mpi.h> > > #define NUM_ITER 10 > > int main(int argc, char **argv) > { > int i, rank; > MPI_Comm comms[NUM_ITER]; > MPI_Request req[NUM_ITER]; > > MPI_Init(&argc, &argv); > MPI_Comm_rank(MPI_COMM_WORLD, &rank); > > for (i = 0; i < NUM_ITER; i++) > MPI_Comm_idup(MPI_COMM_WORLD, &comms[i], &req[i]); > > MPI_Waitall(NUM_ITER, req, MPI_STATUSES_IGNORE); > > for (i = 0; i < NUM_ITER; i++) > MPI_Comm_free(&comms[i]); > > if (rank == 0) > printf(" No Errors\n"); > > MPI_Finalize(); > > return 0; > } > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

MPI_Comm_idup hang with multiple nodes
by Daniel Pou 02 Jun '15

02 Jun '15

Starting with the test comm_idup_mul.c, and increasing the value of NUM_ITER 10, I am seeing a fairly regular hang with a build from latest master (commit 25204 5/31). I am running with 2 ranks (1 per node) from SLURM (v2.6.5). I was able to see the behavior with srun -N 2 -n 2 ./comm_idup_mul (linking against SLURM PMI) and salloc -N 2 -n 2 mpiexec ./comm_idup_mul. I witnessed this with both the MXM and TCP netmods. I don't see any issues on single node runs. Thank you, -Dan For reference: Modified code from http://trac.mpich.org/projects/mpich/ticket/1935 /* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ /* * * (C) 2012 by Argonne National Laboratory. * * See COPYRIGHT in top-level directory. * */ /* * * Test creating multiple communicators with MPI_Comm_idup. * */ #include <stdio.h> #include <mpi.h> #define NUM_ITER 10 int main(int argc, char **argv) { int i, rank; MPI_Comm comms[NUM_ITER]; MPI_Request req[NUM_ITER]; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); for (i = 0; i < NUM_ITER; i++) MPI_Comm_idup(MPI_COMM_WORLD, &comms[i], &req[i]); MPI_Waitall(NUM_ITER, req, MPI_STATUSES_IGNORE); for (i = 0; i < NUM_ITER; i++) MPI_Comm_free(&comms[i]); if (rank == 0) printf(" No Errors\n"); MPI_Finalize(); return 0; } _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0