October 2014 - discuss - mailman3.cels.anl.gov

Problem Running MPI on cluster
by Md. Amjad Hossain 21 Oct '14

21 Oct '14

Hi I am trying to run simple hello world program on cluster nodes. I am running it by following command but getting errors: Command: /usr/lib64/mpich/bin/mpirun -n 4 -f host_file ./mpi_hello_world errors: [mpiexec(a)md-lin-01.mcs.kent.edu] control_cb (./pm/pmiserv/pmiserv_cb.c:202): assert (!closed) failed [mpiexec(a)md-lin-01.mcs.kent.edu] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status [mpiexec(a)md-lin-01.mcs.kent.edu] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:197): error waiting for event [mpiexec(a)md-lin-01.mcs.kent.edu] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion Before running the command I am setting variables MPIRUN =mpi diretory and MPI_HOSTS=host_file. The "host_file" has four nodes and they can ssh to each other without password. MPICH version I am running is: 3.0.4. The MPI code is attached. Any help to solve the problem please? _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Re : MPI process killed and SIGUSR1
by Roy, Hirak 20 Oct '14

20 Oct '14

Will MPI_IProbe going to work here? I don't want to block my thread on a receive. The client can be busy arbitrary amount of time and the server-thread can not be blocked on a receive. Thanks, Hirak From: Roy, Hirak Sent: Tuesday, October 21, 2014 9:37 AM To: discuss(a)mpich.org Subject: Re : MPI process killed and SIGUSR1 Hi Wesley, So if I do the following from the server side : MPI_Irecv (&buf, 1, MPI_INT, 0, MPI_TAG_SANITY, clientCom, &req); MPI_Wait(&req, &stat); Now the client is killed. However, the wait does not terminate. Thanks, Hirak It's not guaranteed that you will receive an error from a send (of any kind other than ssend). The reason is that the implementation can buffer the message internally and return to the application before the message is actually transferred. You will definitely receive an error related to any kind of receive, but sends are less reliable in that sense. Thanks, Wesley On Oct 20, 2014, at 7:53 AM, Roy, Hirak <Hirak_Roy at mentor.com<https://lists.mpich.org/mailman/listinfo/discuss><mailto:Hirak_Roy at mentor.com<https://lists.mpich.org/mailman/listinfo/discuss>>> wrote: Hi Wesley, Suppose my server-client connection is setup using MPI_Comm_accept/MPI_Comm_connect. Now one of the clients dies. If the server sends a non-blocking message to the dead client by MPI_Isend, will MPI_Wait guaranteed to return Error code? Thanks, Hirak >You will only receive the notification if you use a communicating call. That call doesn't do any communication so it won't return the error. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re : MPI process killed and SIGUSR1
by Roy, Hirak 20 Oct '14

20 Oct '14

Hi Wesley, So if I do the following from the server side : MPI_Irecv (&buf, 1, MPI_INT, 0, MPI_TAG_SANITY, clientCom, &req); MPI_Wait(&req, &stat); Now the client is killed. However, the wait does not terminate. Thanks, Hirak It's not guaranteed that you will receive an error from a send (of any kind other than ssend). The reason is that the implementation can buffer the message internally and return to the application before the message is actually transferred. You will definitely receive an error related to any kind of receive, but sends are less reliable in that sense. Thanks, Wesley On Oct 20, 2014, at 7:53 AM, Roy, Hirak <Hirak_Roy at mentor.com<https://lists.mpich.org/mailman/listinfo/discuss><mailto:Hirak_Roy at mentor.com<https://lists.mpich.org/mailman/listinfo/discuss>>> wrote: Hi Wesley, Suppose my server-client connection is setup using MPI_Comm_accept/MPI_Comm_connect. Now one of the clients dies. If the server sends a non-blocking message to the dead client by MPI_Isend, will MPI_Wait guaranteed to return Error code? Thanks, Hirak >You will only receive the notification if you use a communicating call. That call doesn't do any communication so it won't return the error. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Error on make command to install mpich
by Sumit Sharma 20 Oct '14

20 Oct '14

(See attached file: config.log) Thanks in advance. Sumit Sharma Fellow & Area Convenor Centre for Environmental Studies Earth Science and Climate Change Division The Energy and Resources Institute (TERI) IHC complex, Lodi Road, New Delhi-3 Phone: 011-24682100, 24682111 Fax: 011-24682144 From: "Antonio J. Peña" <apenya(a)mcs.anl.gov> To: <discuss(a)mpich.org> Date: 10/20/2014 08:20 PM Subject: Re: [mpich-discuss] Error on make command to install mpich Can you please send your config.log file along? - Antonio On 10/20/2014 05:53 AM, Sumit Sharma wrote: Dear Researchers, I am trying to install mpich on my dual core linux machine. and getting errors when I try to run the make command. The c.txt and m.txt fiels are attached. I am using gcc compiler for the run. Looking forward to your kind help. Thanks (See attached file: m.txt)(See attached file: c.txt) Best Regards Sumit Sharma Fellow & Area Convenor Centre for Environmental Studies Earth Science and Climate Change Division The Energy and Resources Institute (TERI) IHC complex, Lodi Road, New Delhi-3 Email Security, formerly MX Logic.(See attached file: m.txt)(See attached file: c.txt) ------------------------------------------------------------------------------------------------------------ Disclaimer: The information contained in this e-mail is intended for the person or entity to which it is addressed, and it may contain confidential and/or privileged material. Any review or other use of this mail or taking any action based on it by persons or entities other than the intended recipient is strictly prohibited. If you receive this e-mail by mistake, please contact the sender, and delete all copies of this mail.This e-mail has been scanned and verified by McAfee SaaS Email Security, formerly MX Logic. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss -- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya_______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss ------------------------------------------------------------------------------------------------------------ Disclaimer: The information contained in this e-mail is intended for the person or entity to which it is addressed, and it may contain confidential and/or privileged material. Any review or other use of this mail or taking any action based on it by persons or entities other than the intended recipient is strictly prohibited. If you receive this e-mail by mistake, please contact the sender, and delete all copies of this mail.This e-mail has been scanned and verified by McAfee SaaS Email Security, formerly MX Logic. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Error on make command to install mpich
by Seo, Sangmin 20 Oct '14

20 Oct '14

Hi Sumit, In your c.txt, line 242-254 are: checking valgrind/drd.h usability... no checking valgrind/drd.h presence... no checking for valgrind/drd.h... no checking whether the valgrind headers are broken or too old... yes I think your system may have an old version of valgrind or some headers are broken. You may want to try configure with '--without-valgrind' if you don’t need the valgrind support, or you can try to build MPICH after installing a new version of valgrind. Best regards, -- Sangmin Seo Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory http://www.mcs.anl.gov/~sseo/ On Oct 20, 2014, at 5:53 AM, Sumit Sharma <sumits(a)teri.res.in<mailto:[email protected]>> wrote: Dear Researchers, I am trying to install mpich on my dual core linux machine. and getting errors when I try to run the make command. The c.txt and m.txt fiels are attached. I am using gcc compiler for the run. Looking forward to your kind help. Thanks (See attached file: m.txt)(See attached file: c.txt) Best Regards Sumit Sharma Fellow & Area Convenor Centre for Environmental Studies Earth Science and Climate Change Division The Energy and Resources Institute (TERI) IHC complex, Lodi Road, New Delhi-3 Email Security, formerly MX Logic.(See attached file: m.txt)(See attached file: c.txt) ------------------------------------------------------------------------------------------------------------ Disclaimer: The information contained in this e-mail is intended for the person or entity to which it is addressed, and it may contain confidential and/or privileged material. Any review or other use of this mail or taking any action based on it by persons or entities other than the intended recipient is strictly prohibited. If you receive this e-mail by mistake, please contact the sender, and delete all copies of this mail.This e-mail has been scanned and verified by McAfee SaaS Email Security, formerly MX Logic.<m.txt><c.txt>_______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Error on make command to install mpich
by "Antonio J. Peña" 20 Oct '14

20 Oct '14

Can you please send your config.log file along? - Antonio On 10/20/2014 05:53 AM, Sumit Sharma wrote: > > Dear Researchers, > I am trying to install mpich on my dual core linux machine. and getting > errors when I try to run the make command. The c.txt and m.txt fiels are > attached. > > I am using gcc compiler for the run. > > Looking forward to your kind help. > > Thanks > (See attached file: m.txt)(See attached file: c.txt) > > Best Regards > > > > > > > > > > Sumit Sharma > Fellow & Area Convenor > Centre for Environmental Studies > Earth Science and Climate Change Division > The Energy and Resources Institute (TERI) > IHC complex, Lodi Road, New Delhi-3 > > > > Email Security, formerly MX Logic.(See attached file: m.txt)(See attached > file: c.txt) > > > ------------------------------------------------------------------------------------------------------------ > > Disclaimer: > > The information contained in this e-mail is intended for the person or entity > to which it is addressed, and it may contain confidential and/or privileged > material. Any review or other use of this mail or taking any action based on it > by persons or entities other than the intended recipient is strictly prohibited. > If you receive this e-mail by mistake, please contact the sender, and delete all > copies of this mail.This e-mail has been scanned and verified by McAfee SaaS > Email Security, formerly MX Logic. > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI process killed and SIGUSR1
by Bland, Wesley B. 20 Oct '14

20 Oct '14

It's not guaranteed that you will receive an error from a send (of any kind other than ssend). The reason is that the implementation can buffer the message internally and return to the application before the message is actually transferred. You will definitely receive an error related to any kind of receive, but sends are less reliable in that sense. Thanks, Wesley On Oct 20, 2014, at 7:53 AM, Roy, Hirak <Hirak_Roy(a)mentor.com<mailto:[email protected]>> wrote: Hi Wesley, Suppose my server-client connection is setup using MPI_Comm_accept/MPI_Comm_connect. Now one of the clients dies. If the server sends a non-blocking message to the dead client by MPI_Isend, will MPI_Wait guaranteed to return Error code? Thanks, Hirak >You will only receive the notification if you use a communicating call. That call doesn't do any communication so it won't return the error. > On Oct 9, 2014, at 11:37 AM, Roy, Hirak <Hirak_Roy at mentor.com<https://lists.mpich.org/mailman/listinfo/discuss>> wrote: > > Hi Pavan and Wesley, > > Thanks Pavan for the information. > > Wesley, > > If I try MPI_Comm_remote_size periodically, then should I get Error-code, when the client on the other side of the intercom is killed ? ( I have already used MPI_ERRORS_RETURN). > > Thanks, > Hirak _______________________________________________ discuss mailing list discuss(a)mpich.org<mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPI process killed and SIGUSR1
by Roy, Hirak 20 Oct '14

20 Oct '14

Hi Wesley, Suppose my server-client connection is setup using MPI_Comm_accept/MPI_Comm_connect. Now one of the clients dies. If the server sends a non-blocking message to the dead client by MPI_Isend, will MPI_Wait guaranteed to return Error code? Thanks, Hirak >You will only receive the notification if you use a communicating call. That call doesn't do any communication so it won't return the error. > On Oct 9, 2014, at 11:37 AM, Roy, Hirak <Hirak_Roy at mentor.com<https://lists.mpich.org/mailman/listinfo/discuss>> wrote: > > Hi Pavan and Wesley, > > Thanks Pavan for the information. > > Wesley, > > If I try MPI_Comm_remote_size periodically, then should I get Error-code, when the client on the other side of the intercom is killed ? ( I have already used MPI_ERRORS_RETURN). > > Thanks, > Hirak _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Error on make command to install mpich
by Sumit Sharma 20 Oct '14

20 Oct '14

Dear Researchers, I am trying to install mpich on my dual core linux machine. and getting errors when I try to run the make command. The c.txt and m.txt fiels are attached. I am using gcc compiler for the run. Looking forward to your kind help. Thanks (See attached file: m.txt)(See attached file: c.txt) Best Regards Sumit Sharma Fellow & Area Convenor Centre for Environmental Studies Earth Science and Climate Change Division The Energy and Resources Institute (TERI) IHC complex, Lodi Road, New Delhi-3 Email Security, formerly MX Logic.(See attached file: m.txt)(See attached file: c.txt) ------------------------------------------------------------------------------------------------------------ Disclaimer: The information contained in this e-mail is intended for the person or entity to which it is addressed, and it may contain confidential and/or privileged material. Any review or other use of this mail or taking any action based on it by persons or entities other than the intended recipient is strictly prohibited. If you receive this e-mail by mistake, please contact the sender, and delete all copies of this mail.This e-mail has been scanned and verified by McAfee SaaS Email Security, formerly MX Logic. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] where mpi waitsome only returns one request each time
by myself 19 Oct '14

19 Oct '14

Thanks for your help. With your configuration, I get the expected result on the same machine. Previous configuration: ==== 1>--prefix=/home/nbtest/develop/mpich30 --with-device=ch3:nemesis CFLAGS=-fPIC 2> --prefix=/home/nbtest/develop/mpich30 --with-device=ch3:nemesis --enable-romio --enable-nemesis-dbg-localoddeven --enable-g=all --enable-fast=O0 --enable-fortran=f77 Current configuration: ==== --prefix=/home/nbtest/develop/mpich30 --with-device=ch3:nemesis --enable-romio --enable-nemesis-dbg-localoddeven --enable-g=all --enable-fast=O0 --enable-fortran=f77,fc CC=gcc CXX=g++ F77=gfortran FC=gfortran At 2014-10-20 10:39:20, "Junchao Zhang" <jczhang(a)mcs.anl.gov> wrote: Try to run on another machine, you may get different results. My configure is $ mpichversion MPICH Version: 3.1.3 MPICH Release date:unreleased development copy MPICH Device: ch3:nemesis MPICH configure: --enable-romio --enable-nemesis-dbg-localoddeven --enable-g=all --enable-fast=O0 --enable-fortran=f77,fc CC=gcc CXX=g++ F77=gfortran FC=gfortran MPICH CC: gcc -g -O0 MPICH CXX: g++ -g -O0 MPICH F77: gfortran -g -O0 MPICH FC: gfortran -g -O0 --Junchao Zhang On Sun, Oct 19, 2014 at 9:23 PM, myself <chcdlf(a)126.com> wrote: I run several times of the program, the results are as the same as before. Could you tell me your MPICH version and compiling parameters? The version of my MPICH is 3.1.3, I compile it as $ mpirun --version HYDRA build details: Version: 3.1.3 Release Date: Wed Oct 8 09:37:19 CDT 2014 CC: gcc -fPIC CXX: g++ F77: gfortran F90: gfortran Configure options: '--disable-option-checking' '--prefix=/usr/local/mpich3' '--with-device=ch3:nemesis' 'CFLAGS=-fPIC -O2' '--disable-fortran' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LDFLAGS= ' 'LIBS=-lpthread ' 'CPPFLAGS= -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/mpl/include -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/mpl/include -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/openpa/src -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/openpa/src -D_REENTRANT -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/mpi/romio/include' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs cobalt Checkpointing libraries available: Demux engines available: poll select the whole program is ========= #include "mpi.h" #include <stdio.h> #include <stdlib.h> #include <sys/time.h> typedef unsigned char byte; int main(int argc, char *argv[]){ int rank,size; byte buf[10][10]; MPI_Request req[10]; MPI_Status stat[10]; int outcount; int array_of_indices[10]; int i; int complete = 0; MPI_Init(&argc, &argv); MPI_Comm_size(MPI_COMM_WORLD, &size); MPI_Comm_rank(MPI_COMM_WORLD, &rank); if(rank==0) { for(i =0; i< 10;i++){ MPI_Isend(buf[i], 10, MPI_BYTE, 1, 123, MPI_COMM_WORLD, &req[i]); } printf("post send\n"); }else { for(i=0; i< 10;i++){ MPI_Irecv(buf[i], 10, MPI_BYTE, 0, 123, MPI_COMM_WORLD, &req[i]); } printf("post recv\n"); } for (;;){ MPI_Waitsome(10, req, &outcount, array_of_indices, stat); complete += outcount; printf("rank %d %d index %d\n", rank, outcount, array_of_indices[0]); if(complete == 10) break; } printf("rank %d complete\n", rank); MPI_Finalize(); } ========= I compile it as $ mpicc -o tests testsome.c and run as $ mpirun -n 2 ./tests At 2014-10-20 09:02:39, "Junchao Zhang" <jczhang(a)mcs.anl.gov> wrote: I think it is random. For example, I ran your code and its output was: post recv post send rank 0 10 index 0 rank 0 complete rank 1 1 index 0 rank 1 9 index 1 rank 1 complete --Junchao Zhang On Sun, Oct 19, 2014 at 9:48 AM, myself <chcdlf(a)126.com> wrote: Because I want to understand the behavior of MPI_Waitsome, I change the program as fellow. MPI can waitsome 10 Isend request once time, while waitsome 1 Irecv request each time. Why waitsome-Irecv can not be more than 1 as I expect? ===== MPI_Init(&argc, &argv); MPI_Comm_size(MPI_COMM_WORLD, &size); MPI_Comm_rank(MPI_COMM_WORLD, &rank); if(rank==0) { for(i =0; i< 10;i++){ MPI_Isend(buf[i], 10, MPI_BYTE, 1, 123, MPI_COMM_WORLD, &req[i]); } printf("post send\n"); }else { for(i=0; i< 10;i++){ MPI_Irecv(buf[i], 10, MPI_BYTE, 0, 123, MPI_COMM_WORLD, &req[i]); } printf("post recv\n"); } for (;;){ MPI_Waitsome(10, req, &outcount, array_of_indices, stat); complete += outcount; printf("rank %d %d index %d\n", rank, outcount, array_of_indices[0]); if(complete == 10) break; } printf("rank %d complete\n", rank); MPI_Finalize(); ===== $ mpirun -n 2 ./tests post send post recv rank 0 10 index 0 rank 0 complete rank 1 1 index 0 rank 1 1 index 1 rank 1 1 index 2 rank 1 1 index 3 rank 1 1 index 4 rank 1 1 index 5 rank 1 1 index 6 rank 1 1 index 7 rank 1 1 index 8 rank 1 1 index 9 rank 1 complete At 2014-10-19 22:36:25, "Bland, Wesley B." <wbland(a)anl.gov> wrote: You need to move your waitany outside of the if statement so both ranks execute it. On Oct 19, 2014, at 9:33 AM, myself <chcdlf(a)126.com> wrote: Here is my test program ====== #include "mpi.h" #include <stdio.h> #include <stdlib.h> #include <sys/time.h> typedef unsigned char byte; int main(int argc, char *argv[]){ MPI_Init(&argc, &argv); int rank,size; MPI_Comm_size(MPI_COMM_WORLD, &size); MPI_Comm_rank(MPI_COMM_WORLD, &rank); byte buf[10][10]; MPI_Request req[10]; MPI_Status stat[10]; int outcount; int array_of_indices[10]; int i; if(rank==0) { for(i =0; i< 10;i++){ MPI_Isend(buf[i], 10, MPI_BYTE, 1, 123, MPI_COMM_WORLD, &req[i]); } printf("post send\n"); MPI_Waitall(10, req, stat); printf("send over\n"); }else { for(i=0; i< 10;i++){ MPI_Irecv(buf[i], 10, MPI_BYTE, 0, 123, MPI_COMM_WORLD, &req[i]); } printf("post recv\n"); for (i=0;i<10;i++){ MPI_Waitsome(10, req, &outcount, array_of_indices, stat); printf("%d index %d\n", outcount, array_of_indices[0]); } } MPI_Finalize(); } ====== get the result like this ====== $ mpirun -n 2 ./tests post send send over post recv 1 index 0 1 index 1 1 index 2 1 index 3 1 index 4 1 index 5 1 index 6 1 index 7 1 index 8 1 index 9 ====== I think I should get some completed receive connection, not just 1. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0