November 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Force MPICH2 to not use network adapter
by Igor Raskin 07 Nov '14

07 Nov '14

Hi Pavan, Thank you for your response. I should have given more details about the platform and MPICH2 I am using. The platform is Windows 7 Professional. The version of MPICH2 is 1.4.1 (32-bit) which is latest version of MPICH2 that I can find. To what version of MPICH2 should I upgrade? Thank you and best regards, Igor -----Original Message----- From: Balaji, Pavan [mailto:[email protected]] Sent: Friday, November 07, 2014 12:50 AM To: discuss(a)mpich.org Subject: Re: [mpich-discuss] Force MPICH2 to not use network adapter Igor, Based on the fact that you are asking about smpd and -localonly options, I’m assuming you are using a super-ancient vesion of MPICH that dinosaurs used to compute with. Perhaps you can upgrade to the latest version? For the past 10 years or so, we have been automatically detecting shared memory and using it internally without going over the network. — Pavan > On Nov 6, 2014, at 8:28 PM, Igor Raskin <igor.raskin(a)weblakes.com> wrote: > > Hello, > > I am running a multi-process calculations using MPICH2 from Argonne National Laboratory. The run is on a single machine, so -localonly option of mpiexec is used. Usually everything works. > If the network adapter is enabled when the run starts, and if I disable it during the run, the run fails with error stating: > > op_read error on left context: Error = -1 > > op_read error on parent context: Error = -1 > > unable to read the cmd header on the left context, Error = -1 . > unable to read the cmd header on the parent context, Error = -1 . > Error posting readv, An existing connection was forcibly closed by the > remote host.(10054) connection to my parent broken, aborting. > state machine failed. > However, if the network adapter is disabled when the run is started, I can enable/disable the adapter as many times as I want, and the run still proceeds to the end. > > Is there a way to run mpiexec or modify smpd configuration such that MPICH2 is not using network adapter for inter-process communication for local runs even if the adapter is available when the run starts? > > Thank you, > > Igor > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Isend and Recv
by Zhen Wang 07 Nov '14

07 Nov '14

I did more experiments, and results agrees with this. Thanks a lot for your help! Best regards, Zhen On Thu, Nov 6, 2014 at 11:55 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > My understanding is that: In eager mode, sender does not need a Ack > message from receiver (to know recv buf is ready), but in rendezvous > mode, it does. > In your case, if it is in rendezvous mode, Isend() issues the request. But > since it is a nonblocking call, before sender gets the Ack message, it > goes into sleep. The asynchronous progress engine does not progress until > MPI_Wait() is called. At that time, the Ack message is got, and message > passing happens. Therefore, MPI_Recv() finishes after MPI_Wait(). > > --Junchao Zhang > > On Thu, Nov 6, 2014 at 10:33 AM, Zhen Wang <toddwz(a)gmail.com> wrote: > >> Junchao, >> >> Thanks for your reply. It works! I digged deeper into the eager and >> rendezvous modes, and got confused.. >> >> The docs I read were >> https://computing.llnl.gov/tutorials/mpi_performance/#Protocols and >> http://www-01.ibm.com/support/knowledgecenter/SSFK3V_1.3.0/com.ibm.cluster.… >> . >> >> My understanding is: >> >> In eager mode, the receiver allocates a buffer and an additional copy is >> required to copy the data from the buffer to user allocated space. This is >> good for small messages, not for large ones. >> >> In rendezvous mode, the user allocated space on the receiver must be >> ready to receive the data. But if the space is ready, the MPI_Recv() should >> be completed almost immediately (there's a handshake between the send and >> receiver, and a copy operation). >> >> If I was right, in my code example, MPI_Recv() should finish after >> MPI_Isend() no matter in eager or rendezvous modes. The memory has been >> allocated, and the handshake should take like no time. Am I missing >> something? >> >> Thanks a lot. >> >> >> Best regards, >> Zhen >> >> On Wed, Nov 5, 2014 at 2:28 PM, Junchao Zhang <jczhang(a)mcs.anl.gov> >> wrote: >> >>> I think it is because MPICH just crosses over the eager to rendezvous >>> mode threshold, when n goes from 9999 to 99999. OpenMPI certainly uses a >>> different threshold than MPICH. >>> When you install MPICH, a utility program mpivars is also installed. >>> Type 'mpivars | grep EAGER', you will get default values for various eager >>> thresholds. >>> >>> In your case, export MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE=5000000 and you >>> will get the same result as OpenMPI. >>> >>> --Junchao Zhang >>> >>> On Wed, Nov 5, 2014 at 11:20 AM, Zhen Wang <toddwz(a)gmail.com> wrote: >>> >>>> Hi MPIers, >>>> >>>> I have some questions regarding MPI_Isend() and MPI_Recv(). MPICH-3.1.3 >>>> is used to compile and run the attached code on Red Hat Enterprise Linux >>>> 6.3 (a shared memory machine). While n = 9999, the MPI_Recv() finishes >>>> immediately after MPI_Isend(): (This is what I understand and expect) >>>> >>>> MPI 1: Recv started at 09:53:53. >>>> MPI 0: Isend started at 09:53:53. >>>> MPI 1: Recv finished at 09:53:53. >>>> MPI 0: Isend finished at 09:53:58. >>>> >>>> When n = 99999, I get the following. The MPI_Recv() finishes after >>>> MPI_Wait(): >>>> >>>> MPI 1: Recv started at 09:47:56. >>>> MPI 0: Isend started at 09:47:56. >>>> MPI 0: Isend finished at 09:48:01. >>>> MPI 1: Recv finished at 09:48:01. >>>> >>>> But with OpenMPI 1.8 and n = 99999, MPI_Recv() finishes immediately >>>> after MPI_Isend(): >>>> >>>> MPI 0: Isend started at 09:55:28. >>>> MPI 1: Recv started at 09:55:28. >>>> MPI 1: Recv finished at 09:55:28. >>>> MPI 0: Isend finished at 09:55:33. >>>> >>>> Am I misunderstanding something here? In case the attached code is >>>> dropped, the code is included. Thanks in advance. >>>> >>>> >>>> #include "mpi.h" >>>> #include <unistd.h> >>>> #include <stdio.h> >>>> #include "vector" >>>> #include <time.h> >>>> >>>> int main(int argc, char* argv[]) >>>> { >>>> MPI_Init(&argc, &argv); >>>> >>>> int rank; >>>> MPI_Comm_rank(MPI_COMM_WORLD, &rank); >>>> >>>> int n = 9999; >>>> std::vector<int> vec(n); >>>> MPI_Request mpiRequest; >>>> MPI_Status mpiStatus; >>>> char tt[9] = {0}; >>>> >>>> MPI_Barrier(MPI_COMM_WORLD); >>>> >>>> if (rank == 0) >>>> { >>>> MPI_Isend(&vec[0], n, MPI_INT, 1, 0, MPI_COMM_WORLD, &mpiRequest); >>>> time_t t = time(0); >>>> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >>>> printf("MPI %d: Isend started at %s.\n", rank, tt); >>>> >>>> //int done = 0; >>>> //while (done == 0) >>>> //{ >>>> // MPI_Test(&mpiRequest, &done, &mpiStatus); >>>> //} >>>> sleep(5); >>>> MPI_Wait(&mpiRequest, &mpiStatus); >>>> >>>> t = time(0); >>>> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >>>> printf("MPI %d: Isend finished at %s.\n", rank, tt); >>>> } >>>> else >>>> { >>>> time_t t = time(0); >>>> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >>>> printf("MPI %d: Recv started at %s.\n", rank, tt); >>>> >>>> MPI_Recv(&vec[0], n, MPI_INT, 0, 0, MPI_COMM_WORLD, &mpiStatus); >>>> >>>> t = time(0); >>>> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >>>> printf("MPI %d: Recv finished at %s.\n", rank, tt); >>>> } >>>> >>>> MPI_Finalize(); >>>> >>>> return 0; >>>> } >>>> >>>> >>>> >>>> Best regards, >>>> Zhen >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>> >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Force MPICH2 to not use network adapter
by Balaji, Pavan 06 Nov '14

06 Nov '14

Igor, Based on the fact that you are asking about smpd and -localonly options, I’m assuming you are using a super-ancient vesion of MPICH that dinosaurs used to compute with. Perhaps you can upgrade to the latest version? For the past 10 years or so, we have been automatically detecting shared memory and using it internally without going over the network. — Pavan > On Nov 6, 2014, at 8:28 PM, Igor Raskin <igor.raskin(a)weblakes.com> wrote: > > Hello, > > I am running a multi-process calculations using MPICH2 from Argonne National Laboratory. The run is on a single machine, so -localonly option of mpiexec is used. Usually everything works. > If the network adapter is enabled when the run starts, and if I disable it during the run, the run fails with error stating: > > op_read error on left context: Error = -1 > > op_read error on parent context: Error = -1 > > unable to read the cmd header on the left context, Error = -1 > . > unable to read the cmd header on the parent context, Error = -1 > . > Error posting readv, An existing connection was forcibly closed by the remote host.(10054) > connection to my parent broken, aborting. > state machine failed. > However, if the network adapter is disabled when the run is started, I can enable/disable the adapter as many times as I want, and the run still proceeds to the end. > > Is there a way to run mpiexec or modify smpd configuration such that MPICH2 is not using network adapter for inter-process communication for local runs even if the adapter is available when the run starts? > > Thank you, > > Igor > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Force MPICH2 to not use network adapter
by Igor Raskin 06 Nov '14

06 Nov '14

Hello, I am running a multi-process calculations using MPICH2 from Argonne National Laboratory. The run is on a single machine, so -localonly option of mpiexec is used. Usually everything works. If the network adapter is enabled when the run starts, and if I disable it during the run, the run fails with error stating: op_read error on left context: Error = -1 op_read error on parent context: Error = -1 unable to read the cmd header on the left context, Error = -1 . unable to read the cmd header on the parent context, Error = -1 . Error posting readv, An existing connection was forcibly closed by the remote host.(10054) connection to my parent broken, aborting. state machine failed. However, if the network adapter is disabled when the run is started, I can enable/disable the adapter as many times as I want, and the run still proceeds to the end. Is there a way to run mpiexec or modify smpd configuration such that MPICH2 is not using network adapter for inter-process communication for local runs even if the adapter is available when the run starts? Thank you, Igor _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Isend and Recv
by Junchao Zhang 06 Nov '14

06 Nov '14

My understanding is that: In eager mode, sender does not need a Ack message from receiver (to know recv buf is ready), but in rendezvous mode, it does. In your case, if it is in rendezvous mode, Isend() issues the request. But since it is a nonblocking call, before sender gets the Ack message, it goes into sleep. The asynchronous progress engine does not progress until MPI_Wait() is called. At that time, the Ack message is got, and message passing happens. Therefore, MPI_Recv() finishes after MPI_Wait(). --Junchao Zhang On Thu, Nov 6, 2014 at 10:33 AM, Zhen Wang <toddwz(a)gmail.com> wrote: > Junchao, > > Thanks for your reply. It works! I digged deeper into the eager and > rendezvous modes, and got confused.. > > The docs I read were > https://computing.llnl.gov/tutorials/mpi_performance/#Protocols and > http://www-01.ibm.com/support/knowledgecenter/SSFK3V_1.3.0/com.ibm.cluster.… > . > > My understanding is: > > In eager mode, the receiver allocates a buffer and an additional copy is > required to copy the data from the buffer to user allocated space. This is > good for small messages, not for large ones. > > In rendezvous mode, the user allocated space on the receiver must be ready > to receive the data. But if the space is ready, the MPI_Recv() should be > completed almost immediately (there's a handshake between the send and > receiver, and a copy operation). > > If I was right, in my code example, MPI_Recv() should finish after > MPI_Isend() no matter in eager or rendezvous modes. The memory has been > allocated, and the handshake should take like no time. Am I missing > something? > > Thanks a lot. > > > Best regards, > Zhen > > On Wed, Nov 5, 2014 at 2:28 PM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > >> I think it is because MPICH just crosses over the eager to rendezvous >> mode threshold, when n goes from 9999 to 99999. OpenMPI certainly uses a >> different threshold than MPICH. >> When you install MPICH, a utility program mpivars is also installed. Type >> 'mpivars | grep EAGER', you will get default values for various eager >> thresholds. >> >> In your case, export MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE=5000000 and you >> will get the same result as OpenMPI. >> >> --Junchao Zhang >> >> On Wed, Nov 5, 2014 at 11:20 AM, Zhen Wang <toddwz(a)gmail.com> wrote: >> >>> Hi MPIers, >>> >>> I have some questions regarding MPI_Isend() and MPI_Recv(). MPICH-3.1.3 >>> is used to compile and run the attached code on Red Hat Enterprise Linux >>> 6.3 (a shared memory machine). While n = 9999, the MPI_Recv() finishes >>> immediately after MPI_Isend(): (This is what I understand and expect) >>> >>> MPI 1: Recv started at 09:53:53. >>> MPI 0: Isend started at 09:53:53. >>> MPI 1: Recv finished at 09:53:53. >>> MPI 0: Isend finished at 09:53:58. >>> >>> When n = 99999, I get the following. The MPI_Recv() finishes after >>> MPI_Wait(): >>> >>> MPI 1: Recv started at 09:47:56. >>> MPI 0: Isend started at 09:47:56. >>> MPI 0: Isend finished at 09:48:01. >>> MPI 1: Recv finished at 09:48:01. >>> >>> But with OpenMPI 1.8 and n = 99999, MPI_Recv() finishes immediately >>> after MPI_Isend(): >>> >>> MPI 0: Isend started at 09:55:28. >>> MPI 1: Recv started at 09:55:28. >>> MPI 1: Recv finished at 09:55:28. >>> MPI 0: Isend finished at 09:55:33. >>> >>> Am I misunderstanding something here? In case the attached code is >>> dropped, the code is included. Thanks in advance. >>> >>> >>> #include "mpi.h" >>> #include <unistd.h> >>> #include <stdio.h> >>> #include "vector" >>> #include <time.h> >>> >>> int main(int argc, char* argv[]) >>> { >>> MPI_Init(&argc, &argv); >>> >>> int rank; >>> MPI_Comm_rank(MPI_COMM_WORLD, &rank); >>> >>> int n = 9999; >>> std::vector<int> vec(n); >>> MPI_Request mpiRequest; >>> MPI_Status mpiStatus; >>> char tt[9] = {0}; >>> >>> MPI_Barrier(MPI_COMM_WORLD); >>> >>> if (rank == 0) >>> { >>> MPI_Isend(&vec[0], n, MPI_INT, 1, 0, MPI_COMM_WORLD, &mpiRequest); >>> time_t t = time(0); >>> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >>> printf("MPI %d: Isend started at %s.\n", rank, tt); >>> >>> //int done = 0; >>> //while (done == 0) >>> //{ >>> // MPI_Test(&mpiRequest, &done, &mpiStatus); >>> //} >>> sleep(5); >>> MPI_Wait(&mpiRequest, &mpiStatus); >>> >>> t = time(0); >>> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >>> printf("MPI %d: Isend finished at %s.\n", rank, tt); >>> } >>> else >>> { >>> time_t t = time(0); >>> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >>> printf("MPI %d: Recv started at %s.\n", rank, tt); >>> >>> MPI_Recv(&vec[0], n, MPI_INT, 0, 0, MPI_COMM_WORLD, &mpiStatus); >>> >>> t = time(0); >>> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >>> printf("MPI %d: Recv finished at %s.\n", rank, tt); >>> } >>> >>> MPI_Finalize(); >>> >>> return 0; >>> } >>> >>> >>> >>> Best regards, >>> Zhen >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >>> >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Isend and Recv
by Zhen Wang 06 Nov '14

06 Nov '14

Junchao, Thanks for your reply. It works! I digged deeper into the eager and rendezvous modes, and got confused.. The docs I read were https://computing.llnl.gov/tutorials/mpi_performance/#Protocols and http://www-01.ibm.com/support/knowledgecenter/SSFK3V_1.3.0/com.ibm.cluster.… . My understanding is: In eager mode, the receiver allocates a buffer and an additional copy is required to copy the data from the buffer to user allocated space. This is good for small messages, not for large ones. In rendezvous mode, the user allocated space on the receiver must be ready to receive the data. But if the space is ready, the MPI_Recv() should be completed almost immediately (there's a handshake between the send and receiver, and a copy operation). If I was right, in my code example, MPI_Recv() should finish after MPI_Isend() no matter in eager or rendezvous modes. The memory has been allocated, and the handshake should take like no time. Am I missing something? Thanks a lot. Best regards, Zhen On Wed, Nov 5, 2014 at 2:28 PM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > I think it is because MPICH just crosses over the eager to rendezvous mode > threshold, when n goes from 9999 to 99999. OpenMPI certainly uses a > different threshold than MPICH. > When you install MPICH, a utility program mpivars is also installed. Type > 'mpivars | grep EAGER', you will get default values for various eager > thresholds. > > In your case, export MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE=5000000 and you will > get the same result as OpenMPI. > > --Junchao Zhang > > On Wed, Nov 5, 2014 at 11:20 AM, Zhen Wang <toddwz(a)gmail.com> wrote: > >> Hi MPIers, >> >> I have some questions regarding MPI_Isend() and MPI_Recv(). MPICH-3.1.3 >> is used to compile and run the attached code on Red Hat Enterprise Linux >> 6.3 (a shared memory machine). While n = 9999, the MPI_Recv() finishes >> immediately after MPI_Isend(): (This is what I understand and expect) >> >> MPI 1: Recv started at 09:53:53. >> MPI 0: Isend started at 09:53:53. >> MPI 1: Recv finished at 09:53:53. >> MPI 0: Isend finished at 09:53:58. >> >> When n = 99999, I get the following. The MPI_Recv() finishes after >> MPI_Wait(): >> >> MPI 1: Recv started at 09:47:56. >> MPI 0: Isend started at 09:47:56. >> MPI 0: Isend finished at 09:48:01. >> MPI 1: Recv finished at 09:48:01. >> >> But with OpenMPI 1.8 and n = 99999, MPI_Recv() finishes immediately after >> MPI_Isend(): >> >> MPI 0: Isend started at 09:55:28. >> MPI 1: Recv started at 09:55:28. >> MPI 1: Recv finished at 09:55:28. >> MPI 0: Isend finished at 09:55:33. >> >> Am I misunderstanding something here? In case the attached code is >> dropped, the code is included. Thanks in advance. >> >> >> #include "mpi.h" >> #include <unistd.h> >> #include <stdio.h> >> #include "vector" >> #include <time.h> >> >> int main(int argc, char* argv[]) >> { >> MPI_Init(&argc, &argv); >> >> int rank; >> MPI_Comm_rank(MPI_COMM_WORLD, &rank); >> >> int n = 9999; >> std::vector<int> vec(n); >> MPI_Request mpiRequest; >> MPI_Status mpiStatus; >> char tt[9] = {0}; >> >> MPI_Barrier(MPI_COMM_WORLD); >> >> if (rank == 0) >> { >> MPI_Isend(&vec[0], n, MPI_INT, 1, 0, MPI_COMM_WORLD, &mpiRequest); >> time_t t = time(0); >> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >> printf("MPI %d: Isend started at %s.\n", rank, tt); >> >> //int done = 0; >> //while (done == 0) >> //{ >> // MPI_Test(&mpiRequest, &done, &mpiStatus); >> //} >> sleep(5); >> MPI_Wait(&mpiRequest, &mpiStatus); >> >> t = time(0); >> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >> printf("MPI %d: Isend finished at %s.\n", rank, tt); >> } >> else >> { >> time_t t = time(0); >> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >> printf("MPI %d: Recv started at %s.\n", rank, tt); >> >> MPI_Recv(&vec[0], n, MPI_INT, 0, 0, MPI_COMM_WORLD, &mpiStatus); >> >> t = time(0); >> strftime(tt, 9, "%H:%M:%S", localtime(&t)); >> printf("MPI %d: Recv finished at %s.\n", rank, tt); >> } >> >> MPI_Finalize(); >> >> return 0; >> } >> >> >> >> Best regards, >> Zhen >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Installation problem Ubuntu 14.04 LTS
by Bland, Wesley B. 06 Nov '14

06 Nov '14

Unfortunately, BLCR hasn’t been working for quite some time now. At some point, we may revive support for it, but at the moment, you’ll have to use an old version if you want to use it. Thanks, Wesley > On Nov 6, 2014, at 7:17 AM, Joni-Pekka Kurronen <joni.kurronen(a)gmail.com> wrote: > > > CAN NOT FIND -lblcr,... I have blcr version 0.8.6_b4 and > it compiles whitout problem's whit version 3.0.4,... > > 3.1.3 ./config options have changes,... I assume options > have changed,... > > Here is error message at end of make: > > make[2]: Poistutaan hakemistosta "/mpi3/S4/mpich-3.1.3/src/pm/hydra" > Making install in . > make[2]: Siirrytään hakemistoon "/mpi3/S4/mpich-3.1.3" > GEN lib/libmpi.la > /usr/bin/ld: cannot find -lblcr > collect2: error: ld returned 1 exit status > make[2]: *** [lib/libmpi.la] Virhe 1 > make[2]: Poistutaan hakemistosta "/mpi3/S4/mpich-3.1.3" > make[1]: *** [install-recursive] Virhe 1 > make[1]: Poistutaan hakemistosta "/mpi3/S4/mpich-3.1.3" > make: *** [install] Virhe 2 > joni@mpi1:~$ > > > ========= > > build script used: > > export > JPK_THREADS="-L/usr/lib/x86_64-linux-gnu/libevent_pthreads-2.0.so.5 > -L/usr/lib/x86_64-linux-gnu/libevent_core-2.0.so.5" > > export LD_FLAGS="$JPK_THREADS" > > export CFLAGS="-Wl,--no-as-needed -fPIC -m64 -pthread -O3 -fopenmp > -lgomp -L$JPK_BLCR/lib/libcr.so" > > export CC=gcc > export CXX=g++ > export FC=gfortran > export F77=gfortran > export MPICH2LIB_CFLAGS="$CFLAGS $JPK_THREADS" > export MPICH2LIB_CXXFLAGS="$CFLAGS $JPK_THREADS" > export MPICH2LIB_CPPFLAGS="$CFLAGS $JPK_THREADS" > > export CXXFLAGS=$CFLAGS > export FCFLAGS="$CFLAGS" # -fdefault-real-8 -fdefault-double-8" > export FFLAGS="$CFLAGS" # -fdefault-real-8 -fdefault-double-8" > export F90FLAGS="" > export F77FLAGS="" > > > if [ "$1" == "c" ]; then > cd $JPK_MPICH2_S > make distclean > fi > > cd $JPK_MPICH2_S > > ./configure --prefix=$JPK_MPICH2 --enable-fast=all,O3 > --with-thread-package=pthreads --with-device=ch3:nemesis --with-gnu-ld > --enable-checkpointing --with-blcr=$JPK_BLCR --with-blcr-include= > $JPK_BLCR/include --with-blcr-lib=$JPK_BLCR/lib --with-pm=hydra > --with-hydrabss=rsh --enable-threads=runtime --disable-smpcoll > --enable-static --enable-shared 2>&1 | tee > $JPK_MPI_DIR/$JPK_VER_B/mes/mpich2.config > > > make -j$JPK_JOBS 2>&1 | tee $JPK_MPI_DIR/$JPK_VER_B/mes/mpich2.make > sudo make install 2>&1 | tee $JPK_MPI_DIR/$JPK_VER_B/mes/mpich2.install > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Installation problem Ubuntu 14.04 LTS
by Joni-Pekka Kurronen 06 Nov '14

06 Nov '14

CAN NOT FIND -lblcr,... I have blcr version 0.8.6_b4 and it compiles whitout problem's whit version 3.0.4,... 3.1.3 ./config options have changes,... I assume options have changed,... Here is error message at end of make: make[2]: Poistutaan hakemistosta "/mpi3/S4/mpich-3.1.3/src/pm/hydra" Making install in . make[2]: Siirrytään hakemistoon "/mpi3/S4/mpich-3.1.3" GEN lib/libmpi.la /usr/bin/ld: cannot find -lblcr collect2: error: ld returned 1 exit status make[2]: *** [lib/libmpi.la] Virhe 1 make[2]: Poistutaan hakemistosta "/mpi3/S4/mpich-3.1.3" make[1]: *** [install-recursive] Virhe 1 make[1]: Poistutaan hakemistosta "/mpi3/S4/mpich-3.1.3" make: *** [install] Virhe 2 joni@mpi1:~$ ========= build script used: export JPK_THREADS="-L/usr/lib/x86_64-linux-gnu/libevent_pthreads-2.0.so.5 -L/usr/lib/x86_64-linux-gnu/libevent_core-2.0.so.5" export LD_FLAGS="$JPK_THREADS" export CFLAGS="-Wl,--no-as-needed -fPIC -m64 -pthread -O3 -fopenmp -lgomp -L$JPK_BLCR/lib/libcr.so" export CC=gcc export CXX=g++ export FC=gfortran export F77=gfortran export MPICH2LIB_CFLAGS="$CFLAGS $JPK_THREADS" export MPICH2LIB_CXXFLAGS="$CFLAGS $JPK_THREADS" export MPICH2LIB_CPPFLAGS="$CFLAGS $JPK_THREADS" export CXXFLAGS=$CFLAGS export FCFLAGS="$CFLAGS" # -fdefault-real-8 -fdefault-double-8" export FFLAGS="$CFLAGS" # -fdefault-real-8 -fdefault-double-8" export F90FLAGS="" export F77FLAGS="" if [ "$1" == "c" ]; then cd $JPK_MPICH2_S make distclean fi cd $JPK_MPICH2_S ./configure --prefix=$JPK_MPICH2 --enable-fast=all,O3 --with-thread-package=pthreads --with-device=ch3:nemesis --with-gnu-ld --enable-checkpointing --with-blcr=$JPK_BLCR --with-blcr-include= $JPK_BLCR/include --with-blcr-lib=$JPK_BLCR/lib --with-pm=hydra --with-hydrabss=rsh --enable-threads=runtime --disable-smpcoll --enable-static --enable-shared 2>&1 | tee $JPK_MPI_DIR/$JPK_VER_B/mes/mpich2.config make -j$JPK_JOBS 2>&1 | tee $JPK_MPI_DIR/$JPK_VER_B/mes/mpich2.make sudo make install 2>&1 | tee $JPK_MPI_DIR/$JPK_VER_B/mes/mpich2.install _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Isend and Recv
by Junchao Zhang 05 Nov '14

05 Nov '14

I think it is because MPICH just crosses over the eager to rendezvous mode threshold, when n goes from 9999 to 99999. OpenMPI certainly uses a different threshold than MPICH. When you install MPICH, a utility program mpivars is also installed. Type 'mpivars | grep EAGER', you will get default values for various eager thresholds. In your case, export MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE=5000000 and you will get the same result as OpenMPI. --Junchao Zhang On Wed, Nov 5, 2014 at 11:20 AM, Zhen Wang <toddwz(a)gmail.com> wrote: > Hi MPIers, > > I have some questions regarding MPI_Isend() and MPI_Recv(). MPICH-3.1.3 is > used to compile and run the attached code on Red Hat Enterprise Linux 6.3 > (a shared memory machine). While n = 9999, the MPI_Recv() finishes > immediately after MPI_Isend(): (This is what I understand and expect) > > MPI 1: Recv started at 09:53:53. > MPI 0: Isend started at 09:53:53. > MPI 1: Recv finished at 09:53:53. > MPI 0: Isend finished at 09:53:58. > > When n = 99999, I get the following. The MPI_Recv() finishes after > MPI_Wait(): > > MPI 1: Recv started at 09:47:56. > MPI 0: Isend started at 09:47:56. > MPI 0: Isend finished at 09:48:01. > MPI 1: Recv finished at 09:48:01. > > But with OpenMPI 1.8 and n = 99999, MPI_Recv() finishes immediately after > MPI_Isend(): > > MPI 0: Isend started at 09:55:28. > MPI 1: Recv started at 09:55:28. > MPI 1: Recv finished at 09:55:28. > MPI 0: Isend finished at 09:55:33. > > Am I misunderstanding something here? In case the attached code is > dropped, the code is included. Thanks in advance. > > > #include "mpi.h" > #include <unistd.h> > #include <stdio.h> > #include "vector" > #include <time.h> > > int main(int argc, char* argv[]) > { > MPI_Init(&argc, &argv); > > int rank; > MPI_Comm_rank(MPI_COMM_WORLD, &rank); > > int n = 9999; > std::vector<int> vec(n); > MPI_Request mpiRequest; > MPI_Status mpiStatus; > char tt[9] = {0}; > > MPI_Barrier(MPI_COMM_WORLD); > > if (rank == 0) > { > MPI_Isend(&vec[0], n, MPI_INT, 1, 0, MPI_COMM_WORLD, &mpiRequest); > time_t t = time(0); > strftime(tt, 9, "%H:%M:%S", localtime(&t)); > printf("MPI %d: Isend started at %s.\n", rank, tt); > > //int done = 0; > //while (done == 0) > //{ > // MPI_Test(&mpiRequest, &done, &mpiStatus); > //} > sleep(5); > MPI_Wait(&mpiRequest, &mpiStatus); > > t = time(0); > strftime(tt, 9, "%H:%M:%S", localtime(&t)); > printf("MPI %d: Isend finished at %s.\n", rank, tt); > } > else > { > time_t t = time(0); > strftime(tt, 9, "%H:%M:%S", localtime(&t)); > printf("MPI %d: Recv started at %s.\n", rank, tt); > > MPI_Recv(&vec[0], n, MPI_INT, 0, 0, MPI_COMM_WORLD, &mpiStatus); > > t = time(0); > strftime(tt, 9, "%H:%M:%S", localtime(&t)); > printf("MPI %d: Recv finished at %s.\n", rank, tt); > } > > MPI_Finalize(); > > return 0; > } > > > > Best regards, > Zhen > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH examples
by Zhao, Xin 05 Nov '14

05 Nov '14

Hi Mike, I think you mean test/mpi/rma, not example/rma. Under test/mpi/rma, there are two tests that are not in testlist, one is ircpi.c, which is an interactive test. We will move it to examples. Another is nb_test.c, which is related to this bug: https://trac.mpich.org/projects/mpich/ticket/1910. The bug exists in mpich-3.1.3 but test code is not correct, that's why we comment out it for now. You can also find explanation about it in test/mpi/rma/testlist.in. Thanks, Xin ________________________________________ From: Michael L. Stokes [Michael.Stokes(a)uah.edu] Sent: Wednesday, November 05, 2014 11:19 AM To: discuss(a)mpich.org Subject: [mpich-discuss] MPICH examples Under examples/rma there are several .c files that are not listed in testlist (for example, nb_test.c). When I look at the source for nb_test.c it appears it should require at least 2 processes to run correctly, but when I run it with n>=2 it hangs. Do 2 questions. 1) What what is going on with applications not listed in testlist, and 2) for is going on with nb_test.c? Thanks Mike _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0