December 2014 - discuss - mailman3.cels.anl.gov

less busy polling with Nemesis
by Daniel Ibanez 09 Dec '14

09 Dec '14

Hello, I have noticed that simple things like MPI_Barrier will raise my CPU usage to near 100% while waiting, and read in past emails and the FAQ that this is due to busy polling in Nemesis. As far as I can tell, the one solution usually offered to reduce this CPU usage is reconfiguring with sockets instead of Nemesis. I would also like to use the MPI_THREAD_MULTIPLE feature, but I read that the socket channels don't support it, and it would be annoying to have to recompile MPICH to get one benefit or the other. I was just wondering if recent advancements invalidated some of the above and officially ask what the options are for reducing the CPU usage of MPICH during barriers. Thank you, _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] (no subject)
by Chafik sanaa 05 Dec '14

05 Dec '14

hi, thanks 2014-12-05 19:37 GMT+01:00 "Antonio J. Peña" <apenya(a)mcs.anl.gov>: > > Dear user, > > Please note that this mailing list is exclusively intended to discuss > potential issues with the MPICH MPI implementation. Since your question is > about general MPI usage, we cannot provide you with that assistance here. > You will surely find assistance in this kind of topics in general > programming forums such as stackoverflow, where some of our team members > contribute as well. Also, web search engines are likely to be a good > resource in your case. > > Best regards, > Antonio > > > > On 12/05/2014 11:50 AM, Chafik sanaa wrote: > > I have a table "tab[]" with 10 elements "0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 > | 9 |" I am sending with the Scatterv function; 4 elements to the process 0 > "|0 | 1 | 2 | 3 | 4| "and 6 elements to process 1 "| 5 | 6 | 7 | 8 | 9 | ", > after I did the summation of the value" add " to each element of the > received table which gives me a new table "afteradd[] "for each process, I > want this time collecting the both tables "afteradd[]" and send them to the > process 0, after a little research I found that there's a GATHERV function > which do that but I do not know how ? > > program: > #include <malloc.h> > #include <stdlib.h> > #include <stdio.h> > #include <time.h> > #include "math.h" > #include "mpi.h" > > int main(int argc, char** argv) > { > int taskid, ntasks; > int ierr, i, itask; > int sendcounts[2048], displs[2048], recvcount; > double **sendbuff, *recvbuff,*afteradd, buffsum, buffsums[2048]; > double inittime, totaltime; > const int nbr_etat = 10; > double tab[nbr_etat]; > for (int i = 0; i < nbr_etat; i++) > tab[i] = i; > int nbr_elm[2] = { 4, 6 }; > int dpl[2] = { 0, 4 }; > MPI_Init(&argc, &argv); > MPI_Comm_rank(MPI_COMM_WORLD, &taskid); > MPI_Comm_size(MPI_COMM_WORLD, &ntasks); > recvbuff = (double *)malloc(sizeof(double)*nbr_etat); > if (taskid == 0) > { > sendbuff = (double **)malloc(sizeof(double *)*ntasks);// run for 2 proc > sendbuff[0] = (double *)malloc(sizeof(double)*ntasks*nbr_etat); > for (i = 1; i < ntasks; i++) > { > sendbuff[i] = sendbuff[i - 1] + nbr_etat; > } > } > else > { > sendbuff = (double **)malloc(sizeof(double *)* 1); > sendbuff[0] = (double *)malloc(sizeof(double)* 1); > } > > if (taskid == 0){ > > srand((unsigned)time(NULL) + taskid); > for (itask = 0; itask<ntasks; itask++) > { > int k; > displs[itask] = itask*nbr_etat; > int s = dpl[itask]; > sendcounts[itask] = nbr_elm[itask]; > > > for (i = 0; i<sendcounts[itask]; i++) > { > k = i + s; > sendbuff[itask][i] = tab[k]; > printf("+ %0.0f ", sendbuff[itask][i]); > } > printf("\n"); > } > } > > recvcount = nbr_elm[taskid]; > > inittime = MPI_Wtime(); > > ierr = MPI_Scatterv(sendbuff[0], sendcounts, displs, MPI_DOUBLE, > recvbuff, recvcount, MPI_DOUBLE, > 0, MPI_COMM_WORLD); > > totaltime = MPI_Wtime() - inittime; > > buffsum = 0.0; > int add = 3; > afteradd = (double *)malloc(sizeof(double)*nbr_etat); > for (i = 0; i<recvcount; i++) > { > afteradd [i]= add + recvbuff[i]; > } > for (i = 0; i<recvcount; i++) > { > printf("* %0.0f ", afteradd[i]); > } > printf("\n"); > if (taskid == 0) > { > free(sendbuff[0]); > free(sendbuff); > } > else > { > free(sendbuff[0]); > free(sendbuff); > free(recvbuff); > } > MPI_Finalize(); > > } > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe:https://lists.mpich.org/mailman/listinfo/discuss > > > > -- > Antonio J. Peña > Postdoctoral Appointee > Mathematics and Computer Science Division > Argonne National Laboratory > 9700 South Cass Avenue, Bldg. 240, Of. 3148 > Argonne, IL 60439-4847apenya(a)mcs.anl.govwww.mcs.anl.gov/~apenya > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] (no subject)
by "Antonio J. Peña" 05 Dec '14

05 Dec '14

Dear user, Please note that this mailing list is exclusively intended to discuss potential issues with the MPICH MPI implementation. Since your question is about general MPI usage, we cannot provide you with that assistance here. You will surely find assistance in this kind of topics in general programming forums such as stackoverflow, where some of our team members contribute as well. Also, web search engines are likely to be a good resource in your case. Best regards, Antonio On 12/05/2014 11:50 AM, Chafik sanaa wrote: > I have a table "tab[]" with 10 elements "0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 > | 8 | 9 |" I am sending with the Scatterv function; 4 elements to the > process 0 "|0 | 1 | 2 | 3 | 4| "and 6 elements to process 1 "| 5 | 6 | > 7 | 8 | 9 | ", after I did the summation of the value" add " to each > element of the received table which gives me a new table "afteradd[] > "for each process, I want this time collecting the both tables > "afteradd[]" and send them to the process 0, after a little research I > found that there's a GATHERV function which do that but I do not know > how ? > program: > #include <malloc.h> > #include <stdlib.h> > #include <stdio.h> > #include <time.h> > #include "math.h" > #include "mpi.h" > > int main(int argc, char** argv) > { > int taskid, ntasks; > int ierr, i, itask; > intsendcounts[2048], displs[2048], recvcount; > double **sendbuff, *recvbuff,*afteradd, buffsum, buffsums[2048]; > double inittime, totaltime; > const int nbr_etat = 10; > double tab[nbr_etat]; > for (int i = 0; i < nbr_etat; i++) > tab[i] = i; > int nbr_elm[2] = { 4, 6 }; > int dpl[2] = { 0, 4 }; > MPI_Init(&argc, &argv); > MPI_Comm_rank(MPI_COMM_WORLD, &taskid); > MPI_Comm_size(MPI_COMM_WORLD, &ntasks); > recvbuff = (double *)malloc(sizeof(double)*nbr_etat); > if (taskid == 0) > { > sendbuff = (double **)malloc(sizeof(double *)*ntasks);// run for 2 proc > sendbuff[0] = (double *)malloc(sizeof(double)*ntasks*nbr_etat); > for (i = 1; i < ntasks; i++) > { > sendbuff[i] = sendbuff[i - 1] + nbr_etat; > } > } > else > { > sendbuff = (double **)malloc(sizeof(double *)* 1); > sendbuff[0] = (double *)malloc(sizeof(double)* 1); > } > > if (taskid == 0){ > > srand((unsigned)time(NULL) + taskid); > for (itask = 0; itask<ntasks; itask++) > { > int k; > displs[itask] = itask*nbr_etat; > int s = dpl[itask]; > sendcounts[itask] = nbr_elm[itask]; > > > for (i = 0; i<sendcounts[itask]; i++) > { > k = i + s; > sendbuff[itask][i] = tab[k]; > printf("+ %0.0f ", sendbuff[itask][i]); > } > printf("\n"); > } > } > > recvcount = nbr_elm[taskid]; > > inittime = MPI_Wtime(); > > ierr = MPI_Scatterv(sendbuff[0], sendcounts, displs, MPI_DOUBLE, > recvbuff, recvcount, MPI_DOUBLE, > 0, MPI_COMM_WORLD); > > totaltime = MPI_Wtime() - inittime; > > buffsum = 0.0; > int add = 3; > afteradd = (double *)malloc(sizeof(double)*nbr_etat); > for (i = 0; i<recvcount; i++) > { > afteradd [i]= add + recvbuff[i]; > } > for (i = 0; i<recvcount; i++) > { > printf("* %0.0f", afteradd[i]); > } > printf("\n"); > if (taskid == 0) > { > free(sendbuff[0]); > free(sendbuff); > } > else > { > free(sendbuff[0]); > free(sendbuff); > free(recvbuff); > } > MPI_Finalize(); > > } > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

(no subject)
by Chafik sanaa 05 Dec '14

05 Dec '14

I have a table "tab[]" with 10 elements "0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |" I am sending with the Scatterv function; 4 elements to the process 0 "|0 | 1 | 2 | 3 | 4| "and 6 elements to process 1 "| 5 | 6 | 7 | 8 | 9 | ", after I did the summation of the value" add " to each element of the received table which gives me a new table "afteradd[] "for each process, I want this time collecting the both tables "afteradd[]" and send them to the process 0, after a little research I found that there's a GATHERV function which do that but I do not know how ? program: #include <malloc.h> #include <stdlib.h> #include <stdio.h> #include <time.h> #include "math.h" #include "mpi.h" int main(int argc, char** argv) { int taskid, ntasks; int ierr, i, itask; int sendcounts[2048], displs[2048], recvcount; double **sendbuff, *recvbuff,*afteradd, buffsum, buffsums[2048]; double inittime, totaltime; const int nbr_etat = 10; double tab[nbr_etat]; for (int i = 0; i < nbr_etat; i++) tab[i] = i; int nbr_elm[2] = { 4, 6 }; int dpl[2] = { 0, 4 }; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &taskid); MPI_Comm_size(MPI_COMM_WORLD, &ntasks); recvbuff = (double *)malloc(sizeof(double)*nbr_etat); if (taskid == 0) { sendbuff = (double **)malloc(sizeof(double *)*ntasks);// run for 2 proc sendbuff[0] = (double *)malloc(sizeof(double)*ntasks*nbr_etat); for (i = 1; i < ntasks; i++) { sendbuff[i] = sendbuff[i - 1] + nbr_etat; } } else { sendbuff = (double **)malloc(sizeof(double *)* 1); sendbuff[0] = (double *)malloc(sizeof(double)* 1); } if (taskid == 0){ srand((unsigned)time(NULL) + taskid); for (itask = 0; itask<ntasks; itask++) { int k; displs[itask] = itask*nbr_etat; int s = dpl[itask]; sendcounts[itask] = nbr_elm[itask]; for (i = 0; i<sendcounts[itask]; i++) { k = i + s; sendbuff[itask][i] = tab[k]; printf("+ %0.0f ", sendbuff[itask][i]); } printf("\n"); } } recvcount = nbr_elm[taskid]; inittime = MPI_Wtime(); ierr = MPI_Scatterv(sendbuff[0], sendcounts, displs, MPI_DOUBLE, recvbuff, recvcount, MPI_DOUBLE, 0, MPI_COMM_WORLD); totaltime = MPI_Wtime() - inittime; buffsum = 0.0; int add = 3; afteradd = (double *)malloc(sizeof(double)*nbr_etat); for (i = 0; i<recvcount; i++) { afteradd [i]= add + recvbuff[i]; } for (i = 0; i<recvcount; i++) { printf("* %0.0f ", afteradd[i]); } printf("\n"); if (taskid == 0) { free(sendbuff[0]); free(sendbuff); } else { free(sendbuff[0]); free(sendbuff); free(recvbuff); } MPI_Finalize(); } _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] f77 bindings and profiling
by Kenneth Raffenetti 05 Dec '14

05 Dec '14

Hi Jan, I think you identified the issue correctly in your last email, based on which symbols now appear in the libmpifort library compared to libfmpich in 3.1. I've made a ticket to make sure we fix this before the next release. http://trac.mpich.org/projects/mpich/ticket/2209 Ken On 12/03/2014 07:56 AM, Jan Bierbaum wrote: > On 28.11.2014 18:07, Jan Bierbaum wrote: >> Is this a bug or are we missing something here? > Any new insights? Did anybody manage to properly use the PMPI interface > with a Fortran application and MPICH 3.1.3? > > As a workaround I just use the Fortran wrapper from MPICH 3.1 now which > still works fine with MPICH 3.1.3. But a proper fix would be better of > course. > > > Regards, Jan > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Error code for MPI_Type_commit and free
by Junchao Zhang 05 Dec '14

05 Dec '14

It is because MPICH auto-generates manpage's explanation based on error codes tagged on each MPI routine. A generic explanation is applied to all routines with the same error code. So you can even see "May be an uncommitted MPI_Datatype (see MPI_Type_commit)." in manpage of MPI_Type_commit". --Junchao Zhang On Fri, Dec 5, 2014 at 9:16 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > I think it is an oversight. MPI_Type_free in MPICH is ok with uncommitted > datatypes. I will correct it. > Thanks. > > --Junchao Zhang > > On Fri, Dec 5, 2014 at 8:04 AM, Marc-Andre Hermanns < > m.a.hermanns(a)grs-sim.de> wrote: > >> Dear MPICH team, >> >> I just recently stumbled over the MPICH manpage for MPI_Type_commit and >> MPI_Type_free. >> >> In the description of MPI_ERR_TYPE it states: >> >> "MPI_ERR_TYPE >> Invalid datatype argument. May be an uncommitted MPI_Datatype (see >> MPI_Type_commit). " >> >> What is meant by an uncommitted MPI_Datatype? As far as I know, >> MPI_Type_free should be callable on uncommitted datatypes. So why is >> there this reference to the commit state? >> >> Cheers, >> Marc-Andre >> -- >> Marc-Andre Hermanns >> Jülich Aachen Research Alliance, >> High Performance Computing (JARA-HPC) >> German Research School for Simulation Sciences GmbH >> >> Schinkelstrasse 2 >> 52062 Aachen >> Germany >> >> Phone: +49 241 80 99753 >> Fax: +49 241 80 6 99753 >> www.grs-sim.de/parallel >> email: m.a.hermanns(a)grs-sim.de >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> > > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Error code for MPI_Type_commit and free
by Junchao Zhang 05 Dec '14

05 Dec '14

I think it is an oversight. MPI_Type_free in MPICH is ok with uncommitted datatypes. I will correct it. Thanks. --Junchao Zhang On Fri, Dec 5, 2014 at 8:04 AM, Marc-Andre Hermanns <m.a.hermanns(a)grs-sim.de > wrote: > Dear MPICH team, > > I just recently stumbled over the MPICH manpage for MPI_Type_commit and > MPI_Type_free. > > In the description of MPI_ERR_TYPE it states: > > "MPI_ERR_TYPE > Invalid datatype argument. May be an uncommitted MPI_Datatype (see > MPI_Type_commit). " > > What is meant by an uncommitted MPI_Datatype? As far as I know, > MPI_Type_free should be callable on uncommitted datatypes. So why is > there this reference to the commit state? > > Cheers, > Marc-Andre > -- > Marc-Andre Hermanns > Jülich Aachen Research Alliance, > High Performance Computing (JARA-HPC) > German Research School for Simulation Sciences GmbH > > Schinkelstrasse 2 > 52062 Aachen > Germany > > Phone: +49 241 80 99753 > Fax: +49 241 80 6 99753 > www.grs-sim.de/parallel > email: m.a.hermanns(a)grs-sim.de > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Error code for MPI_Type_commit and free
by Marc-Andre Hermanns 05 Dec '14

05 Dec '14

Dear MPICH team, I just recently stumbled over the MPICH manpage for MPI_Type_commit and MPI_Type_free. In the description of MPI_ERR_TYPE it states: "MPI_ERR_TYPE Invalid datatype argument. May be an uncommitted MPI_Datatype (see MPI_Type_commit). " What is meant by an uncommitted MPI_Datatype? As far as I know, MPI_Type_free should be callable on uncommitted datatypes. So why is there this reference to the commit state? Cheers, Marc-Andre -- Marc-Andre Hermanns Jülich Aachen Research Alliance, High Performance Computing (JARA-HPC) German Research School for Simulation Sciences GmbH Schinkelstrasse 2 52062 Aachen Germany Phone: +49 241 80 99753 Fax: +49 241 80 6 99753 www.grs-sim.de/parallel email: m.a.hermanns(a)grs-sim.de _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] MPICH 3.1.3 installs potentially broken libtool .la files
by Kenneth Raffenetti 04 Dec '14

04 Dec '14

Hi Markus, Thanks for reporting. This indeed looks broken to me. I've created a ticket to fix this before the next release. http://trac.mpich.org/projects/mpich/ticket/2208 Ken On 12/03/2014 11:05 AM, Markus Geimer wrote: > Dear MPICH developers, > > I noticed that the installed '.la' files for MPICH's libraries include > two linker path ('-L') specifications in the 'dependency_libs' line > referring to the directory where MPICH was built: > > dependency_libs=' -L<builddir>/src/mpl -L<builddir>/src/openpa/src \ > -lrt -lpthread' > > In the past, I ran into link failures with specifying non-existent > directories when using the gold linker, though I'm currently not able > to reproduce this with a minimal example. In any case, pointing to > the build directory in an '.la' file generally seems like a bad idea. > > As a fix, I propose to only add the '-L' entries to 'external_ldflags' > (Makefile.am, line 51) if external MPL and/or OpenPA libraries are > used. Using the internal copies is already taken care of by adding > the corresponding local '.la' files to 'pmpi_convenience_libs'. > > Thanks, > Markus > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] (no subject)
by Chafik sanaa 04 Dec '14

04 Dec '14

thanks 2014-12-04 17:33 GMT+01:00 Seo, Sangmin <sseo(a)anl.gov>: > Hi Chafik, > > You can find examples using MPI_GATHERV if you google it. Here are some > I found: > > http://mpi.deino.net/mpi_functions/MPI_Gatherv.html > http://www.mpi-forum.org/docs/mpi-1.1/mpi-11-html/node70.html > > Please use this mailing list to ask questions related to MPICH. For > general questions, I would suggest to use other developer websites, e.g., > http://stackoverflow.com/. > > Best, > Sangmin > > > On Dec 4, 2014, at 10:27 AM, Chafik sanaa <san.chafik(a)gmail.com> wrote: > > hi > I want a simple program with the function MPI_GATHERV > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0