April 2014 - devel - mailman3.cels.anl.gov

Re: [mpich-devel] detecting mpirun mismatch
by Larry Baker 22 Apr '14

22 Apr '14

This seems like a good suggestion for the kind of sanity checking that an mpirun could do, though I'm sure how. If an executable is statically linked, there would have to be some kind of signature that mpirun could deduced from, e.g., "strings" on the executable. Or, mpirun and the MPI library could cooperate, e.g., mpirun could define some environment variable that MPI_Init() validated. This could be spoofed, I suppose. But, a simplistic implementation might be all that is required to satisfy Jeff's request. Larry Baker US Geological Survey 650-329-5608 baker(a)usgs.gov On 22 Apr 2014, at 2:48 PM, Jeff Hammond wrote: > Does anyone know of a good way to detect when the user has built an > MPI application with mpicc from implementation A and tries to launch > with mpirun/mpiexec from implementation B? > > I know that inside of the application, I can use either > MPI_Get_library_version or preprocessor info to know what > implementation I am using. But can I discover anything about mpirun > inside of the application? Is there anything in the environment that > I can query to know if my PM is Hydra, for example? If not, can that > be added? > > This is a relatively common user error that leads to very bad things > in some cases so it would be nice to be able to detect it and throw a > warning/error. > > Thanks, > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com

1 0

Re: [mpich-devel] detecting mpirun mismatch
by Jeff Hammond 22 Apr '14

22 Apr '14

PMI is specific to MPICH-family, right? The problem that was observed had to do with MPICH/OpenMPI mixing. 'mpirun -np 2' led to two copies of the application running in serial... If PMI implementations are version specific, should they not export PMI_VERSION to the environment so bad pairings can be detected? Thanks, Jeff On Tue, Apr 22, 2014 at 4:52 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > Jeff, > > Do you need to know if it’s Hydra or if it’s a PMI server? Any PMI server can work with any PMI client. Ideally, they’ll negotiate on the PMI version to use, but a bad client/server might assume a particular version. > > You can make sure it’s a PMI server by checking for PMI environment variables. Either PMI_FD or PMI_PORT will need to be set. > > — Pavan > > On Apr 22, 2014, at 4:48 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > >> Does anyone know of a good way to detect when the user has built an >> MPI application with mpicc from implementation A and tries to launch >> with mpirun/mpiexec from implementation B? >> >> I know that inside of the application, I can use either >> MPI_Get_library_version or preprocessor info to know what >> implementation I am using. But can I discover anything about mpirun >> inside of the application? Is there anything in the environment that >> I can query to know if my PM is Hydra, for example? If not, can that >> be added? >> >> This is a relatively common user error that leads to very bad things >> in some cases so it would be nice to be able to detect it and throw a >> warning/error. >> >> Thanks, >> >> Jeff >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com > -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-devel] Issue with MPI_Error_string() for user-defined codes/clases
by Rob Latham 22 Apr '14

22 Apr '14

On 04/22/2014 04:04 AM, Lisandro Dalcin wrote: > The following simple tests segfaults. A quick fix would be to return > an empty string in MPIR_Err_get_dynerr_string() (file dynerrutil.c), I don't muck around in this code very often, so after thinking about it for a bit, and coming up with another way to do it, I opened a ticket so i could get a second (third, I guess) opinion: https://trac.mpich.org/projects/mpich/ticket/2067 ==rob > e.g: > > diff --git a/src/mpi/errhan/dynerrutil.c b/src/mpi/errhan/dynerrutil.c > index 943e8c3..fb40469 100644 > --- a/src/mpi/errhan/dynerrutil.c > +++ b/src/mpi/errhan/dynerrutil.c > @@ -297,11 +297,13 @@ const char *MPIR_Err_get_dynerr_string( int code ) > if (errcode) { > if (errcode < first_free_code) { > errstr = user_code_msgs[errcode]; > + if (!errstr) errstr = ""; > } > } > else { > if (errclass < first_free_class) { > errstr = user_class_msgs[errclass]; > + if (!errstr) errstr = ""; > } > } > > > [dalcinl@kw2060 ~]$ cat error_string2.c > #include <stdio.h> > #include <mpi.h> > int main(int argc, char *argv[]) > { > int errorclass; > char errorstring[MPI_MAX_ERROR_STRING] = {64,0}; > int slen; > MPI_Init(&argc, &argv); > MPI_Add_error_class(&errorclass); > MPI_Error_string(errorclass, errorstring, &slen); > printf("errorclass:%d errorstring:'%s' len:%d\n", errorclass, > errorstring, slen); > MPI_Finalize(); > return 0; > } -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] detecting mpirun mismatch
by Balaji, Pavan 22 Apr '14

22 Apr '14

Jeff, Do you need to know if it’s Hydra or if it’s a PMI server? Any PMI server can work with any PMI client. Ideally, they’ll negotiate on the PMI version to use, but a bad client/server might assume a particular version. You can make sure it’s a PMI server by checking for PMI environment variables. Either PMI_FD or PMI_PORT will need to be set. — Pavan On Apr 22, 2014, at 4:48 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > Does anyone know of a good way to detect when the user has built an > MPI application with mpicc from implementation A and tries to launch > with mpirun/mpiexec from implementation B? > > I know that inside of the application, I can use either > MPI_Get_library_version or preprocessor info to know what > implementation I am using. But can I discover anything about mpirun > inside of the application? Is there anything in the environment that > I can query to know if my PM is Hydra, for example? If not, can that > be added? > > This is a relatively common user error that leads to very bad things > in some cases so it would be nice to be able to detect it and throw a > warning/error. > > Thanks, > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com

1 0

detecting mpirun mismatch
by Jeff Hammond 22 Apr '14

22 Apr '14

Does anyone know of a good way to detect when the user has built an MPI application with mpicc from implementation A and tries to launch with mpirun/mpiexec from implementation B? I know that inside of the application, I can use either MPI_Get_library_version or preprocessor info to know what implementation I am using. But can I discover anything about mpirun inside of the application? Is there anything in the environment that I can query to know if my PM is Hydra, for example? If not, can that be added? This is a relatively common user error that leads to very bad things in some cases so it would be nice to be able to detect it and throw a warning/error. Thanks, Jeff -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-devel] Issue with MPI_Error_string() for user-defined codes/clases
by Rob Latham 22 Apr '14

22 Apr '14

On 04/22/2014 04:04 AM, Lisandro Dalcin wrote: > The MPI-3 standard says (pp.354, lines 39-40): > > """ > If MPI_ERROR_STRING is called when no string has been set, it will > return a empty > string (all spaces in Fortran, "" in C). > """ > > The following simple tests segfaults. A quick fix would be to return > an empty string in MPIR_Err_get_dynerr_string() (file dynerrutil.c), > e.g: Now that I look at this a bit more closely, shouldn't user_code_msgs[] and user_class_msgs[] be initialized to "", not 0 ? That messes up some other assumptions of the errorcode handling code, though.. ==rob > > diff --git a/src/mpi/errhan/dynerrutil.c b/src/mpi/errhan/dynerrutil.c > index 943e8c3..fb40469 100644 > --- a/src/mpi/errhan/dynerrutil.c > +++ b/src/mpi/errhan/dynerrutil.c > @@ -297,11 +297,13 @@ const char *MPIR_Err_get_dynerr_string( int code ) > if (errcode) { > if (errcode < first_free_code) { > errstr = user_code_msgs[errcode]; > + if (!errstr) errstr = ""; > } > } > else { > if (errclass < first_free_class) { > errstr = user_class_msgs[errclass]; > + if (!errstr) errstr = ""; > } > } > > > [dalcinl@kw2060 ~]$ cat error_string2.c > #include <stdio.h> > #include <mpi.h> > int main(int argc, char *argv[]) > { > int errorclass; > char errorstring[MPI_MAX_ERROR_STRING] = {64,0}; > int slen; > MPI_Init(&argc, &argv); > MPI_Add_error_class(&errorclass); > MPI_Error_string(errorclass, errorstring, &slen); > printf("errorclass:%d errorstring:'%s' len:%d\n", errorclass, > errorstring, slen); > MPI_Finalize(); > return 0; > } > > [dalcinl@kw2060 ~]$ mpicc error_string2.c > [dalcinl@kw2060 ~]$ ./a.out > Segmentation fault (core dumped) > > [dalcinl@kw2060 ~]$ valgrind -q ./a.out > ==14176== Invalid read of size 1 > ==14176== at 0x4C6F5E7: MPIU_Strncpy (safestr.c:65) > ==14176== by 0x4C61864: MPIR_Err_get_string (errutil.c:601) > ==14176== by 0x4DA4DB4: PMPI_Error_string (error_string.c:80) > ==14176== by 0x400888: main (in /home/dalcinl/Devel/BUGS-MPI/~/a.out) > ==14176== Address 0x0 is not stack'd, malloc'd or (recently) free'd > ==14176== > ==14176== > ==14176== Process terminating with default action of signal 11 (SIGSEGV) > ==14176== Access not within mapped region at address 0x0 > ==14176== at 0x4C6F5E7: MPIU_Strncpy (safestr.c:65) > ==14176== by 0x4C61864: MPIR_Err_get_string (errutil.c:601) > ==14176== by 0x4DA4DB4: PMPI_Error_string (error_string.c:80) > ==14176== by 0x400888: main (in /home/dalcinl/Devel/BUGS-MPI/~/a.out) > ==14176== If you believe this happened as a result of a stack > ==14176== overflow in your program's main thread (unlikely but > ==14176== possible), you can try to increase the size of the > ==14176== main thread stack using the --main-stacksize= flag. > ==14176== The main thread stack size used in this run was 8720384. > Segmentation fault (core dumped) > > > -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] Issue with MPI_Error_string() for user-defined codes/clases
by Rob Latham 22 Apr '14

22 Apr '14

On 04/22/2014 04:04 AM, Lisandro Dalcin wrote: > The MPI-3 standard says (pp.354, lines 39-40): > > """ > If MPI_ERROR_STRING is called when no string has been set, it will > return a empty > string (all spaces in Fortran, "" in C). > """ > > The following simple tests segfaults. A quick fix would be to return > an empty string in MPIR_Err_get_dynerr_string() (file dynerrutil.c), > e.g: Makes sense to me, but maybe Pavan has an issue with buffering? ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] Default print name for MPI-3 C++ datatypes
by Rob Latham 22 Apr '14

22 Apr '14

On 04/22/2014 07:10 AM, Lisandro Dalcin wrote: > The attached patch is so trivial I'm surprised this is not already > fixed. Any chance you missed this email? I pushed this to master just now. thanks for pinging the list again. ==rob > > On 9 April 2014 10:46, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >> Please review the following trivial patch (also attached), >> >> diff --git a/src/mpi/datatype/type_get_name.c b/src/mpi/datatype/type_get_name.c >> index 07c2f84..e550dfb 100644 >> --- a/src/mpi/datatype/type_get_name.c >> +++ b/src/mpi/datatype/type_get_name.c >> @@ -108,6 +108,12 @@ static mpi_names_t mpi_names[] = { >> type_name_entry(MPI_C_DOUBLE_COMPLEX), >> type_name_entry(MPI_C_LONG_DOUBLE_COMPLEX), >> >> + /* C++ types */ >> + type_name_entry(MPI_CXX_BOOL), >> + type_name_entry(MPI_CXX_FLOAT_COMPLEX), >> + type_name_entry(MPI_CXX_DOUBLE_COMPLEX), >> + type_name_entry(MPI_CXX_LONG_DOUBLE_COMPLEX), >> + >> /* address/offset types */ >> type_name_entry(MPI_AINT), >> type_name_entry(MPI_OFFSET), >> >> -- >> Lisandro Dalcin >> --------------- >> CIMEC (UNL/CONICET) >> Predio CONICET-Santa Fe >> Colectora RN 168 Km 472, Paraje El Pozo >> 3000 Santa Fe, Argentina >> Tel: +54-342-4511594 (ext 1016) >> Tel/Fax: +54-342-4511169 > > > -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] Default print name for MPI-3 C++ datatypes
by Jeff Hammond 22 Apr '14

22 Apr '14

If you file trac tickets, the chance of these type of things getting lost is smaller. Do you not have trac account? I thought you did before. Best, Jeff Sent from my iPhone > On Apr 22, 2014, at 7:10 AM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: > > The attached patch is so trivial I'm surprised this is not already > fixed. Any chance you missed this email? > >> On 9 April 2014 10:46, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >> Please review the following trivial patch (also attached), >> >> diff --git a/src/mpi/datatype/type_get_name.c b/src/mpi/datatype/type_get_name.c >> index 07c2f84..e550dfb 100644 >> --- a/src/mpi/datatype/type_get_name.c >> +++ b/src/mpi/datatype/type_get_name.c >> @@ -108,6 +108,12 @@ static mpi_names_t mpi_names[] = { >> type_name_entry(MPI_C_DOUBLE_COMPLEX), >> type_name_entry(MPI_C_LONG_DOUBLE_COMPLEX), >> >> + /* C++ types */ >> + type_name_entry(MPI_CXX_BOOL), >> + type_name_entry(MPI_CXX_FLOAT_COMPLEX), >> + type_name_entry(MPI_CXX_DOUBLE_COMPLEX), >> + type_name_entry(MPI_CXX_LONG_DOUBLE_COMPLEX), >> + >> /* address/offset types */ >> type_name_entry(MPI_AINT), >> type_name_entry(MPI_OFFSET), >> >> -- >> Lisandro Dalcin >> --------------- >> CIMEC (UNL/CONICET) >> Predio CONICET-Santa Fe >> Colectora RN 168 Km 472, Paraje El Pozo >> 3000 Santa Fe, Argentina >> Tel: +54-342-4511594 (ext 1016) >> Tel/Fax: +54-342-4511169 > > > > -- > Lisandro Dalcin > --------------- > CIMEC (UNL/CONICET) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1016) > Tel/Fax: +54-342-4511169

1 0

Re: [mpich-devel] Default print name for MPI-3 C++ datatypes
by Lisandro Dalcin 22 Apr '14

22 Apr '14

The attached patch is so trivial I'm surprised this is not already fixed. Any chance you missed this email? On 9 April 2014 10:46, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: > Please review the following trivial patch (also attached), > > diff --git a/src/mpi/datatype/type_get_name.c b/src/mpi/datatype/type_get_name.c > index 07c2f84..e550dfb 100644 > --- a/src/mpi/datatype/type_get_name.c > +++ b/src/mpi/datatype/type_get_name.c > @@ -108,6 +108,12 @@ static mpi_names_t mpi_names[] = { > type_name_entry(MPI_C_DOUBLE_COMPLEX), > type_name_entry(MPI_C_LONG_DOUBLE_COMPLEX), > > + /* C++ types */ > + type_name_entry(MPI_CXX_BOOL), > + type_name_entry(MPI_CXX_FLOAT_COMPLEX), > + type_name_entry(MPI_CXX_DOUBLE_COMPLEX), > + type_name_entry(MPI_CXX_LONG_DOUBLE_COMPLEX), > + > /* address/offset types */ > type_name_entry(MPI_AINT), > type_name_entry(MPI_OFFSET), > > -- > Lisandro Dalcin > --------------- > CIMEC (UNL/CONICET) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1016) > Tel/Fax: +54-342-4511169 -- Lisandro Dalcin --------------- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169

1 0