April 2018 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Smith, Barry F. 25 Apr '18

25 Apr '18

> On Apr 25, 2018, at 2:47 PM, Balay, Satish <balay(a)mcs.anl.gov> wrote: > > On Wed, 25 Apr 2018, Smith, Barry F. wrote: > >> >> >>> On Apr 25, 2018, at 2:31 PM, Jed Brown <jed(a)jedbrown.org> wrote: >>> >>> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >>> >>>>> On Apr 25, 2018, at 1:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: >>>>> >>>>> It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h >>>>> includes it as <mpi.h>, which means that users of MPIUNI need to put >>>>> -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree >>>>> that this is bad. We disagree on the solution. >>>>> >>>>> He wants to install it to /prefix/include/mpi.h as though the user had >>>>> written --download-mpich. This would conflict if a user later installs >>>>> a real MPI to that location. >>>> >>>> Jed, >>>> >>>> So you propose in petscsys.h ? >>>> >>>> #if defined(PETSC_HAVE_MPIUNI) >>>> #include <petsc/mpiuni/mpi.h> >>>> #else >>>> #include <mpi.h> >>>> #endif >>>> >>>> I don't have a problem with this. >>> >>> Yes, and same installation layout as today. >> >> Ok, this is far better than copying the mpiuni mpi.h file to a public place (Matt's suggestion) and is a bit better than requiring the extra -I flag (Satish's suggestion) >> > > All approaches have drawbacks. This approach breaks user code. I guess thats easy to work arround [and fix these > examples awell] > > $ git grep '<mpi\.h>' |grep examples > src/dm/examples/tests/ex42.c:#include <mpi.h> > src/tao/leastsquares/examples/tutorials/chwirut2.c:#include <mpi.h> Fix these examples at the same time. > > >>> >>>> I notice that the Fortran petscsys.h is scattered full of weird MPIUni specific stuff like >>>> >>>> #if defined (PETSC_HAVE_MPIUNI) >>>> #include "mpiunifdef.h" >>>> #endif >>>> >>>> #if defined(PETSC_HAVE_MPIUNI) >>>> #define MPI_Comm PetscFortranInt >>>> #define MPI_Group PetscFortranInt >>>> #define PetscMPIInt PetscFortranInt >>>> #else >>>> #define MPI_Comm integer >>>> #define MPI_Group integer >>>> #define PetscMPIInt integer >>>> #endif > > I might have diverged from mpi standard to prevent '-i8' affecting petsc+mpiuni code. Ah, yes > > Satish > >>>> >>>> It seems PETSc Fortran does not use the standard mpif.h file? >>> >>> Yuck. >> >> I don't remember the history of this construct. There must have been a reason years ago that may no longer exist, I don't know. >> >> Barry

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Satish Balay 25 Apr '18

25 Apr '18

On Wed, 25 Apr 2018, Smith, Barry F. wrote: > > > > On Apr 25, 2018, at 2:31 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > > > "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: > > > >>> On Apr 25, 2018, at 1:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: > >>> > >>> It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h > >>> includes it as <mpi.h>, which means that users of MPIUNI need to put > >>> -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree > >>> that this is bad. We disagree on the solution. > >>> > >>> He wants to install it to /prefix/include/mpi.h as though the user had > >>> written --download-mpich. This would conflict if a user later installs > >>> a real MPI to that location. > >> > >> Jed, > >> > >> So you propose in petscsys.h ? > >> > >> #if defined(PETSC_HAVE_MPIUNI) > >> #include <petsc/mpiuni/mpi.h> > >> #else > >> #include <mpi.h> > >> #endif > >> > >> I don't have a problem with this. > > > > Yes, and same installation layout as today. > > Ok, this is far better than copying the mpiuni mpi.h file to a public place (Matt's suggestion) and is a bit better than requiring the extra -I flag (Satish's suggestion) > All approaches have drawbacks. This approach breaks user code. I guess thats easy to work arround [and fix these examples awell] $ git grep '<mpi\.h>' |grep examples src/dm/examples/tests/ex42.c:#include <mpi.h> src/tao/leastsquares/examples/tutorials/chwirut2.c:#include <mpi.h> > > > >> I notice that the Fortran petscsys.h is scattered full of weird MPIUni specific stuff like > >> > >> #if defined (PETSC_HAVE_MPIUNI) > >> #include "mpiunifdef.h" > >> #endif > >> > >> #if defined(PETSC_HAVE_MPIUNI) > >> #define MPI_Comm PetscFortranInt > >> #define MPI_Group PetscFortranInt > >> #define PetscMPIInt PetscFortranInt > >> #else > >> #define MPI_Comm integer > >> #define MPI_Group integer > >> #define PetscMPIInt integer > >> #endif I might have diverged from mpi standard to prevent '-i8' affecting petsc+mpiuni code. Satish > >> > >> It seems PETSc Fortran does not use the standard mpif.h file? > > > > Yuck. > > I don't remember the history of this construct. There must have been a reason years ago that may no longer exist, I don't know. > > Barry > > >

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Smith, Barry F. 25 Apr '18

25 Apr '18

> On Apr 25, 2018, at 2:40 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: > >>> On Apr 25, 2018, at 2:31 PM, Jed Brown <jed(a)jedbrown.org> wrote: >>> >>> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >>> >>>>> On Apr 25, 2018, at 1:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: >>>>> >>>>> It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h >>>>> includes it as <mpi.h>, which means that users of MPIUNI need to put >>>>> -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree >>>>> that this is bad. We disagree on the solution. >>>>> >>>>> He wants to install it to /prefix/include/mpi.h as though the user had >>>>> written --download-mpich. This would conflict if a user later installs >>>>> a real MPI to that location. >>>> >>>> Jed, >>>> >>>> So you propose in petscsys.h ? >>>> >>>> #if defined(PETSC_HAVE_MPIUNI) >>>> #include <petsc/mpiuni/mpi.h> >>>> #else >>>> #include <mpi.h> >>>> #endif >>>> >>>> I don't have a problem with this. >>> >>> Yes, and same installation layout as today. >> >> Ok, this is far better than copying the mpiuni mpi.h file to a >> public place (Matt's suggestion) and is a bit better than requiring >> the extra -I flag (Satish's suggestion) > > Is this okay for 'maint'? It feels aggressive, but I'm having trouble > constructing a scenario in which it would break an existing installation > or existing code. Seems ok to me.

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Jed Brown 25 Apr '18

25 Apr '18

"Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >> On Apr 25, 2018, at 2:31 PM, Jed Brown <jed(a)jedbrown.org> wrote: >> >> "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >> >>>> On Apr 25, 2018, at 1:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: >>>> >>>> It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h >>>> includes it as <mpi.h>, which means that users of MPIUNI need to put >>>> -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree >>>> that this is bad. We disagree on the solution. >>>> >>>> He wants to install it to /prefix/include/mpi.h as though the user had >>>> written --download-mpich. This would conflict if a user later installs >>>> a real MPI to that location. >>> >>> Jed, >>> >>> So you propose in petscsys.h ? >>> >>> #if defined(PETSC_HAVE_MPIUNI) >>> #include <petsc/mpiuni/mpi.h> >>> #else >>> #include <mpi.h> >>> #endif >>> >>> I don't have a problem with this. >> >> Yes, and same installation layout as today. > > Ok, this is far better than copying the mpiuni mpi.h file to a > public place (Matt's suggestion) and is a bit better than requiring > the extra -I flag (Satish's suggestion) Is this okay for 'maint'? It feels aggressive, but I'm having trouble constructing a scenario in which it would break an existing installation or existing code.

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Smith, Barry F. 25 Apr '18

25 Apr '18

> On Apr 25, 2018, at 2:31 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > "Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: > >>> On Apr 25, 2018, at 1:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: >>> >>> It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h >>> includes it as <mpi.h>, which means that users of MPIUNI need to put >>> -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree >>> that this is bad. We disagree on the solution. >>> >>> He wants to install it to /prefix/include/mpi.h as though the user had >>> written --download-mpich. This would conflict if a user later installs >>> a real MPI to that location. >> >> Jed, >> >> So you propose in petscsys.h ? >> >> #if defined(PETSC_HAVE_MPIUNI) >> #include <petsc/mpiuni/mpi.h> >> #else >> #include <mpi.h> >> #endif >> >> I don't have a problem with this. > > Yes, and same installation layout as today. Ok, this is far better than copying the mpiuni mpi.h file to a public place (Matt's suggestion) and is a bit better than requiring the extra -I flag (Satish's suggestion) > >> I notice that the Fortran petscsys.h is scattered full of weird MPIUni specific stuff like >> >> #if defined (PETSC_HAVE_MPIUNI) >> #include "mpiunifdef.h" >> #endif >> >> #if defined(PETSC_HAVE_MPIUNI) >> #define MPI_Comm PetscFortranInt >> #define MPI_Group PetscFortranInt >> #define PetscMPIInt PetscFortranInt >> #else >> #define MPI_Comm integer >> #define MPI_Group integer >> #define PetscMPIInt integer >> #endif >> >> It seems PETSc Fortran does not use the standard mpif.h file? > > Yuck. I don't remember the history of this construct. There must have been a reason years ago that may no longer exist, I don't know. Barry

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Jed Brown 25 Apr '18

25 Apr '18

"Smith, Barry F." <bsmith(a)mcs.anl.gov> writes: >> On Apr 25, 2018, at 1:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: >> >> It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h >> includes it as <mpi.h>, which means that users of MPIUNI need to put >> -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree >> that this is bad. We disagree on the solution. >> >> He wants to install it to /prefix/include/mpi.h as though the user had >> written --download-mpich. This would conflict if a user later installs >> a real MPI to that location. > > Jed, > > So you propose in petscsys.h ? > > #if defined(PETSC_HAVE_MPIUNI) > #include <petsc/mpiuni/mpi.h> > #else > #include <mpi.h> > #endif > > I don't have a problem with this. Yes, and same installation layout as today. > I notice that the Fortran petscsys.h is scattered full of weird MPIUni specific stuff like > > #if defined (PETSC_HAVE_MPIUNI) > #include "mpiunifdef.h" > #endif > > #if defined(PETSC_HAVE_MPIUNI) > #define MPI_Comm PetscFortranInt > #define MPI_Group PetscFortranInt > #define PetscMPIInt PetscFortranInt > #else > #define MPI_Comm integer > #define MPI_Group integer > #define PetscMPIInt integer > #endif > > It seems PETSc Fortran does not use the standard mpif.h file? Yuck.

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Jed Brown 25 Apr '18

25 Apr '18

Satish Balay <balay(a)mcs.anl.gov> writes: > On Wed, 25 Apr 2018, Jed Brown wrote: > >> Satish Balay <balay(a)mcs.anl.gov> writes: >> >> > On Wed, 25 Apr 2018, Jed Brown wrote: >> > >> >> It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h >> >> includes it as <mpi.h>, which means that users of MPIUNI need to put >> >> -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree >> >> that this is bad. We disagree on the solution. >> > >> > Well in this context MPIUNI behaves like any other externalpackage. >> > >> >> He wants to install it to /prefix/include/mpi.h as though the user had >> >> written --download-mpich. This would conflict if a user later installs >> >> a real MPI to that location. >> > >> > We already have this conflict when switching between mpich and openmpi [so this issue would extend to mpiuni] >> >> You mean if someone --download-mpich and then manually installs Open MPI >> to the same prefix, overwiting the MPICH header? > > Yes [and more generally with non-prefix install] > >> >> Note that the option isn't --download-mpiuni, it's --with-mpi=0. > > Yes - and configure deals with it like externalpackage [wrt other dependencies] > >> >> >> I would rather that petscsys.h include <petsc/mpiuni/mpi.h> because it >> >> can't be used without PETSc and nobody who ever wrote #include <mpi.h> >> >> in their own code will be happy if they get MPIUNI. >> > >> > The reason for using supporting '#include <mpi.h>' - is so that user >> > codes that might use '#include <mpi.h>' would also work with mpiuni >> > build of petsc. >> >> If a user does this and doesn't link PETSc, it literally won't work. I >> conjecture that nobody that writes #include <mpi.h> in their code will >> be happy if they get MPIUNI. > > I'm not sure how many externalpackages work with mpiuni with '#include <mpi.h>' > >> > Yeah - tradeoffs for each choice - so we settled on the current one - >> > with petsc-makefile usage as the primary supported mode for users. >> > >> > This issue does not come up for such users. And non-petsc-makefile >> > users were to grab all the flags from petsc makefiles and use them >> > anyway. >> > >> > This issue is primarily comes up for users who expect the following to >> > work [which never worked portably] >> > >> > gcc -Ipath_to_petsc_include -Lpath_to_petsc_lib -lpetsc >> >> It works now (as long as you use shared libraries), modulo MPI. > > i.e its a small subset of all use cases [prefix install + shared libraires + all externalpackages are also shared] Which coincides with packaged installs. > Note: for regular non-prefix install - there is the extra -I$PETSC_DIR/$PETSC_ARCH/include [or you could say I$PETSC_DIR/include is the extra one that would have to be explicitly listed] You can just always give it (they refer to the same directory when PETSC_ARCH is unset) and then it works more generally. Requiring the path to MPIUNI is definitely an outlier.

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Smith, Barry F. 25 Apr '18

25 Apr '18

> On Apr 25, 2018, at 1:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h > includes it as <mpi.h>, which means that users of MPIUNI need to put > -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree > that this is bad. We disagree on the solution. > > He wants to install it to /prefix/include/mpi.h as though the user had > written --download-mpich. This would conflict if a user later installs > a real MPI to that location. Jed, So you propose in petscsys.h ? #if defined(PETSC_HAVE_MPIUNI) #include <petsc/mpiuni/mpi.h> #else #include <mpi.h> #endif I don't have a problem with this. I notice that the Fortran petscsys.h is scattered full of weird MPIUni specific stuff like #if defined (PETSC_HAVE_MPIUNI) #include "mpiunifdef.h" #endif #if defined(PETSC_HAVE_MPIUNI) #define MPI_Comm PetscFortranInt #define MPI_Group PetscFortranInt #define PetscMPIInt PetscFortranInt #else #define MPI_Comm integer #define MPI_Group integer #define PetscMPIInt integer #endif It seems PETSc Fortran does not use the standard mpif.h file? Barry > > I would rather that petscsys.h include <petsc/mpiuni/mpi.h> because it > can't be used without PETSc and nobody who ever wrote #include <mpi.h> > in their own code will be happy if they get MPIUNI.

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Jed Brown 25 Apr '18

25 Apr '18

Satish Balay <balay(a)mcs.anl.gov> writes: > If we really have to change - I'm inclined towards Matt's prefered approach. > > [issues with 'dirty builddir' is a separate problem I don't care about the build directory, I care about the install directory. It should be possible to install serial PETSc to prefix=/usr without conflicting with an MPI that may exist or may later exist. If you want to install something to the public namespace, the option would need to be --download-mpiuni which I think is less desirable for users. When they write --with-mpi=0, it's because they don't want MPI, not that they want to install a dysfunctional MPI. > - it comes up with switching between replaceable packages > [openmpi,mpich,mpiuni] - or switching versions of the same package] > > Satish > > On Wed, 25 Apr 2018, Matthew Knepley wrote: > >> On Wed, Apr 25, 2018 at 2:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: >> >> > It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h >> > includes it as <mpi.h>, which means that users of MPIUNI need to put >> > -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree >> > that this is bad. We disagree on the solution. >> > >> > He wants to install it to /prefix/include/mpi.h as though the user had >> > written --download-mpich. This would conflict if a user later installs >> > a real MPI to that location. >> > >> >> The idea should be that "installing" means that everyone is using >> compatible things >> from that location. If you allow an MPIUNI PETSc to be installed to a >> location that >> also has another MPI, what does mpiexec in ./bin do? There is just as much >> potential >> for confusion as there is when putting an MPICH PETSc in a location with >> OpenMPI >> installed. >> >> Matt >> >> >> > I would rather that petscsys.h include <petsc/mpiuni/mpi.h> because it >> > can't be used without PETSc and nobody who ever wrote #include <mpi.h> >> > in their own code will be happy if they get MPIUNI. >> > >> >> >> >>

1 0

Re: [petsc-dev] Install location for MPIUNI mpi.h
by Satish Balay 25 Apr '18

25 Apr '18

If we really have to change - I'm inclined towards Matt's prefered approach. [issues with 'dirty builddir' is a separate problem - it comes up with switching between replaceable packages [openmpi,mpich,mpiuni] - or switching versions of the same package] Satish On Wed, 25 Apr 2018, Matthew Knepley wrote: > On Wed, Apr 25, 2018 at 2:36 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > > It is currently installed to include/petsc/mpiuni/mpi.h and petscsys.h > > includes it as <mpi.h>, which means that users of MPIUNI need to put > > -I/prefix/include/petsc/mpiuni in their command lines. Matt and I agree > > that this is bad. We disagree on the solution. > > > > He wants to install it to /prefix/include/mpi.h as though the user had > > written --download-mpich. This would conflict if a user later installs > > a real MPI to that location. > > > > The idea should be that "installing" means that everyone is using > compatible things > from that location. If you allow an MPIUNI PETSc to be installed to a > location that > also has another MPI, what does mpiexec in ./bin do? There is just as much > potential > for confusion as there is when putting an MPICH PETSc in a location with > OpenMPI > installed. > > Matt > > > > I would rather that petscsys.h include <petsc/mpiuni/mpi.h> because it > > can't be used without PETSc and nobody who ever wrote #include <mpi.h> > > in their own code will be happy if they get MPIUNI. > > > > > >

1 0