December 2014 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Sean is going to love this
by Geoff Oxberry 31 Dec '14

31 Dec '14

On Tue, Dec 30, 2014 at 10:40 AM, Sean Farley <sean.michael.farley(a)gmail.com > wrote: > > Barry Smith writes: > > >> On Dec 29, 2014, at 9:56 PM, Geoff Oxberry <goxberry(a)gmail.com> wrote: > >> > >> > Date: Sun, 28 Dec 2014 21:27:21 -0600 > >> > From: Barry Smith <bsmith(a)mcs.anl.gov> > >> > To: Sean Farley <sean.michael.farley(a)gmail.com> > >> > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov> > >> > Subject: Re: [petsc-dev] Sean is going to love this > >> > Message-ID: <BA03ABA6-C693-4404-9FA5-2211BC371836(a)mcs.anl.gov> > >> > Content-Type: text/plain; charset="utf-8" > >> > > >> > > >> > Sean, > >> > > >> > brew install /homebrew/science/petsc > >> > brew install /homebrew/science/petsc --HEAD --with-x11 > >> > > >> > Is there any reason not to use home-brew for everything now? Should > we be working with the homebrew/science guys to work out the rough edges? > (Like their hypre build is sequential and they don't pass the MPI compiler > wrappers properly to build packages and ... and ...) > >> > >> I contributed most of the hypre recipe; brew install hypre --with-mpi > should get you a parallel build, and it should also run some parallel > exampled as smoke tests. If there's anything missing that should be added, > let me know and I can get to it when I return from vacation. > > > > Geoff, > > > > Thanks, I noticed your name on a bunch of things there. I may bother > you later with some issues when I understand brew better but, what the > heck, here are a few things I noticed. (By the way, overall I was impressed > with brew working with all these "nasty" scientific packages) > > No love for the work I did with all the nasty scientific packages? Oh, > ok :-( > > > 1) I ran brew install /homebrew/science/petsc --HEAD --with-hypre and > it failed because it just used the bottled hypre (which is sequential) as a > dependent package and then tried to build the parallel PETSc with the > sequential hypre so failed in the middle of compiling PETSc because of > inconsistent hypre include files (ones for sequential). > > Brew bottles were originally only used in situations where building from source would take a long time, and under the assumption that most users are interested in a "standard" build without any command-line options. Recently, this idea seems to have been extended to most brew formula. I don't think brew bottles make sense for something like hypre, which is relatively quick to build, because the convenience savings are negligible. > Yep, this is a problem that doesn't exist for PETSc in MacPorts. > > > Presumably had I done a brew install hypre --with-mpi first and then > the brew install /homebrew/science/petsc --HEAD it may have worked > properly? (but how is a naive new user to know?) > > Indeed. > You're right. The PETSc formula should require that if hypre is used, it must have been built with the "--with-mpi" flag. This issue would have tripped me up also. I will add a GitHub issue for this point: https://github.com/Homebrew/homebrew-science/issues/1602 > This seems to be related to my first meta issue which is how is brew > really suppose to work with versions of packages (and build option versions > for dependent packages)? For example petsc 3.5.2 uses certain versions of > other packages such as mumps, superlu_dist etc but petsc --HEAD (which will > become PETSc 3.6) may work with later versions of some of the packages. How > is this suppose to be indicated, handled, etc? I couldn't see if brew had > the concept of handling multiple versions of packages and different version > dependencies for each version? Brew formulas can express package dependencies that include install flags; https://github.com/Homebrew/homebrew/blob/master/Library/Contributions/exam… <https://github.com/Homebrew/homebrew/blob/master/Library/Contributions/exam…> is a comprehensive example of what can and can't be done. Indicating version dependencies is problematic. The brew package model assumes users want the latest versions of packages, which has led to sets of formulas like {python, python3} in the main homebrew repo, and {metis, metis4} in cases where packages depend on specific versions. So brew does not have a sophisticated way of handling packages with multiple version numbers. If one has used brew to install different versions of a package over time, such as installing `open-mpi` version 1.7.4, then later upgrading to version 1.8.1, the `brew-switch` command can be used to switch among versions (e.g., `brew switch open-mpi 1.7.4` or `brew switch open-mpi 1.8.1` in this example, depending on which version is currently active). `brew info` can be used to query the currently active version (e.g., `brew info open-mpi`). If PETSc only uses the latest versions of packages, the best way forward is probably to update the formulas for PETSc dependencies. If PETSc is tied to specific package versions, then it might be better to have a separate PETSc tap, with namespaced dependencies that don't conflict with homebrew-science. Dominique Orban was working on a FEniCS tap for a while ( https://github.com/optimizers/homebrew-fenics) I had mixed success when using it. Since Sean is going to say it if I don't, yes, I think MacPorts does a better job indicating version dependencies, and Homebrew has room for improvement here. > > > Second meta issue, is there a way to force a compiled of the package > instead of it using the prepared bottle? Or even better to just tell brew I never want it to use bottles but always > want it to compile for all packages? One reason I ask this is because PETSc > has so many optional dependencies it seems maybe? better to just have it > build based on the choices the user made rather than using some pre-built > bottle that only has certain things turned on? But maybe I don't understand > bottles and when they are used instead of building. > https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/FAQ.md#… To summarize, a formula will build from source if: - any installation option is added (e.g., `brew install petsc --HEAD`) - the environment variable HOMEBREW_BUILD_FROM_SOURCE is set - the option `--build-from-source` is invoked (e.g., `brew install --build-from-source hypre`) > This is done in MacPorts. > > > 2) I think PETSc is just "lucky" that it builds at all. brew seems to > set the environmental variables CC, CXX, and FC to clang, clang++ and > gfortran but then PETSc actually ignores these variables (printing a > warning if you run with -vd) and hunts for mpicc, mpicxx, mpifc which it > happens to find in /usr/local/bin since open-mpi is a dependency of PETSc > and thus must have been installed first. > > This is why sandboxing is important for package managers. > I agree with Sean here. Brew sets a bunch of environmental variables in something it calls superenv; the rationale is given in https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/Homebre…. Much of the code for the sandboxing is in https://github.com/Homebrew/homebrew/tree/master/Library/Homebrew/extend, in `ENV.rb` or the `ENV` directory. > > So this leads the third meta issue, it seems brew doesn't seem to > have any concept of some packages requiring mpi wrapper compilers to be > used to compile the package and so it is kind of catch as catch can if MPI > based packages will build properly instead of having a systematic way of > handling it. For example if PETSc did not ignore CC, CXX, and FC it would > try to use clang, clang++ and gfortran and then crap out that it didn't > have MPI compilers. Any chance that brew would ever be smart enough to > "reset" the compilers to the MPI ones before building packages that depend > on MPI? > I think the next step is to improve support for MPI wrapper compilers within formulas of packages that depend on MPI or have MPI options. I can look at MacPorts and BuildSystem for inspiration here to see what could be done. > > A hard graph problem that I hacked in MacPorts. I have no idea about how > to do it in brew (hence why I haven't used it). > > > 3) PETSc has a bunch of other optional dependencies such as hdf5, netcdf > ... that it would be nice to support. Presumably I could try to make a pull > request with support for them but I'm concerned that my lack of > understanding of meta issue two might mess me up. > > All of those are supported in MacPorts. > As long as formulas exist (or are created) for these optional dependencies, I think adding support for them is doable, and probably straightforward. > > > 4) The issue of MPICH2 and open-mpi conflicting is a pain. Some package > managers handle this by having a concept of a meta-package or > abstract-package (such as MPI) that can have multiple implementation. Then, > for example, one could say PETSc is dependent on MPI (and not open-mpi or > MPICH) and the package system would work properly if either open-mpi or > MPICH had already been installed (instead of crapping out like it does if I > first install MPICH and then install PETSc; since it tries to install > open-mpi for PETSc which conflicts with MPICH and thus stops). > > Also, this is done in MacPorts. > Brew does not handle open-mpi vs MPICH well. An inelegant solution that would probably work would be to hardcode paths for each MPI implementation. If the only implementations used are open-mpi and MPICH, this solution is expedient and probably fine. I started along this path for MUMPS in a pull request (https://github.com/Homebrew/homebrew-science/pull/771) that brought up relevant discussion with Jed, Aron, and Dominique Orban. One issue arising with brew is that if a package like `mumps` is build with `mpich2`, and then `mpich2` is uninstalled and replaced with `open-mpi`, brew should rebuild `mumps`. Right now, it won't force a rebuild like that. Also, the `:mpi` dependency only resolves to `open-mpi`. I think it would be worth revisiting this issue to point out the demand for better MPI dependency tracking and resolution. > > With that being said, I must also declare that I have no love for > MacPorts. It gets the job done for these specific cases and works just > well enough but still has other problems. MacPorts will never be able to > solve harder DAG problems for dependencies (just hacks). I wish I had > the time to build a new package manager, but alas. The quote from ESR > seems to ring true: > MacPorts is okay. Sean's put a lot of work into it, which I appreciate, and I've used MacPorts in the past. I eventually found it hard to maintain because duplicating system programs into their own sandbox required environment hacking to make sure I got the right versions on my PATH. I found it more difficult to maintain. Since Homebrew avoids duplicating system utilities (e.g., compilers), I find it easier to maintain a cleaner system setup. I also found it difficult to contribute to MacPorts. I'd rather use Git and Ruby over SVN and TCL; in both cases, the former technology is more widely used than the latter. Getting a pull request accepted as a new contributed to Homebrew was easy; the same did not appear to be true for MacPorts. I think these reasons might contribute to why MacPorts has 194 contributors ( https://www.openhub.net/p/MacPorts) and Homebrew has 5,034 contributors ( https://www.openhub.net/p/homebrew) Last I checked, Homebrew was the 12th-most starred and 9th-most forked repo on GitHub. >From a feature standpoint, I think MacPorts does a lot of things feature-wise that Homebrew should be doing, so I take it as a guide for how to improve Homebrew. > > The most dangerous enemy of a better solution is an existing codebase > that is just good enough. > -- Geoffrey Oxberry, Ph.D., E.I.T. goxberry(a)gmail.com

1 0

Re: [petsc-dev] Better package manager support in PETSc
by Barry Smith 30 Dec '14

30 Dec '14

Thanks. The $PREFIX/include/makefile is just a hiccup from the installer and should never be installed. I have fixed next and master to not copy it. In fact there were a truckload of include/*/*/makefile that should never have been copied over,. FIXED Another bad copy of install stuff not mentioned is ls ~/petscinstall/include/finclude/ftn-auto/ -tmpdir pcexotic-tmpdir ao-tmpdir pcgasm-tmpdir petscthreadcomm-tmpdir dm-tmpdir pcmg-tmpdir dmcomposite-tmpdir .... i.e. tons of work directories that should never have been copied over. FIXED The other complaint is $PREFIX/lib/modules/3.4.1 but in fact there are many other things that should be complained about as well. include/finclude -nonname-spaced directory which includes two additional nonname-spaced sub-directories fnt-auto and fnt-custom WHAT SHOULD THIS STUFF BE CALLED? all subdirectories of bin get copied over including unnamed spaced maint and julia plus lots of silly stuff gets installed that should not be installed. I WILL FIX THIS ls ~/petscinstall/bin/ FASTMathInstaller.py julia mpiexec.sshsync popup uncrustify.cfg TOPSGenerator.py maint mpiexec.uni portabilitycheck.py update.py adiforfix.py mapnames parseargs.py pstogif urlget mpiexec.gmalloc petsclogformat.py pstoxbm urlget.py mpiexec.lam petscmpiexec pythonscripts win32fe configVars.py mpiexec.llrun petscnagupgrade.py saws mpiexec.poe petscnagupgrade.pyc taucc.py mpiexec.prun petscrun The famous conf directory gets copied over > On Dec 30, 2014, at 4:00 PM, Aron Ahmadia <aron(a)ahmadia.net> wrote: > > > On Tue, Dec 30, 2014 at 4:54 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Tell us what we need to fix. > > The issues were brought up in: http://lists.mcs.anl.gov/pipermail/petsc-dev/2013-October/013420.html Hi All, The PR has raised a few issues in what PETSc does when it performs a prefix install. In particular, PETSc generates the following files: $PREFIX/include/makefile $PREFIX/lib/modules/3.4.1 Neither of these have been properly namespaced. I'd suggest changing these to: $PREFIX/include/petsc/makefile $PREFIX/lib/modules/petsc/3.4.1 I'm not sure about the latter, I really don't know how package managers deal with local modules install, or if it's worth even putting it in if the user is requesting a PREFIX install. Another option would be to include the PETSc ARCH in the directory namespacing: $PREFIX/include/petsc/linux-gnu/makefile $PREFIX/lib/modules/petsc/linux-gnu/3.4.1 Thoughts? -A > > I don't know how you track these issues internally, but it didn't seem to be resolved when I looked at the mailing list. > > A

1 0

Re: [petsc-dev] Better package manager support in PETSc
by Aron Ahmadia 30 Dec '14

30 Dec '14

On Tue, Dec 30, 2014 at 4:54 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > Tell us what we need to fix. > The issues were brought up in: http://lists.mcs.anl.gov/pipermail/petsc-dev/2013-October/013420.html I don't know how you track these issues internally, but it didn't seem to be resolved when I looked at the mailing list. A

1 0

Re: [petsc-dev] Better package manager support in PETSc
by Barry Smith 30 Dec '14

30 Dec '14

> On Dec 30, 2014, at 1:26 PM, Aron Ahmadia <aron(a)ahmadia.net> wrote: > > From Sean's list of annoying things: > > 1) Non-standard prefix > > Yup, this is one of the most annoying things for us right now. Tell us what we need to fix. > > 4) Better coordination with dependent packages > > This is really work we just need to do at some point and haven't done. > > Is anybody building PETSc on Windows or It builds with cygwin, with either gnu compilers or commercial compilers but the usual caveats of cygwin being a pain. Tech-x allegedly is going to provide an installer that plays nicely with Developer Studio any day now for commercial users. > exotic non-supercomputer devices these days? What does this mean? We have an installer for Xcode for iPhone/iPad development. There is actually an iPhone/iPad app that uses PETSc (guess what it is). > > A > > On Tue, Dec 30, 2014 at 1:40 PM, Sean Farley <sean.michael.farley(a)gmail.com> wrote: > > Barry Smith writes: > > >> On Dec 29, 2014, at 9:56 PM, Geoff Oxberry <goxberry(a)gmail.com> wrote: > >> > >> > Date: Sun, 28 Dec 2014 21:27:21 -0600 > >> > From: Barry Smith <bsmith(a)mcs.anl.gov> > >> > To: Sean Farley <sean.michael.farley(a)gmail.com> > >> > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov> > >> > Subject: Re: [petsc-dev] Sean is going to love this > >> > Message-ID: <BA03ABA6-C693-4404-9FA5-2211BC371836(a)mcs.anl.gov> > >> > Content-Type: text/plain; charset="utf-8" > >> > > >> > > >> > Sean, > >> > > >> > brew install /homebrew/science/petsc > >> > brew install /homebrew/science/petsc --HEAD --with-x11 > >> > > >> > Is there any reason not to use home-brew for everything now? Should we be working with the homebrew/science guys to work out the rough edges? (Like their hypre build is sequential and they don't pass the MPI compiler wrappers properly to build packages and ... and ...) > >> > >> I contributed most of the hypre recipe; brew install hypre --with-mpi should get you a parallel build, and it should also run some parallel exampled as smoke tests. If there's anything missing that should be added, let me know and I can get to it when I return from vacation. > > > > Geoff, > > > > Thanks, I noticed your name on a bunch of things there. I may bother you later with some issues when I understand brew better but, what the heck, here are a few things I noticed. (By the way, overall I was impressed with brew working with all these "nasty" scientific packages) > > No love for the work I did with all the nasty scientific packages? Oh, > ok :-( > > > 1) I ran brew install /homebrew/science/petsc --HEAD --with-hypre and it failed because it just used the bottled hypre (which is sequential) as a dependent package and then tried to build the parallel PETSc with the sequential hypre so failed in the middle of compiling PETSc because of inconsistent hypre include files (ones for sequential). > > Yep, this is a problem that doesn't exist for PETSc in MacPorts. > > > Presumably had I done a brew install hypre --with-mpi first and then the brew install /homebrew/science/petsc --HEAD it may have worked properly? (but how is a naive new user to know?) > > Indeed. > > > Second meta issue, is there a way to force a compiled of the package instead of it using the prepared bottle? Or even better to just tell brew I never want it to use bottles but always want it to compile for all packages? One reason I ask this is because PETSc has so many optional dependencies it seems maybe? better to just have it build based on the choices the user made rather than using some pre-built bottle that only has certain things turned on? But maybe I don't understand bottles and when they are used instead of building. > > This is done in MacPorts. > > > 2) I think PETSc is just "lucky" that it builds at all. brew seems to set the environmental variables CC, CXX, and FC to clang, clang++ and gfortran but then PETSc actually ignores these variables (printing a warning if you run with -vd) and hunts for mpicc, mpicxx, mpifc which it happens to find in /usr/local/bin since open-mpi is a dependency of PETSc and thus must have been installed first. > > This is why sandboxing is important for package managers. > > > So this leads the third meta issue, it seems brew doesn't seem to have any concept of some packages requiring mpi wrapper compilers to be used to compile the package and so it is kind of catch as catch can if MPI based packages will build properly instead of having a systematic way of handling it. For example if PETSc did not ignore CC, CXX, and FC it would try to use clang, clang++ and gfortran and then crap out that it didn't have MPI compilers. Any chance that brew would ever be smart enough to "reset" the compilers to the MPI ones before building packages that depend on MPI? > > A hard graph problem that I hacked in MacPorts. I have no idea about how > to do it in brew (hence why I haven't used it). > > > 3) PETSc has a bunch of other optional dependencies such as hdf5, netcdf ... that it would be nice to support. Presumably I could try to make a pull request with support for them but I'm concerned that my lack of understanding of meta issue two might mess me up. > > All of those are supported in MacPorts. > > > 4) The issue of MPICH2 and open-mpi conflicting is a pain. Some package managers handle this by having a concept of a meta-package or abstract-package (such as MPI) that can have multiple implementation. Then, for example, one could say PETSc is dependent on MPI (and not open-mpi or MPICH) and the package system would work properly if either open-mpi or MPICH had already been installed (instead of crapping out like it does if I first install MPICH and then install PETSc; since it tries to install open-mpi for PETSc which conflicts with MPICH and thus stops). > > Also, this is done in MacPorts. > > With that being said, I must also declare that I have no love for > MacPorts. It gets the job done for these specific cases and works just > well enough but still has other problems. MacPorts will never be able to > solve harder DAG problems for dependencies (just hacks). I wish I had > the time to build a new package manager, but alas. The quote from ESR > seems to ring true: > > The most dangerous enemy of a better solution is an existing codebase > that is just good enough. >

1 0

GAMG coarsening in-place
by Jed Brown 30 Dec '14

30 Dec '14

I have a pretty well-conditioned problem, eigenvalues in (0.2, 1.55) with bjacobi/ilu. It converges in 20-25 iterations with GMRES or with Chebyshev targeting this range (eigenvalues are almost uniformly distributed). I'd like to make an attempt to do better using GAMG, but the mass term is big enough that I don't want a real coarse grid. Instead, I want to coarsen only once or twice "in-place" and not have any real coarse grid. But it looks like GAMG is hardwired to put the coarsest grid on one process. How should we add this flexibility?

1 0

Better package manager support in PETSc
by Aron Ahmadia 30 Dec '14

30 Dec '14

>From Sean's list of annoying things: 1) Non-standard prefix Yup, this is one of the most annoying things for us right now. 4) Better coordination with dependent packages This is really work we just need to do at some point and haven't done. Is anybody building PETSc on Windows or exotic non-supercomputer devices these days? A On Tue, Dec 30, 2014 at 1:40 PM, Sean Farley <sean.michael.farley(a)gmail.com> wrote: > > Barry Smith writes: > > >> On Dec 29, 2014, at 9:56 PM, Geoff Oxberry <goxberry(a)gmail.com> wrote: > >> > >> > Date: Sun, 28 Dec 2014 21:27:21 -0600 > >> > From: Barry Smith <bsmith(a)mcs.anl.gov> > >> > To: Sean Farley <sean.michael.farley(a)gmail.com> > >> > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov> > >> > Subject: Re: [petsc-dev] Sean is going to love this > >> > Message-ID: <BA03ABA6-C693-4404-9FA5-2211BC371836(a)mcs.anl.gov> > >> > Content-Type: text/plain; charset="utf-8" > >> > > >> > > >> > Sean, > >> > > >> > brew install /homebrew/science/petsc > >> > brew install /homebrew/science/petsc --HEAD --with-x11 > >> > > >> > Is there any reason not to use home-brew for everything now? Should > we be working with the homebrew/science guys to work out the rough edges? > (Like their hypre build is sequential and they don't pass the MPI compiler > wrappers properly to build packages and ... and ...) > >> > >> I contributed most of the hypre recipe; brew install hypre --with-mpi > should get you a parallel build, and it should also run some parallel > exampled as smoke tests. If there's anything missing that should be added, > let me know and I can get to it when I return from vacation. > > > > Geoff, > > > > Thanks, I noticed your name on a bunch of things there. I may bother > you later with some issues when I understand brew better but, what the > heck, here are a few things I noticed. (By the way, overall I was impressed > with brew working with all these "nasty" scientific packages) > > No love for the work I did with all the nasty scientific packages? Oh, > ok :-( > > > 1) I ran brew install /homebrew/science/petsc --HEAD --with-hypre and > it failed because it just used the bottled hypre (which is sequential) as a > dependent package and then tried to build the parallel PETSc with the > sequential hypre so failed in the middle of compiling PETSc because of > inconsistent hypre include files (ones for sequential). > > Yep, this is a problem that doesn't exist for PETSc in MacPorts. > > > Presumably had I done a brew install hypre --with-mpi first and then > the brew install /homebrew/science/petsc --HEAD it may have worked > properly? (but how is a naive new user to know?) > > Indeed. > > > Second meta issue, is there a way to force a compiled of the package > instead of it using the prepared bottle? Or even better to just tell brew I > never want it to use bottles but always want it to compile for all > packages? One reason I ask this is because PETSc has so many optional > dependencies it seems maybe? better to just have it build based on the > choices the user made rather than using some pre-built bottle that only has > certain things turned on? But maybe I don't understand bottles and when > they are used instead of building. > > This is done in MacPorts. > > > 2) I think PETSc is just "lucky" that it builds at all. brew seems to > set the environmental variables CC, CXX, and FC to clang, clang++ and > gfortran but then PETSc actually ignores these variables (printing a > warning if you run with -vd) and hunts for mpicc, mpicxx, mpifc which it > happens to find in /usr/local/bin since open-mpi is a dependency of PETSc > and thus must have been installed first. > > This is why sandboxing is important for package managers. > > > So this leads the third meta issue, it seems brew doesn't seem to > have any concept of some packages requiring mpi wrapper compilers to be > used to compile the package and so it is kind of catch as catch can if MPI > based packages will build properly instead of having a systematic way of > handling it. For example if PETSc did not ignore CC, CXX, and FC it would > try to use clang, clang++ and gfortran and then crap out that it didn't > have MPI compilers. Any chance that brew would ever be smart enough to > "reset" the compilers to the MPI ones before building packages that depend > on MPI? > > A hard graph problem that I hacked in MacPorts. I have no idea about how > to do it in brew (hence why I haven't used it). > > > 3) PETSc has a bunch of other optional dependencies such as hdf5, netcdf > ... that it would be nice to support. Presumably I could try to make a pull > request with support for them but I'm concerned that my lack of > understanding of meta issue two might mess me up. > > All of those are supported in MacPorts. > > > 4) The issue of MPICH2 and open-mpi conflicting is a pain. Some package > managers handle this by having a concept of a meta-package or > abstract-package (such as MPI) that can have multiple implementation. Then, > for example, one could say PETSc is dependent on MPI (and not open-mpi or > MPICH) and the package system would work properly if either open-mpi or > MPICH had already been installed (instead of crapping out like it does if I > first install MPICH and then install PETSc; since it tries to install > open-mpi for PETSc which conflicts with MPICH and thus stops). > > Also, this is done in MacPorts. > > With that being said, I must also declare that I have no love for > MacPorts. It gets the job done for these specific cases and works just > well enough but still has other problems. MacPorts will never be able to > solve harder DAG problems for dependencies (just hacks). I wish I had > the time to build a new package manager, but alas. The quote from ESR > seems to ring true: > > The most dangerous enemy of a better solution is an existing codebase > that is just good enough. >

1 0

Re: [petsc-dev] Sean is going to love this
by Sean Farley 30 Dec '14

30 Dec '14

Barry Smith writes: >> On Dec 29, 2014, at 9:56 PM, Geoff Oxberry <goxberry(a)gmail.com> wrote: >> >> > Date: Sun, 28 Dec 2014 21:27:21 -0600 >> > From: Barry Smith <bsmith(a)mcs.anl.gov> >> > To: Sean Farley <sean.michael.farley(a)gmail.com> >> > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov> >> > Subject: Re: [petsc-dev] Sean is going to love this >> > Message-ID: <BA03ABA6-C693-4404-9FA5-2211BC371836(a)mcs.anl.gov> >> > Content-Type: text/plain; charset="utf-8" >> > >> > >> > Sean, >> > >> > brew install /homebrew/science/petsc >> > brew install /homebrew/science/petsc --HEAD --with-x11 >> > >> > Is there any reason not to use home-brew for everything now? Should we be working with the homebrew/science guys to work out the rough edges? (Like their hypre build is sequential and they don't pass the MPI compiler wrappers properly to build packages and ... and ...) >> >> I contributed most of the hypre recipe; brew install hypre --with-mpi should get you a parallel build, and it should also run some parallel exampled as smoke tests. If there's anything missing that should be added, let me know and I can get to it when I return from vacation. > > Geoff, > > Thanks, I noticed your name on a bunch of things there. I may bother you later with some issues when I understand brew better but, what the heck, here are a few things I noticed. (By the way, overall I was impressed with brew working with all these "nasty" scientific packages) No love for the work I did with all the nasty scientific packages? Oh, ok :-( > 1) I ran brew install /homebrew/science/petsc --HEAD --with-hypre and it failed because it just used the bottled hypre (which is sequential) as a dependent package and then tried to build the parallel PETSc with the sequential hypre so failed in the middle of compiling PETSc because of inconsistent hypre include files (ones for sequential). Yep, this is a problem that doesn't exist for PETSc in MacPorts. > Presumably had I done a brew install hypre --with-mpi first and then the brew install /homebrew/science/petsc --HEAD it may have worked properly? (but how is a naive new user to know?) Indeed. > Second meta issue, is there a way to force a compiled of the package instead of it using the prepared bottle? Or even better to just tell brew I never want it to use bottles but always want it to compile for all packages? One reason I ask this is because PETSc has so many optional dependencies it seems maybe? better to just have it build based on the choices the user made rather than using some pre-built bottle that only has certain things turned on? But maybe I don't understand bottles and when they are used instead of building. This is done in MacPorts. > 2) I think PETSc is just "lucky" that it builds at all. brew seems to set the environmental variables CC, CXX, and FC to clang, clang++ and gfortran but then PETSc actually ignores these variables (printing a warning if you run with -vd) and hunts for mpicc, mpicxx, mpifc which it happens to find in /usr/local/bin since open-mpi is a dependency of PETSc and thus must have been installed first. This is why sandboxing is important for package managers. > So this leads the third meta issue, it seems brew doesn't seem to have any concept of some packages requiring mpi wrapper compilers to be used to compile the package and so it is kind of catch as catch can if MPI based packages will build properly instead of having a systematic way of handling it. For example if PETSc did not ignore CC, CXX, and FC it would try to use clang, clang++ and gfortran and then crap out that it didn't have MPI compilers. Any chance that brew would ever be smart enough to "reset" the compilers to the MPI ones before building packages that depend on MPI? A hard graph problem that I hacked in MacPorts. I have no idea about how to do it in brew (hence why I haven't used it). > 3) PETSc has a bunch of other optional dependencies such as hdf5, netcdf ... that it would be nice to support. Presumably I could try to make a pull request with support for them but I'm concerned that my lack of understanding of meta issue two might mess me up. All of those are supported in MacPorts. > 4) The issue of MPICH2 and open-mpi conflicting is a pain. Some package managers handle this by having a concept of a meta-package or abstract-package (such as MPI) that can have multiple implementation. Then, for example, one could say PETSc is dependent on MPI (and not open-mpi or MPICH) and the package system would work properly if either open-mpi or MPICH had already been installed (instead of crapping out like it does if I first install MPICH and then install PETSc; since it tries to install open-mpi for PETSc which conflicts with MPICH and thus stops). Also, this is done in MacPorts. With that being said, I must also declare that I have no love for MacPorts. It gets the job done for these specific cases and works just well enough but still has other problems. MacPorts will never be able to solve harder DAG problems for dependencies (just hacks). I wish I had the time to build a new package manager, but alas. The quote from ESR seems to ring true: The most dangerous enemy of a better solution is an existing codebase that is just good enough.

1 0

Re: [petsc-dev] Sean is going to love this
by Barry Smith 30 Dec '14

30 Dec '14

> On Dec 29, 2014, at 9:56 PM, Geoff Oxberry <goxberry(a)gmail.com> wrote: > > > Date: Sun, 28 Dec 2014 21:27:21 -0600 > > From: Barry Smith <bsmith(a)mcs.anl.gov> > > To: Sean Farley <sean.michael.farley(a)gmail.com> > > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov> > > Subject: Re: [petsc-dev] Sean is going to love this > > Message-ID: <BA03ABA6-C693-4404-9FA5-2211BC371836(a)mcs.anl.gov> > > Content-Type: text/plain; charset="utf-8" > > > > > > Sean, > > > > brew install /homebrew/science/petsc > > brew install /homebrew/science/petsc --HEAD --with-x11 > > > > Is there any reason not to use home-brew for everything now? Should we be working with the homebrew/science guys to work out the rough edges? (Like their hypre build is sequential and they don't pass the MPI compiler wrappers properly to build packages and ... and ...) > > I contributed most of the hypre recipe; brew install hypre --with-mpi should get you a parallel build, and it should also run some parallel exampled as smoke tests. If there's anything missing that should be added, let me know and I can get to it when I return from vacation. Geoff, Thanks, I noticed your name on a bunch of things there. I may bother you later with some issues when I understand brew better but, what the heck, here are a few things I noticed. (By the way, overall I was impressed with brew working with all these "nasty" scientific packages) 1) I ran brew install /homebrew/science/petsc --HEAD --with-hypre and it failed because it just used the bottled hypre (which is sequential) as a dependent package and then tried to build the parallel PETSc with the sequential hypre so failed in the middle of compiling PETSc because of inconsistent hypre include files (ones for sequential). Presumably had I done a brew install hypre --with-mpi first and then the brew install /homebrew/science/petsc --HEAD it may have worked properly? (but how is a naive new user to know?) This seems to be related to my first meta issue which is how is brew really suppose to work with versions of packages (and build option versions for dependent packages)? For example petsc 3.5.2 uses certain versions of other packages such as mumps, superlu_dist etc but petsc --HEAD (which will become PETSc 3.6) may work with later versions of some of the packages. How is this suppose to be indicated, handled, etc? I couldn't see if brew had the concept of handling multiple versions of packages and different version dependencies for each version? Second meta issue, is there a way to force a compiled of the package instead of it using the prepared bottle? Or even better to just tell brew I never want it to use bottles but always want it to compile for all packages? One reason I ask this is because PETSc has so many optional dependencies it seems maybe? better to just have it build based on the choices the user made rather than using some pre-built bottle that only has certain things turned on? But maybe I don't understand bottles and when they are used instead of building. 2) I think PETSc is just "lucky" that it builds at all. brew seems to set the environmental variables CC, CXX, and FC to clang, clang++ and gfortran but then PETSc actually ignores these variables (printing a warning if you run with -vd) and hunts for mpicc, mpicxx, mpifc which it happens to find in /usr/local/bin since open-mpi is a dependency of PETSc and thus must have been installed first. So this leads the third meta issue, it seems brew doesn't seem to have any concept of some packages requiring mpi wrapper compilers to be used to compile the package and so it is kind of catch as catch can if MPI based packages will build properly instead of having a systematic way of handling it. For example if PETSc did not ignore CC, CXX, and FC it would try to use clang, clang++ and gfortran and then crap out that it didn't have MPI compilers. Any chance that brew would ever be smart enough to "reset" the compilers to the MPI ones before building packages that depend on MPI? 3) PETSc has a bunch of other optional dependencies such as hdf5, netcdf ... that it would be nice to support. Presumably I could try to make a pull request with support for them but I'm concerned that my lack of understanding of meta issue two might mess me up. 4) The issue of MPICH2 and open-mpi conflicting is a pain. Some package managers handle this by having a concept of a meta-package or abstract-package (such as MPI) that can have multiple implementation. Then, for example, one could say PETSc is dependent on MPI (and not open-mpi or MPICH) and the package system would work properly if either open-mpi or MPICH had already been installed (instead of crapping out like it does if I first install MPICH and then install PETSc; since it tries to install open-mpi for PETSc which conflicts with MPICH and thus stops). Thanks Barry

1 0

Re: [petsc-dev] Sean is going to love this
by Geoff Oxberry 29 Dec '14

29 Dec '14

> Date: Sun, 28 Dec 2014 21:27:21 -0600 > From: Barry Smith <bsmith(a)mcs.anl.gov> > To: Sean Farley <sean.michael.farley(a)gmail.com> > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov> > Subject: Re: [petsc-dev] Sean is going to love this > Message-ID: <BA03ABA6-C693-4404-9FA5-2211BC371836(a)mcs.anl.gov> > Content-Type: text/plain; charset="utf-8" > > > Sean, > > brew install /homebrew/science/petsc > brew install /homebrew/science/petsc --HEAD --with-x11 > > Is there any reason not to use home-brew for everything now? Should we be working with the homebrew/science guys to work out the rough edges? (Like their hypre build is sequential and they don't pass the MPI compiler wrappers properly to build packages and ... and ...) I contributed most of the hypre recipe; brew install hypre --with-mpi should get you a parallel build, and it should also run some parallel exampled as smoke tests. If there's anything missing that should be added, let me know and I can get to it when I return from vacation.

1 0

Re: [petsc-dev] Sean is going to love this
by Sean Farley 28 Dec '14

28 Dec '14

Barry Smith writes: > Sean, > > brew install /homebrew/science/petsc > brew install /homebrew/science/petsc --HEAD --with-x11 > > Is there any reason not to use home-brew for everything now? Should we be working with the homebrew/science guys to work out the rough edges? (Like their hypre build is sequential and they don't pass the MPI compiler wrappers properly to build packages and ... and ...) That's up to you. Regardless of which package manager is used, the discussion of how PETSc can work better in such a system still applies.

1 0