petsc-dev

Download

[email protected]

January 2015

30 participants
191 discussions

Re: [petsc-dev] Sean is going to love this
by Geoff Oxberry 31 Dec '14

31 Dec '14

On Wed, Dec 31, 2014 at 11:40 AM, Geoff Oxberry <goxberry(a)gmail.com> wrote: > > > On Tue, Dec 30, 2014 at 10:40 AM, Sean Farley < > sean.michael.farley(a)gmail.com> wrote: > >> >> Barry Smith writes: >> >> >> On Dec 29, 2014, at 9:56 PM, Geoff Oxberry <goxberry(a)gmail.com> wrote: >> >> >> >> > Date: Sun, 28 Dec 2014 21:27:21 -0600 >> >> > From: Barry Smith <bsmith(a)mcs.anl.gov> >> >> > To: Sean Farley <sean.michael.farley(a)gmail.com> >> >> > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov> >> >> > Subject: Re: [petsc-dev] Sean is going to love this >> >> > Message-ID: <BA03ABA6-C693-4404-9FA5-2211BC371836(a)mcs.anl.gov> >> >> > Content-Type: text/plain; charset="utf-8" >> >> > >> >> > >> >> > Sean, >> >> > >> >> > brew install /homebrew/science/petsc >> >> > brew install /homebrew/science/petsc --HEAD --with-x11 >> >> > >> >> > Is there any reason not to use home-brew for everything now? >> Should we be working with the homebrew/science guys to work out the rough >> edges? (Like their hypre build is sequential and they don't pass the MPI >> compiler wrappers properly to build packages and ... and ...) >> >> >> >> I contributed most of the hypre recipe; brew install hypre --with-mpi >> should get you a parallel build, and it should also run some parallel >> exampled as smoke tests. If there's anything missing that should be added, >> let me know and I can get to it when I return from vacation. >> > >> > Geoff, >> > >> > Thanks, I noticed your name on a bunch of things there. I may >> bother you later with some issues when I understand brew better but, what >> the heck, here are a few things I noticed. (By the way, overall I was >> impressed with brew working with all these "nasty" scientific packages) >> >> No love for the work I did with all the nasty scientific packages? Oh, >> ok :-( >> >> > 1) I ran brew install /homebrew/science/petsc --HEAD --with-hypre and >> it failed because it just used the bottled hypre (which is sequential) as a >> dependent package and then tried to build the parallel PETSc with the >> sequential hypre so failed in the middle of compiling PETSc because of >> inconsistent hypre include files (ones for sequential). >> >> > Brew bottles were originally only used in situations where building from > source would take a long time, and under the assumption that most users are > interested in a "standard" build without any command-line options. > Recently, this idea seems to have been extended to most brew formula. I > don't think brew bottles make sense for something like hypre, which is > relatively quick to build, because the convenience savings are negligible. > > >> Yep, this is a problem that doesn't exist for PETSc in MacPorts. >> >> > Presumably had I done a brew install hypre --with-mpi first and then >> the brew install /homebrew/science/petsc --HEAD it may have worked >> properly? (but how is a naive new user to know?) >> >> Indeed. >> > > You're right. The PETSc formula should require that if hypre is used, it > must have been built with the "--with-mpi" flag. This issue would have > tripped me up also. I will add a GitHub issue for this point: > https://github.com/Homebrew/homebrew-science/issues/1602 > Dominique Orban (@dpo) has submitted a pull request that should remedy this issue: https://github.com/Homebrew/homebrew-science/pull/1604 He's also added some commentary responding to Barry's e-mail re: brew; some of the material is covered in my earlier response, and he raises some additional interesting points: https://github.com/Homebrew/homebrew-science/issues/1602#issuecomment-68472… > > > This seems to be related to my first meta issue which is how is brew >> really suppose to work with versions of packages (and build option versions >> for dependent packages)? For example petsc 3.5.2 uses certain versions of >> other packages such as mumps, superlu_dist etc but petsc --HEAD (which will >> become PETSc 3.6) may work with later versions of some of the packages. How >> is this suppose to be indicated, handled, etc? I couldn't see if brew had >> the concept of handling multiple versions of packages and different version >> dependencies for each version? > > Brew formulas can express package dependencies that include install flags; > https://github.com/Homebrew/homebrew/blob/master/Library/Contributions/exam… > <https://github.com/Homebrew/homebrew/blob/master/Library/Contributions/exam…> is > a comprehensive example of what can and can't be done. > > Indicating version dependencies is problematic. The brew package model > assumes users want the latest versions of packages, which has led to sets > of formulas like {python, python3} in the main homebrew repo, and {metis, > metis4} in cases where packages depend on specific versions. So brew does > not have a sophisticated way of handling packages with multiple version > numbers. > > If one has used brew to install different versions of a package over time, > such as installing `open-mpi` version 1.7.4, then later upgrading to > version 1.8.1, the `brew-switch` command can be used to switch among > versions (e.g., `brew switch open-mpi 1.7.4` or `brew switch open-mpi > 1.8.1` in this example, depending on which version is currently active). > `brew info` can be used to query the currently active version (e.g., `brew > info open-mpi`). > > If PETSc only uses the latest versions of packages, the best way forward > is probably to update the formulas for PETSc dependencies. If PETSc is tied > to specific package versions, then it might be better to have a separate > PETSc tap, with namespaced dependencies that don't conflict with > homebrew-science. Dominique Orban was working on a FEniCS tap for a while ( > https://github.com/optimizers/homebrew-fenics) I had mixed success when > using it. > > Since Sean is going to say it if I don't, yes, I think MacPorts does a > better job indicating version dependencies, and Homebrew has room for > improvement here. > >> >> > Second meta issue, is there a way to force a compiled of the package >> instead of it using the prepared bottle? > > Or even better to just tell brew I never want it to use bottles but always >> want it to compile for all packages? One reason I ask this is because PETSc >> has so many optional dependencies it seems maybe? better to just have it >> build based on the choices the user made rather than using some pre-built >> bottle that only has certain things turned on? But maybe I don't understand >> bottles and when they are used instead of building. >> > > > https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/FAQ.md#… > > To summarize, a formula will build from source if: > > - any installation option is added (e.g., `brew install petsc --HEAD`) > - the environment variable HOMEBREW_BUILD_FROM_SOURCE is set > - the option `--build-from-source` is invoked (e.g., `brew install > --build-from-source hypre`) > > >> This is done in MacPorts. >> >> > 2) I think PETSc is just "lucky" that it builds at all. brew seems to >> set the environmental variables CC, CXX, and FC to clang, clang++ and >> gfortran but then PETSc actually ignores these variables (printing a >> warning if you run with -vd) and hunts for mpicc, mpicxx, mpifc which it >> happens to find in /usr/local/bin since open-mpi is a dependency of PETSc >> and thus must have been installed first. >> >> This is why sandboxing is important for package managers. >> > > I agree with Sean here. > > Brew sets a bunch of environmental variables in something it calls > superenv; the rationale is given in > https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/Homebre…. > Much of the code for the sandboxing is in > https://github.com/Homebrew/homebrew/tree/master/Library/Homebrew/extend, > in `ENV.rb` or the `ENV` directory. > > >> > So this leads the third meta issue, it seems brew doesn't seem to >> have any concept of some packages requiring mpi wrapper compilers to be >> used to compile the package and so it is kind of catch as catch can if MPI >> based packages will build properly instead of having a systematic way of >> handling it. For example if PETSc did not ignore CC, CXX, and FC it would >> try to use clang, clang++ and gfortran and then crap out that it didn't >> have MPI compilers. Any chance that brew would ever be smart enough to >> "reset" the compilers to the MPI ones before building packages that depend >> on MPI? >> > > I think the next step is to improve support for MPI wrapper compilers > within formulas of packages that depend on MPI or have MPI options. I can > look at MacPorts and BuildSystem for inspiration here to see what could be > done. > > >> >> A hard graph problem that I hacked in MacPorts. I have no idea about how >> to do it in brew (hence why I haven't used it). >> >> > 3) PETSc has a bunch of other optional dependencies such as hdf5, >> netcdf ... that it would be nice to support. Presumably I could try to make >> a pull request with support for them but I'm concerned that my lack of >> understanding of meta issue two might mess me up. >> >> All of those are supported in MacPorts. >> > > As long as formulas exist (or are created) for these optional > dependencies, I think adding support for them is doable, and probably > straightforward. > > >> >> > 4) The issue of MPICH2 and open-mpi conflicting is a pain. Some package >> managers handle this by having a concept of a meta-package or >> abstract-package (such as MPI) that can have multiple implementation. Then, >> for example, one could say PETSc is dependent on MPI (and not open-mpi or >> MPICH) and the package system would work properly if either open-mpi or >> MPICH had already been installed (instead of crapping out like it does if I >> first install MPICH and then install PETSc; since it tries to install >> open-mpi for PETSc which conflicts with MPICH and thus stops). >> >> Also, this is done in MacPorts. >> > > Brew does not handle open-mpi vs MPICH well. An inelegant solution that > would probably work would be to hardcode paths for each MPI implementation. > If the only implementations used are open-mpi and MPICH, this solution is > expedient and probably fine. I started along this path for MUMPS in a pull > request (https://github.com/Homebrew/homebrew-science/pull/771) that > brought up relevant discussion with Jed, Aron, and Dominique Orban. > > One issue arising with brew is that if a package like `mumps` is build > with `mpich2`, and then `mpich2` is uninstalled and replaced with > `open-mpi`, brew should rebuild `mumps`. Right now, it won't force a > rebuild like that. Also, the `:mpi` dependency only resolves to `open-mpi`. > I think it would be worth revisiting this issue to point out the demand for > better MPI dependency tracking and resolution. > > >> >> With that being said, I must also declare that I have no love for >> MacPorts. It gets the job done for these specific cases and works just >> well enough but still has other problems. MacPorts will never be able to >> solve harder DAG problems for dependencies (just hacks). I wish I had >> the time to build a new package manager, but alas. The quote from ESR >> seems to ring true: >> > > MacPorts is okay. Sean's put a lot of work into it, which I appreciate, > and I've used MacPorts in the past. I eventually found it hard to maintain > because duplicating system programs into their own sandbox required > environment hacking to make sure I got the right versions on my PATH. I > found it more difficult to maintain. Since Homebrew avoids duplicating > system utilities (e.g., compilers), I find it easier to maintain a cleaner > system setup. > > I also found it difficult to contribute to MacPorts. I'd rather use Git > and Ruby over SVN and TCL; in both cases, the former technology is more > widely used than the latter. Getting a pull request accepted as a new > contributed to Homebrew was easy; the same did not appear to be true for > MacPorts. I think these reasons might contribute to why MacPorts has 194 > contributors (https://www.openhub.net/p/MacPorts) and Homebrew has 5,034 > contributors (https://www.openhub.net/p/homebrew) Last I checked, > Homebrew was the 12th-most starred and 9th-most forked repo on GitHub. > > From a feature standpoint, I think MacPorts does a lot of things > feature-wise that Homebrew should be doing, so I take it as a guide for how > to improve Homebrew. > > >> >> The most dangerous enemy of a better solution is an existing codebase >> that is just good enough. >> > > > -- > Geoffrey Oxberry, Ph.D., E.I.T. > goxberry(a)gmail.com >

1 0

Jump to page: