April 2013 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
by Matthew Knepley 11 Apr '13

11 Apr '13

On Thu, Apr 11, 2013 at 10:39 AM, Satish Balay <balay(a)mcs.anl.gov> wrote: > On Thu, 11 Apr 2013, Satish Balay wrote: > > > On Thu, 11 Apr 2013, Matthew Knepley wrote: > > > > > On Thu, Apr 11, 2013 at 10:12 AM, Satish Balay <balay(a)mcs.anl.gov> > wrote: > > > > > > > On Thu, 11 Apr 2013, Satish Balay wrote: > > > > > > > > > > > > > > We have had this same discussion a million times, and we > always > > > > > > > > > eventually > > > > > > > > > > get to the same answer and forget. Do we need our own > error > > > > message > > > > > > > for > > > > > > > > > > petsc-dev? > > > > > > > > > > > > > > > > > > > > We cannot possibly guess everything that can go wrong. > What we > > > > need > > > > > > > is a > > > > > > > > > > system > > > > > > > > > > where the user can specify what he actually wants and we > check > > > > it. > > > > > > > So, > > > > > > > > > zlib > > > > > > > > > > and > > > > > > > > > > everything else are optional dependencies for hdf5. If > > > > something is > > > > > > > not > > > > > > > > > > specified, it > > > > > > > > > > fails, and the user goes back and specifies the right > thing. > > > > > > > > > > > > > > > > > > Sure - but specify how? specify everything with > > > > > > > > > --with-hdf5-lib,--with-hdf5-include options? > > > > > > > > > > > > > > > > > > Or somehow he knows that he has to specify > > > > > > > > > --with-zlib-include/-with-zlib-lib options? > > > > > > > > > > > > > > > > > > > > > > > > > This. It would be nice if HDF5 would tell us about its > > > > dependencies, but > > > > > > > if > > > > > > > > it does not, then > > > > > > > > the user has to, and this is our system (packages). > > > > > > > > > > > > > > So we would start adding zlib.py, szip.py and manymore [and > support > > > > > > > --download-zlib --download-szip]. And if user gets the error: > > > > > > > > > > > > > > --with-hdf5-dir=/foo/bar does not work. > > > > > > > > > > > > > > So he/she send us configure.log > > > > > > > > > > > > > > Then we look at it and respond saying: you need to specify > > > > --with-zlib-dir > > > > > > > --with-szip-dir options aswell. > > > > > > > > > > > > > > Does this look right? > > > > > > > > > > > > > > > > > > Yes, that is what I think makes sense. > > > > > > > > > > > > We can always try to add another layer that 'greps' for libz in > the > > > > hdf5 > > > > > > error message and guesses that > > > > > > the user forgot it, but hopefully before we get to it, a sane > storage > > > > > > library appears, or HDF5 gets its act > > > > > > together. > > > > > > > > > > In this case - why bother with libz.py and szip.py? Why not tell > the > > > > > user to specify hdf5 completely with --with-hdf5-include and > > > > > --with-hdf5-lib options? > > > > > > > > > > Just printing the link errors on the screen might be sufficient to > > > > > clue in the user. Currently its hidden inside configure. [And its > not > > > > > clear which of the 10 tests configure did for this case should be > > > > > printed on the screen as the appropriate message] > > > > > > > > I might have been the one that advocated self.needsCompression > approach > > > > thats > > > > currently in configure. > > > > > > > > This is currently used only by hdf5.py. > > > > > > > > My current thought is to strip it out - and just add it to hdf5.py > as: > > > > > > > > self.liblist = > > > > > [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] > > > > > > > > This would be equivalent to the current functionality - and perhaps > > > > what was there previously - before we wanted better error messages > and > > > > detection of zlib. Since this is not possible anyway why bother > > > > with the additional complexity? > > > > > > > > > What is not possible? Detecting zlib? Of course this is possible, and > we do > > > it. > > > > The primary thing we need is detecting hdf5 provided by user requires > > zlib [and or szip]. This is not poissible. > > > > And current code can't detect zlib anyway [outside system locations] > > so we need libz.py to detect it better. But whats the point of it? > > [detecting zlib or szip via --with-zlib-dir and --with-szip-dir options?] > > > > When use has a mesages: > > > > --with-hdf5=foo/bar provides does not work > > > > I'd rather respond with: > > > > Specify hdf5 and all its dependencies correctly with > > --with-hdf5-lib,--with-hdf5-include options. > > > > > > > It is not possible to know without testing what crazy link line is > > > required by HDF5, at least I don't know how to do that. > > > > yes > > > > > This is no way affects the utility of testing for zlib. > > > > But whats the point of testing for zlib - if not for hdf5? And if we > > can't relate them automatically. > > I'll concede that having zlib.py szip.py will be useful if you need > them for --download-hdf5 [either as mandatory dependencies or optonal > dependencies]. > > Currently --download-hdf5 isn't built with either. > > My contention here is - depencency management stuff in configure > doesn't help for the optional dependencies of user installed packages > that are specified to petsc configure with the option > --with-package-dir Okay, I would like to distinguish between good behavior and HDF5. If we can ask whether the package enables it, we are fine. If there is a simple test we can do, we are fine. What can we do for HDF5? Matt > > Satish > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] examples/benchmarks for weak and strong scaling exercise
by Jed Brown 11 Apr '13

11 Apr '13

Chris Kees <cekees(a)gmail.com> writes: > Thanks a lot. I did a little example with the Bratu problem and posted it here: > > https://proteus.usace.army.mil/home/pub/17/ > > I used boomeramg instead of geometric multigrid because I was getting > an error with the options above: > > %mpiexec -np 4 ./ex5 -mx 129 -my 129 -Nx 2 -Ny 2 -pc_type mg -pc_mg_levels 2 > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Argument out of range! > [0]PETSC ERROR: New nonzero at (66,1) caused a malloc! > [0]PETSC ERROR: > ------------------------------------------------------------------------ That test hard-codes evil things (presumably for testing purposes, though maybe the functionality has been subsumed). Please use src/snes/examples/tutorials/ex5.c instead. mpiexec -n 4 ./ex5 -da_grid_x 65 -da_grid_y 65 -pc_type mg -log_summary -da_refine 1 Increase '-da_refine 1' to get higher resolution. (This will increase the number of MG levels used by PCMG.) Switch '-da_refine 1' to '-snes_grid_sequence 1' if you want FMG, but note that it's trickier to profile because proportionately more time is spent in coarse levels (although the total solve time is lower). > > I like the ice paper and will try to get the contractor started on > reproducing those results. > > -Chris > > On Wed, Apr 10, 2013 at 1:13 PM, Nystrom, William D <wdn(a)lanl.gov> wrote: >> Sorry. I overlooked that the URL was using git protocol. My bad. >> >> Dave >> >> ________________________________________ >> From: Jed Brown [five9a2(a)gmail.com] on behalf of Jed Brown [jedbrown(a)mcs.anl.gov] >> Sent: Wednesday, April 10, 2013 12:10 PM >> To: Nystrom, William D; For users of the development version of PETSc; Chris Kees >> Subject: Re: [petsc-dev] examples/benchmarks for weak and strong scaling exercise >> >> "Nystrom, William D" <wdn(a)lanl.gov> writes: >> >>> Jed, >>> >>> I tried cloning your tme-ice git repo as follows and it failed: >>> >>> % git clone --recursive git://github.com/jedbrown/tme-ice.git tme_ice >>> Cloning into 'tme_ice'... >>> fatal: unable to connect to github.com: >>> github.com[0: 204.232.175.90]: errno=Connection timed out >>> >>> I'm doing this from an xterm that allows me to clone petsc just fine. >> >> You're using https or ssh to clone PETSc, but the git:// to clone >> tme-ice. The LANL network is blocking that port, so just use the https >> or ssh protocol.

1 0

Re: [petsc-dev] examples/benchmarks for weak and strong scaling exercise
by Aron Ahmadia 11 Apr '13

11 Apr '13

What is going on with those results? In both cases the first parallel code is seriously outperforming the single-core. I'd be interested in seeing the two log_summary outputs. I can only assume that the PETSc developers have an (if (serial); sleep(10);) buried somewhere in the src from their last Gordon Bell run :) A On Thu, Apr 11, 2013 at 6:33 PM, Chris Kees <cekees(a)gmail.com> wrote: > Thanks a lot. I did a little example with the Bratu problem and posted it > here: > > https://proteus.usace.army.mil/home/pub/17/ > > I used boomeramg instead of geometric multigrid because I was getting > an error with the options above: > > %mpiexec -np 4 ./ex5 -mx 129 -my 129 -Nx 2 -Ny 2 -pc_type mg -pc_mg_levels > 2 > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Argument out of range! > [0]PETSC ERROR: New nonzero at (66,1) caused a malloc! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > > I like the ice paper and will try to get the contractor started on > reproducing those results. > > -Chris > > On Wed, Apr 10, 2013 at 1:13 PM, Nystrom, William D <wdn(a)lanl.gov> wrote: > > Sorry. I overlooked that the URL was using git protocol. My bad. > > > > Dave > > > > ________________________________________ > > From: Jed Brown [five9a2(a)gmail.com] on behalf of Jed Brown [ > jedbrown(a)mcs.anl.gov] > > Sent: Wednesday, April 10, 2013 12:10 PM > > To: Nystrom, William D; For users of the development version of PETSc; > Chris Kees > > Subject: Re: [petsc-dev] examples/benchmarks for weak and strong scaling > exercise > > > > "Nystrom, William D" <wdn(a)lanl.gov> writes: > > > >> Jed, > >> > >> I tried cloning your tme-ice git repo as follows and it failed: > >> > >> % git clone --recursive git://github.com/jedbrown/tme-ice.git tme_ice > >> Cloning into 'tme_ice'... > >> fatal: unable to connect to github.com: > >> github.com[0: 204.232.175.90]: errno=Connection timed out > >> > >> I'm doing this from an xterm that allows me to clone petsc just fine. > > > > You're using https or ssh to clone PETSc, but the git:// to clone > > tme-ice. The LANL network is blocking that port, so just use the https > > or ssh protocol. >

1 0

Re: [petsc-dev] examples/benchmarks for weak and strong scaling exercise
by Chris Kees 11 Apr '13

11 Apr '13

Thanks a lot. I did a little example with the Bratu problem and posted it here: https://proteus.usace.army.mil/home/pub/17/ I used boomeramg instead of geometric multigrid because I was getting an error with the options above: %mpiexec -np 4 ./ex5 -mx 129 -my 129 -Nx 2 -Ny 2 -pc_type mg -pc_mg_levels 2 [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Argument out of range! [0]PETSC ERROR: New nonzero at (66,1) caused a malloc! [0]PETSC ERROR: ------------------------------------------------------------------------ I like the ice paper and will try to get the contractor started on reproducing those results. -Chris On Wed, Apr 10, 2013 at 1:13 PM, Nystrom, William D <wdn(a)lanl.gov> wrote: > Sorry. I overlooked that the URL was using git protocol. My bad. > > Dave > > ________________________________________ > From: Jed Brown [five9a2(a)gmail.com] on behalf of Jed Brown [jedbrown(a)mcs.anl.gov] > Sent: Wednesday, April 10, 2013 12:10 PM > To: Nystrom, William D; For users of the development version of PETSc; Chris Kees > Subject: Re: [petsc-dev] examples/benchmarks for weak and strong scaling exercise > > "Nystrom, William D" <wdn(a)lanl.gov> writes: > >> Jed, >> >> I tried cloning your tme-ice git repo as follows and it failed: >> >> % git clone --recursive git://github.com/jedbrown/tme-ice.git tme_ice >> Cloning into 'tme_ice'... >> fatal: unable to connect to github.com: >> github.com[0: 204.232.175.90]: errno=Connection timed out >> >> I'm doing this from an xterm that allows me to clone petsc just fine. > > You're using https or ssh to clone PETSc, but the git:// to clone > tme-ice. The LANL network is blocking that port, so just use the https > or ssh protocol.

1 0

Re: [petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
by Satish Balay 11 Apr '13

11 Apr '13

On Thu, 11 Apr 2013, Satish Balay wrote: > On Thu, 11 Apr 2013, Matthew Knepley wrote: > > > On Thu, Apr 11, 2013 at 10:12 AM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > > > > On Thu, 11 Apr 2013, Satish Balay wrote: > > > > > > > > > > > > We have had this same discussion a million times, and we always > > > > > > > > eventually > > > > > > > > > get to the same answer and forget. Do we need our own error > > > message > > > > > > for > > > > > > > > > petsc-dev? > > > > > > > > > > > > > > > > > > We cannot possibly guess everything that can go wrong. What we > > > need > > > > > > is a > > > > > > > > > system > > > > > > > > > where the user can specify what he actually wants and we check > > > it. > > > > > > So, > > > > > > > > zlib > > > > > > > > > and > > > > > > > > > everything else are optional dependencies for hdf5. If > > > something is > > > > > > not > > > > > > > > > specified, it > > > > > > > > > fails, and the user goes back and specifies the right thing. > > > > > > > > > > > > > > > > Sure - but specify how? specify everything with > > > > > > > > --with-hdf5-lib,--with-hdf5-include options? > > > > > > > > > > > > > > > > Or somehow he knows that he has to specify > > > > > > > > --with-zlib-include/-with-zlib-lib options? > > > > > > > > > > > > > > > > > > > > > > This. It would be nice if HDF5 would tell us about its > > > dependencies, but > > > > > > if > > > > > > > it does not, then > > > > > > > the user has to, and this is our system (packages). > > > > > > > > > > > > So we would start adding zlib.py, szip.py and manymore [and support > > > > > > --download-zlib --download-szip]. And if user gets the error: > > > > > > > > > > > > --with-hdf5-dir=/foo/bar does not work. > > > > > > > > > > > > So he/she send us configure.log > > > > > > > > > > > > Then we look at it and respond saying: you need to specify > > > --with-zlib-dir > > > > > > --with-szip-dir options aswell. > > > > > > > > > > > > Does this look right? > > > > > > > > > > > > > > > Yes, that is what I think makes sense. > > > > > > > > > > We can always try to add another layer that 'greps' for libz in the > > > hdf5 > > > > > error message and guesses that > > > > > the user forgot it, but hopefully before we get to it, a sane storage > > > > > library appears, or HDF5 gets its act > > > > > together. > > > > > > > > In this case - why bother with libz.py and szip.py? Why not tell the > > > > user to specify hdf5 completely with --with-hdf5-include and > > > > --with-hdf5-lib options? > > > > > > > > Just printing the link errors on the screen might be sufficient to > > > > clue in the user. Currently its hidden inside configure. [And its not > > > > clear which of the 10 tests configure did for this case should be > > > > printed on the screen as the appropriate message] > > > > > > I might have been the one that advocated self.needsCompression approach > > > thats > > > currently in configure. > > > > > > This is currently used only by hdf5.py. > > > > > > My current thought is to strip it out - and just add it to hdf5.py as: > > > > > > self.liblist = > > > [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] > > > > > > This would be equivalent to the current functionality - and perhaps > > > what was there previously - before we wanted better error messages and > > > detection of zlib. Since this is not possible anyway why bother > > > with the additional complexity? > > > > > > What is not possible? Detecting zlib? Of course this is possible, and we do > > it. > > The primary thing we need is detecting hdf5 provided by user requires > zlib [and or szip]. This is not poissible. > > And current code can't detect zlib anyway [outside system locations] > so we need libz.py to detect it better. But whats the point of it? > [detecting zlib or szip via --with-zlib-dir and --with-szip-dir options?] > > When use has a mesages: > > --with-hdf5=foo/bar provides does not work > > I'd rather respond with: > > Specify hdf5 and all its dependencies correctly with > --with-hdf5-lib,--with-hdf5-include options. > > > > It is not possible to know without testing what crazy link line is > > required by HDF5, at least I don't know how to do that. > > yes > > > This is no way affects the utility of testing for zlib. > > But whats the point of testing for zlib - if not for hdf5? And if we > can't relate them automatically. I'll concede that having zlib.py szip.py will be useful if you need them for --download-hdf5 [either as mandatory dependencies or optonal dependencies]. Currently --download-hdf5 isn't built with either. My contention here is - depencency management stuff in configure doesn't help for the optional dependencies of user installed packages that are specified to petsc configure with the option --with-package-dir Satish

1 0

Re: [petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
by Satish Balay 11 Apr '13

11 Apr '13

On Thu, 11 Apr 2013, Matthew Knepley wrote: > On Thu, Apr 11, 2013 at 10:12 AM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > > On Thu, 11 Apr 2013, Satish Balay wrote: > > > > > > > > > > We have had this same discussion a million times, and we always > > > > > > > eventually > > > > > > > > get to the same answer and forget. Do we need our own error > > message > > > > > for > > > > > > > > petsc-dev? > > > > > > > > > > > > > > > > We cannot possibly guess everything that can go wrong. What we > > need > > > > > is a > > > > > > > > system > > > > > > > > where the user can specify what he actually wants and we check > > it. > > > > > So, > > > > > > > zlib > > > > > > > > and > > > > > > > > everything else are optional dependencies for hdf5. If > > something is > > > > > not > > > > > > > > specified, it > > > > > > > > fails, and the user goes back and specifies the right thing. > > > > > > > > > > > > > > Sure - but specify how? specify everything with > > > > > > > --with-hdf5-lib,--with-hdf5-include options? > > > > > > > > > > > > > > Or somehow he knows that he has to specify > > > > > > > --with-zlib-include/-with-zlib-lib options? > > > > > > > > > > > > > > > > > > > This. It would be nice if HDF5 would tell us about its > > dependencies, but > > > > > if > > > > > > it does not, then > > > > > > the user has to, and this is our system (packages). > > > > > > > > > > So we would start adding zlib.py, szip.py and manymore [and support > > > > > --download-zlib --download-szip]. And if user gets the error: > > > > > > > > > > --with-hdf5-dir=/foo/bar does not work. > > > > > > > > > > So he/she send us configure.log > > > > > > > > > > Then we look at it and respond saying: you need to specify > > --with-zlib-dir > > > > > --with-szip-dir options aswell. > > > > > > > > > > Does this look right? > > > > > > > > > > > > Yes, that is what I think makes sense. > > > > > > > > We can always try to add another layer that 'greps' for libz in the > > hdf5 > > > > error message and guesses that > > > > the user forgot it, but hopefully before we get to it, a sane storage > > > > library appears, or HDF5 gets its act > > > > together. > > > > > > In this case - why bother with libz.py and szip.py? Why not tell the > > > user to specify hdf5 completely with --with-hdf5-include and > > > --with-hdf5-lib options? > > > > > > Just printing the link errors on the screen might be sufficient to > > > clue in the user. Currently its hidden inside configure. [And its not > > > clear which of the 10 tests configure did for this case should be > > > printed on the screen as the appropriate message] > > > > I might have been the one that advocated self.needsCompression approach > > thats > > currently in configure. > > > > This is currently used only by hdf5.py. > > > > My current thought is to strip it out - and just add it to hdf5.py as: > > > > self.liblist = > > [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] > > > > This would be equivalent to the current functionality - and perhaps > > what was there previously - before we wanted better error messages and > > detection of zlib. Since this is not possible anyway why bother > > with the additional complexity? > > > What is not possible? Detecting zlib? Of course this is possible, and we do > it. The primary thing we need is detecting hdf5 provided by user requires zlib [and or szip]. This is not poissible. And current code can't detect zlib anyway [outside system locations] so we need libz.py to detect it better. But whats the point of it? [detecting zlib or szip via --with-zlib-dir and --with-szip-dir options?] When use has a mesages: --with-hdf5=foo/bar provides does not work I'd rather respond with: Specify hdf5 and all its dependencies correctly with --with-hdf5-lib,--with-hdf5-include options. > It is not possible to know without testing what crazy link line is > required by HDF5, at least I don't know how to do that. yes > This is no way affects the utility of testing for zlib. But whats the point of testing for zlib - if not for hdf5? And if we can't relate them automatically. Satish

1 0

Re: [petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
by Matthew Knepley 11 Apr '13

11 Apr '13

On Thu, Apr 11, 2013 at 10:12 AM, Satish Balay <balay(a)mcs.anl.gov> wrote: > On Thu, 11 Apr 2013, Satish Balay wrote: > > > > > > > > We have had this same discussion a million times, and we always > > > > > > eventually > > > > > > > get to the same answer and forget. Do we need our own error > message > > > > for > > > > > > > petsc-dev? > > > > > > > > > > > > > > We cannot possibly guess everything that can go wrong. What we > need > > > > is a > > > > > > > system > > > > > > > where the user can specify what he actually wants and we check > it. > > > > So, > > > > > > zlib > > > > > > > and > > > > > > > everything else are optional dependencies for hdf5. If > something is > > > > not > > > > > > > specified, it > > > > > > > fails, and the user goes back and specifies the right thing. > > > > > > > > > > > > Sure - but specify how? specify everything with > > > > > > --with-hdf5-lib,--with-hdf5-include options? > > > > > > > > > > > > Or somehow he knows that he has to specify > > > > > > --with-zlib-include/-with-zlib-lib options? > > > > > > > > > > > > > > > > This. It would be nice if HDF5 would tell us about its > dependencies, but > > > > if > > > > > it does not, then > > > > > the user has to, and this is our system (packages). > > > > > > > > So we would start adding zlib.py, szip.py and manymore [and support > > > > --download-zlib --download-szip]. And if user gets the error: > > > > > > > > --with-hdf5-dir=/foo/bar does not work. > > > > > > > > So he/she send us configure.log > > > > > > > > Then we look at it and respond saying: you need to specify > --with-zlib-dir > > > > --with-szip-dir options aswell. > > > > > > > > Does this look right? > > > > > > > > > Yes, that is what I think makes sense. > > > > > > We can always try to add another layer that 'greps' for libz in the > hdf5 > > > error message and guesses that > > > the user forgot it, but hopefully before we get to it, a sane storage > > > library appears, or HDF5 gets its act > > > together. > > > > In this case - why bother with libz.py and szip.py? Why not tell the > > user to specify hdf5 completely with --with-hdf5-include and > > --with-hdf5-lib options? > > > > Just printing the link errors on the screen might be sufficient to > > clue in the user. Currently its hidden inside configure. [And its not > > clear which of the 10 tests configure did for this case should be > > printed on the screen as the appropriate message] > > I might have been the one that advocated self.needsCompression approach > thats > currently in configure. > > This is currently used only by hdf5.py. > > My current thought is to strip it out - and just add it to hdf5.py as: > > self.liblist = > [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] > > This would be equivalent to the current functionality - and perhaps > what was there previously - before we wanted better error messages and > detection of zlib. Since this is not possible anyway why bother > with the additional complexity? What is not possible? Detecting zlib? Of course this is possible, and we do it. It is not possible to know without testing what crazy link line is required by HDF5, at least I don't know how to do that. This is no way affects the utility of testing for zlib. Matt > > Satish > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
by Satish Balay 11 Apr '13

11 Apr '13

On Thu, 11 Apr 2013, Satish Balay wrote: > > > > > > We have had this same discussion a million times, and we always > > > > > eventually > > > > > > get to the same answer and forget. Do we need our own error message > > > for > > > > > > petsc-dev? > > > > > > > > > > > > We cannot possibly guess everything that can go wrong. What we need > > > is a > > > > > > system > > > > > > where the user can specify what he actually wants and we check it. > > > So, > > > > > zlib > > > > > > and > > > > > > everything else are optional dependencies for hdf5. If something is > > > not > > > > > > specified, it > > > > > > fails, and the user goes back and specifies the right thing. > > > > > > > > > > Sure - but specify how? specify everything with > > > > > --with-hdf5-lib,--with-hdf5-include options? > > > > > > > > > > Or somehow he knows that he has to specify > > > > > --with-zlib-include/-with-zlib-lib options? > > > > > > > > > > > > > This. It would be nice if HDF5 would tell us about its dependencies, but > > > if > > > > it does not, then > > > > the user has to, and this is our system (packages). > > > > > > So we would start adding zlib.py, szip.py and manymore [and support > > > --download-zlib --download-szip]. And if user gets the error: > > > > > > --with-hdf5-dir=/foo/bar does not work. > > > > > > So he/she send us configure.log > > > > > > Then we look at it and respond saying: you need to specify --with-zlib-dir > > > --with-szip-dir options aswell. > > > > > > Does this look right? > > > > > > Yes, that is what I think makes sense. > > > > We can always try to add another layer that 'greps' for libz in the hdf5 > > error message and guesses that > > the user forgot it, but hopefully before we get to it, a sane storage > > library appears, or HDF5 gets its act > > together. > > In this case - why bother with libz.py and szip.py? Why not tell the > user to specify hdf5 completely with --with-hdf5-include and > --with-hdf5-lib options? > > Just printing the link errors on the screen might be sufficient to > clue in the user. Currently its hidden inside configure. [And its not > clear which of the 10 tests configure did for this case should be > printed on the screen as the appropriate message] I might have been the one that advocated self.needsCompression approach thats currently in configure. This is currently used only by hdf5.py. My current thought is to strip it out - and just add it to hdf5.py as: self.liblist = [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] This would be equivalent to the current functionality - and perhaps what was there previously - before we wanted better error messages and detection of zlib. Since this is not possible anyway why bother with the additional complexity? Satish

1 0

Re: [petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
by Satish Balay 11 Apr '13

11 Apr '13

On Thu, 11 Apr 2013, Matthew Knepley wrote: > On Thu, Apr 11, 2013 at 9:48 AM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > > On Thu, 11 Apr 2013, Matthew Knepley wrote: > > > > > On Thu, Apr 11, 2013 at 9:38 AM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > > > > > > On Thu, 11 Apr 2013, Matthew Knepley wrote: > > > > > > > > > On Thu, Apr 11, 2013 at 9:17 AM, Satish Balay <balay(a)mcs.anl.gov> > > wrote: > > > > > > > > > > > On Thu, 11 Apr 2013, Matthew Knepley wrote: > > > > > > > > > > > > > On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay <balay(a)mcs.anl.gov > > > > > > > wrote: > > > > > > > > > > > > > > > How do we handle these packages with complicated dependencies > > [they > > > > > > > > are complicated as petsc dependencies?] > > > > > > > > > > > > > > > > In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So > > how > > > > > > > > does petsc configure automatically detect this? > > > > > > > > > > > > > > > > And I've recently used the following for hdf5/necddf5 on fusion > > > > [with > > > > > > > > the extra implicit depenceny of hdf5 on zlib, and netcdf5 on > > hdf5 > > > > from > > > > > > > > within configure] > > > > > > > > > > > > > > > > '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl > > > > -lhdf5 > > > > > > > > -lgpfs', > > > > > > > > > > > > > > > > '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib > > -lnetcdff > > > > > > > > -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 > > > > > > -lk5crypto > > > > > > > > -lcom_err -lidn -lssl -lcrypto -lz', > > > > > > > > > > > > > > > > Yeah - things work if the user knows the dependencies and the > > link > > > > > > > > command for those dependencies - and specify it to petsc > > configure > > > > as > > > > > > > > above. But I'm not sure how to autodetect this. > > > > > > > > > > > > > > > > Also currently -lz is handled in package.py with > > > > > > > > 'self.needsCompression' similar to 'self.needsMath' with the > > > > detection > > > > > > > > of -lz in libraires.py:checkCompression() - but one can't > > specify a > > > > > > > > --with-zlib-lib option this way. I guess this part can be > > fixed by > > > > > > > > migrating it a standalone package z.py. [And somehow handle the > > > > > > > > optional part of this dependency for hdf5] > > > > > > > > > > > > > > > > > > > > > We already have a mechanism for this. Lots of packages depend on > > > > > > > other packages. This is just screwed up in the case of libz > > because > > > > > > > someone (maybe me) did not want to write an entire package file > > for > > > > > > > it, and instead copped out with the needsCompression. > > > > > > > > > > > > Currently we handle 'mandatory' dependencies properly - and > > optional > > > > > > >depencencies for --download-pakcage somewhat - but not for > > > > > > --with-package-dir > > > > > > > > > > > > For hdf5 - handling just zlib.py is not sufficient. Do we add > > szip.py, > > > > > > zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use > > these as > > > > > > optionaly when testing for user provided --with-hdf5? Which package > > > > > > currently implements this usage [to check for this optionaly > > > > > > dependency - and give the correct error]? > > > > > > > > > > > > i.e In this case - when the user specified --with-hdf5-dir option. > > We > > > > > > had to print an error.: > > > > > > * please provide --with-zlib-dir and --with-szip-dir options. > > > > > > [For a different user - with the same usage - there would be no > > error] > > > > > > > > > > > > For netcdf above one would have kerbros.py, curl.py and perhaps > > more? > > > > > > > > > > > > > > > We have had this same discussion a million times, and we always > > > > eventually > > > > > get to the same answer and forget. Do we need our own error message > > for > > > > > petsc-dev? > > > > > > > > > > We cannot possibly guess everything that can go wrong. What we need > > is a > > > > > system > > > > > where the user can specify what he actually wants and we check it. > > So, > > > > zlib > > > > > and > > > > > everything else are optional dependencies for hdf5. If something is > > not > > > > > specified, it > > > > > fails, and the user goes back and specifies the right thing. > > > > > > > > Sure - but specify how? specify everything with > > > > --with-hdf5-lib,--with-hdf5-include options? > > > > > > > > Or somehow he knows that he has to specify > > > > --with-zlib-include/-with-zlib-lib options? > > > > > > > > > > This. It would be nice if HDF5 would tell us about its dependencies, but > > if > > > it does not, then > > > the user has to, and this is our system (packages). > > > > So we would start adding zlib.py, szip.py and manymore [and support > > --download-zlib --download-szip]. And if user gets the error: > > > > --with-hdf5-dir=/foo/bar does not work. > > > > So he/she send us configure.log > > > > Then we look at it and respond saying: you need to specify --with-zlib-dir > > --with-szip-dir options aswell. > > > > Does this look right? > > > Yes, that is what I think makes sense. > > We can always try to add another layer that 'greps' for libz in the hdf5 > error message and guesses that > the user forgot it, but hopefully before we get to it, a sane storage > library appears, or HDF5 gets its act > together. In this case - why bother with libz.py and szip.py? Why not tell the user to specify hdf5 completely with --with-hdf5-include and --with-hdf5-lib options? Just printing the link errors on the screen might be sufficient to clue in the user. Currently its hidden inside configure. [And its not clear which of the 10 tests configure did for this case should be printed on the screen as the appropriate message] Satish

1 0

Re: [petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
by Matthew Knepley 11 Apr '13

11 Apr '13

On Thu, Apr 11, 2013 at 9:48 AM, Satish Balay <balay(a)mcs.anl.gov> wrote: > On Thu, 11 Apr 2013, Matthew Knepley wrote: > > > On Thu, Apr 11, 2013 at 9:38 AM, Satish Balay <balay(a)mcs.anl.gov> wrote: > > > > > On Thu, 11 Apr 2013, Matthew Knepley wrote: > > > > > > > On Thu, Apr 11, 2013 at 9:17 AM, Satish Balay <balay(a)mcs.anl.gov> > wrote: > > > > > > > > > On Thu, 11 Apr 2013, Matthew Knepley wrote: > > > > > > > > > > > On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay <balay(a)mcs.anl.gov > > > > > wrote: > > > > > > > > > > > > > How do we handle these packages with complicated dependencies > [they > > > > > > > are complicated as petsc dependencies?] > > > > > > > > > > > > > > In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So > how > > > > > > > does petsc configure automatically detect this? > > > > > > > > > > > > > > And I've recently used the following for hdf5/necddf5 on fusion > > > [with > > > > > > > the extra implicit depenceny of hdf5 on zlib, and netcdf5 on > hdf5 > > > from > > > > > > > within configure] > > > > > > > > > > > > > > '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl > > > -lhdf5 > > > > > > > -lgpfs', > > > > > > > > > > > > > > '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib > -lnetcdff > > > > > > > -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 > > > > > -lk5crypto > > > > > > > -lcom_err -lidn -lssl -lcrypto -lz', > > > > > > > > > > > > > > Yeah - things work if the user knows the dependencies and the > link > > > > > > > command for those dependencies - and specify it to petsc > configure > > > as > > > > > > > above. But I'm not sure how to autodetect this. > > > > > > > > > > > > > > Also currently -lz is handled in package.py with > > > > > > > 'self.needsCompression' similar to 'self.needsMath' with the > > > detection > > > > > > > of -lz in libraires.py:checkCompression() - but one can't > specify a > > > > > > > --with-zlib-lib option this way. I guess this part can be > fixed by > > > > > > > migrating it a standalone package z.py. [And somehow handle the > > > > > > > optional part of this dependency for hdf5] > > > > > > > > > > > > > > > > > > We already have a mechanism for this. Lots of packages depend on > > > > > > other packages. This is just screwed up in the case of libz > because > > > > > > someone (maybe me) did not want to write an entire package file > for > > > > > > it, and instead copped out with the needsCompression. > > > > > > > > > > Currently we handle 'mandatory' dependencies properly - and > optional > > > > > >depencencies for --download-pakcage somewhat - but not for > > > > > --with-package-dir > > > > > > > > > > For hdf5 - handling just zlib.py is not sufficient. Do we add > szip.py, > > > > > zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use > these as > > > > > optionaly when testing for user provided --with-hdf5? Which package > > > > > currently implements this usage [to check for this optionaly > > > > > dependency - and give the correct error]? > > > > > > > > > > i.e In this case - when the user specified --with-hdf5-dir option. > We > > > > > had to print an error.: > > > > > * please provide --with-zlib-dir and --with-szip-dir options. > > > > > [For a different user - with the same usage - there would be no > error] > > > > > > > > > > For netcdf above one would have kerbros.py, curl.py and perhaps > more? > > > > > > > > > > > > We have had this same discussion a million times, and we always > > > eventually > > > > get to the same answer and forget. Do we need our own error message > for > > > > petsc-dev? > > > > > > > > We cannot possibly guess everything that can go wrong. What we need > is a > > > > system > > > > where the user can specify what he actually wants and we check it. > So, > > > zlib > > > > and > > > > everything else are optional dependencies for hdf5. If something is > not > > > > specified, it > > > > fails, and the user goes back and specifies the right thing. > > > > > > Sure - but specify how? specify everything with > > > --with-hdf5-lib,--with-hdf5-include options? > > > > > > Or somehow he knows that he has to specify > > > --with-zlib-include/-with-zlib-lib options? > > > > > > > This. It would be nice if HDF5 would tell us about its dependencies, but > if > > it does not, then > > the user has to, and this is our system (packages). > > So we would start adding zlib.py, szip.py and manymore [and support > --download-zlib --download-szip]. And if user gets the error: > > --with-hdf5-dir=/foo/bar does not work. > > So he/she send us configure.log > > Then we look at it and respond saying: you need to specify --with-zlib-dir > --with-szip-dir options aswell. > > Does this look right? Yes, that is what I think makes sense. We can always try to add another layer that 'greps' for libz in the hdf5 error message and guesses that the user forgot it, but hopefully before we get to it, a sane storage library appears, or HDF5 gets its act together. Matt > > Satish > > > > > Matt > > > > > > > This thread started becasue one expected --with-hdf5-dir to be > > > sufficient - and its not. current workarround [becuase of > > > self.needsCompression] is to specify zlib with LIBS otpions - and all > > > other dependencies with --with-hdf5-lib. It works - but if were were > > > satisfactory - this whole thread on petsc-dev wouldn't arise. > > > > > > Satish > > > > > > > > > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0