July 2013 - petsc-dev - mailman3.cels.anl.gov

OpenCL 2.0 provisional
by Karl Rupp 25 Jul '13

25 Jul '13

Hi guys, A couple of days back the Khronos group released their OpenCL 2.0 preview: http://www.khronos.org/registry/cl/ They pretty much extend their memory model, including shared virtual memory for APU-like processors sharing CPU and accelerator hardware on the same chip and also (try to) standardize an LLVM-like IR. As I was discussing with Matt that the OpenCL compilation on the CPU is completely messed up as-is, and considering that Jed played a bit with LLVM recently, it may be worth targeting such an IR for our basic kernels. Of course this does not solve the memory problems of NUMA and dealing properly with the increasing number of memory channels... Best regards, Karli

1 0

Re: [petsc-dev] SNESConvergedDefault() side effect?
by Peter Brune 24 Jul '13

24 Jul '13

On Wed, Jul 24, 2013 at 5:26 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Dmitry Karpeyev <karpeev(a)mcs.anl.gov> writes: > > > Yes. How should EW version 3 be implemented then? Currently EW3 > > piggybacks on snes->ttol to get a measure of the initial norm of the > > residual, and relies on it having been set elsewhere. If no > > references to snes->ttol in SNESKSPEW_PreSolve() are allowed, what > > should take its place? > > I would say that EW3 should call VecNorm() on the first iteration. Vecs > cache their norms internally so this will only do one reduction, even if > the SNES convergence test calls VecNorm() itself. > > > I was just thinking of a single cache for the initial residual norm. > > Currently there are at least two, it seems to me. > > Well, snes->norm_init is currently serving a different purpose, Yep, this was a cached value used to initialize the value in order to avoid extra global reductions for FAS and nonlinear preconditioning, not something to be used in tolerance tests. This was pretty obvious from the docs. The SNESSetInitialFunction(Norm)() stuff has some logic in it based upon the function being the same or different from the user set one, so it's really not the place to set things for tolerance tests either. > but it's > not entirely clear that it needs to be set. In particular, do we ever > compute a norm using a side channel rather than by calling VecNorm, > possibly followed by VecCopy? If not, isn't calling > SNESSetInitialFunction enough? I'm convinced. Created a branch getting rid of it and merged it into next. https://bitbucket.org/petsc/petsc/commits/082749e3fd29b5b736d87e346c0f1a523… > SNES can just call VecNorm on F and > it'll use the cached value if it was set. >

1 0

Re: [petsc-dev] SNESConvergedDefault() side effect?
by Jed Brown 24 Jul '13

24 Jul '13

Dmitry Karpeyev <karpeev(a)mcs.anl.gov> writes: > Also, this is a minor point, but isn't SNESKSPEW_PreSolve() misnamed? > It operates on a KSP and takes a SNES context, so shouldn't it be > KSPPreSolve_SNESEW() or some such? Yes, it is passed to KSPSetPreSolve, so it should be named something like KSPPreSolve_SNESEW. > Should we also fix SNESSkipConverged() --> SNESConvergedSkip() at the same > time? Yes, that was the intent of my last comment. It was missed when SNESDefaultConverged was renamed to SNESConvergedDefault.

1 0

Re: [petsc-dev] SNESConvergedDefault() side effect?
by Dmitry Karpeyev 24 Jul '13

24 Jul '13

On Wed, Jul 24, 2013 at 6:26 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Dmitry Karpeyev <karpeev(a)mcs.anl.gov> writes: > > > Yes. How should EW version 3 be implemented then? Currently EW3 > > piggybacks on snes->ttol to get a measure of the initial norm of the > > residual, and relies on it having been set elsewhere. If no > > references to snes->ttol in SNESKSPEW_PreSolve() are allowed, what > > should take its place? > > I would say that EW3 should call VecNorm() on the first iteration. Vecs > cache their norms internally so this will only do one reduction, even if > the SNES convergence test calls VecNorm() itself. > Okay. Also, this is a minor point, but isn't SNESKSPEW_PreSolve() misnamed? It operates on a KSP and takes a SNES context, so shouldn't it be KSPPreSolve_SNESEW() or some such? Should we also fix SNESSkipConverged() --> SNESConvergedSkip() at the same time? Dmitry. > > > I was just thinking of a single cache for the initial residual norm. > > Currently there are at least two, it seems to me. > > Well, snes->norm_init is currently serving a different purpose, but it's > not entirely clear that it needs to be set. In particular, do we ever > compute a norm using a side channel rather than by calling VecNorm, > possibly followed by VecCopy? If not, isn't calling > SNESSetInitialFunction enough? SNES can just call VecNorm on F and > it'll use the cached value if it was set. >

1 0

Re: [petsc-dev] SNESConvergedDefault() side effect?
by Jed Brown 24 Jul '13

24 Jul '13

Dmitry Karpeyev <karpeev(a)mcs.anl.gov> writes: > Yes. How should EW version 3 be implemented then? Currently EW3 > piggybacks on snes->ttol to get a measure of the initial norm of the > residual, and relies on it having been set elsewhere. If no > references to snes->ttol in SNESKSPEW_PreSolve() are allowed, what > should take its place? I would say that EW3 should call VecNorm() on the first iteration. Vecs cache their norms internally so this will only do one reduction, even if the SNES convergence test calls VecNorm() itself. > I was just thinking of a single cache for the initial residual norm. > Currently there are at least two, it seems to me. Well, snes->norm_init is currently serving a different purpose, but it's not entirely clear that it needs to be set. In particular, do we ever compute a norm using a side channel rather than by calling VecNorm, possibly followed by VecCopy? If not, isn't calling SNESSetInitialFunction enough? SNES can just call VecNorm on F and it'll use the cached value if it was set.

1 0

Re: [petsc-dev] SNESConvergedDefault() side effect?
by Dmitry Karpeyev 24 Jul '13

24 Jul '13

On Wed, Jul 24, 2013 at 3:24 PM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Dmitry Karpeyev <karpeev(a)mcs.anl.gov> writes: > > > It appears that SNESConvergedDefault() sets snes->ttol here > > > https://bitbucket.org/petsc/petsc/src/40bc6eb114e6a8e7f27fd216b5c1226efddf3… > > > > That should be harmless, albeit unnecessary, since snes->ttol is also set > > at the beginning of the solve in SNESSolve_NEWTONLS() (and other > > SNESSolve_XXX) right before calling the convergence test: > > > https://bitbucket.org/petsc/petsc/src/40bc6eb114e6a8e7f27fd216b5c1226efddf3… > > > > > > However, this might make for some fragile code and may confuse the user > > modeling a custom convergence test on SNESConvergedDefault(). In > > particular, a local reading of SNESConvergedDefault() creates the > > impression that it is the convergence monitor that is responsible for > > setting snes->ttol on the zeroth iteration -- snes->ttol is used outside > of > > SNES, for example, in one of the variants of the EW code (maybe only > > there). That would require access to SNES internals by user code. > > Well, when using SNESSkipConverged (should have been named > SNESConvergedSkip when changing to SNESConvergedDefault!), we don't need > an initial function norm, so setting ttol outside the convergence test > is wasteful. (I'm thinking in terms of reductions when using an > inexpensive SNES iteration like a point smoother.) If the convergence > test implementation wants to use a relative tolerance, then it should > store the target tolerance (or equivalently, the initial residual) in > its own context. I think all reference to ttol should be removed from > outside the convergence tests. > Yes. How should EW version 3 be implemented then? Currently EW3 piggybacks on snes->ttol to get a measure of the initial norm of the residual, and relies on it having been set elsewhere. If no references to snes->ttol in SNESKSPEW_PreSolve() are allowed, what should take its place? > > > More generally, should this sort of thing be handled everywhere using > > snes->norm_init for consistency and to prevent potential breakage? > > snes->norm_init goes with SNESSetInitialFunctionNorm and I don't think > it's used otherwise. > I was just thinking of a single cache for the initial residual norm. Currently there are at least two, it seems to me. Dmitry. > > > At the very least, should snes->ttol = ... be removed from > > SNESConvergedDefault()? > > > > > Thanks. > > Dmitry. >

1 0

Re: [petsc-dev] SNESConvergedDefault() side effect?
by Jed Brown 24 Jul '13

24 Jul '13

Dmitry Karpeyev <karpeev(a)mcs.anl.gov> writes: > It appears that SNESConvergedDefault() sets snes->ttol here > https://bitbucket.org/petsc/petsc/src/40bc6eb114e6a8e7f27fd216b5c1226efddf3… > > That should be harmless, albeit unnecessary, since snes->ttol is also set > at the beginning of the solve in SNESSolve_NEWTONLS() (and other > SNESSolve_XXX) right before calling the convergence test: > https://bitbucket.org/petsc/petsc/src/40bc6eb114e6a8e7f27fd216b5c1226efddf3… > > > However, this might make for some fragile code and may confuse the user > modeling a custom convergence test on SNESConvergedDefault(). In > particular, a local reading of SNESConvergedDefault() creates the > impression that it is the convergence monitor that is responsible for > setting snes->ttol on the zeroth iteration -- snes->ttol is used outside of > SNES, for example, in one of the variants of the EW code (maybe only > there). That would require access to SNES internals by user code. Well, when using SNESSkipConverged (should have been named SNESConvergedSkip when changing to SNESConvergedDefault!), we don't need an initial function norm, so setting ttol outside the convergence test is wasteful. (I'm thinking in terms of reductions when using an inexpensive SNES iteration like a point smoother.) If the convergence test implementation wants to use a relative tolerance, then it should store the target tolerance (or equivalently, the initial residual) in its own context. I think all reference to ttol should be removed from outside the convergence tests. > More generally, should this sort of thing be handled everywhere using > snes->norm_init for consistency and to prevent potential breakage? snes->norm_init goes with SNESSetInitialFunctionNorm and I don't think it's used otherwise. > At the very least, should snes->ttol = ... be removed from > SNESConvergedDefault()? > Thanks. > Dmitry.

1 0

SNESConvergedDefault() side effect?
by Dmitry Karpeyev 24 Jul '13

24 Jul '13

It appears that SNESConvergedDefault() sets snes->ttol here https://bitbucket.org/petsc/petsc/src/40bc6eb114e6a8e7f27fd216b5c1226efddf3… That should be harmless, albeit unnecessary, since snes->ttol is also set at the beginning of the solve in SNESSolve_NEWTONLS() (and other SNESSolve_XXX) right before calling the convergence test: https://bitbucket.org/petsc/petsc/src/40bc6eb114e6a8e7f27fd216b5c1226efddf3… However, this might make for some fragile code and may confuse the user modeling a custom convergence test on SNESConvergedDefault(). In particular, a local reading of SNESConvergedDefault() creates the impression that it is the convergence monitor that is responsible for setting snes->ttol on the zeroth iteration -- snes->ttol is used outside of SNES, for example, in one of the variants of the EW code (maybe only there). That would require access to SNES internals by user code. More generally, should this sort of thing be handled everywhere using snes->norm_init for consistency and to prevent potential breakage? At the very least, should snes->ttol = ... be removed from SNESConvergedDefault()? Thanks. Dmitry.

1 0

Re: [petsc-dev] petsc-hg
by Jed Brown 24 Jul '13

24 Jul '13

"Hammond, Glenn" <glenn.hammond(a)pnnl.gov> writes: > PETSc, > > Is the petsc-hg mercurial mirror site on bitbucket no longer in use? > The last update was June 28th? Oops, it got stuck. Updated now.

1 0

petsc-hg
by Hammond, Glenn 24 Jul '13

24 Jul '13

PETSc, Is the petsc-hg mercurial mirror site on bitbucket no longer in use? The last update was June 28th? Thanks, Glenn

1 0