October 2014 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] [petsc-users] unreliable AMG in PETSc
by Jed Brown 27 Oct '14

27 Oct '14

Barry Smith <bsmith(a)mcs.anl.gov> writes: > Well, we did report the entire preconditioner was indefinite, but not what part of the preconditioner caused it to be indefinite. > > Could run the smoother with CG for many steps and see if that reports indefiniteness. Would it, I’m not even sure of that. I think it would report indefiniteness, but it depends on the RHS vector/initial guess. > We could easily add a test MatIsNonsingularM() i.e. a_ii > =0 a_ij <= 0, useful information but I don’t think it helps the user “fix” the problem. If they actually have an M-matrix, everything should just work. So how to help them when they have a problem/formulation that produces a matrix where methods aren't working?

1 0

Re: [petsc-dev] [petsc-users] unreliable AMG in PETSc
by Barry Smith 27 Oct '14

27 Oct '14

> On Oct 27, 2014, at 5:09 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > Matthew Knepley <knepley(a)gmail.com> writes: >> Definitely debuggability. > > Thanks. > >> It would be great if we automatically diagnose the failure of >> definiteness (I assume all these retain symmetry). > > So long as a symmetric operator is input. > >> Maybe we are already doing this. Then we should improve the error >> message to give the user easy options for fixing it. > > We only notice it at a higher level. The smoother could check whether > its action is SPD with respect to each vector it is applied to at the > cost of a dot product (check that x^T S x > 0). > > Actually, the problem is more subtle because if one wants an SPD > multigrid cycle, they only need that the post-smoother is the transpose > of the pre-smoother. Hmm. Is this true? This guy’s matrix causes grief with indefiniteness but since it uses SSOR smoother I think that post-smooth should have been the transpose of the pre smooth automatically? Hmm Barry > This could also be checked, but I don't know how > to do it without significant wasted work.

1 0

Re: [petsc-dev] [petsc-users] unreliable AMG in PETSc
by Barry Smith 27 Oct '14

27 Oct '14

> On Oct 27, 2014, at 4:56 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > > On Mon, Oct 27, 2014 at 4:53 PM, Jed Brown <jed(a)jedbrown.org> wrote: > Barry Smith <bsmith(a)mcs.anl.gov> writes: > > How can we insure that that the smoother is SPD? Which are known to be. In his case it was Richardson + SSOR on a bunch of processes. > > The Richardson scaling cannot be too ambitious and the SSOR needs to be > an SPD operator. I think this is true any time the forward and back > sweep is stable, which is guaranteed for an M-matrix, for example. Pretty sure that guy’s matrix is not an M-matrix. > > I'm not aware of an inexpensive way to guarantee stability/SPD smoother > for a general matrix, and even if you could choose parameters to make it > so, you might end up with such aggressive under-relaxation that the > solver configuration is not useful. > > Focus on debuggability or automation? > > Definitely debuggability. It would be great if we automatically diagnose the failure of definiteness (I assume > all these retain symmetry). Maybe we are already doing this. Then we should improve the error message > to give the user easy options for fixing it. Well, we did report the entire preconditioner was indefinite, but not what part of the preconditioner caused it to be indefinite. Could run the smoother with CG for many steps and see if that reports indefiniteness. Would it, I’m not even sure of that. We could easily add a test MatIsNonsingularM() i.e. a_ii > =0 a_ij <= 0, useful information but I don’t think it helps the user “fix” the problem. Barry > > Matt > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener

1 0

Re: [petsc-dev] [petsc-users] unreliable AMG in PETSc
by Jed Brown 27 Oct '14

27 Oct '14

Matthew Knepley <knepley(a)gmail.com> writes: > Definitely debuggability. Thanks. > It would be great if we automatically diagnose the failure of > definiteness (I assume all these retain symmetry). So long as a symmetric operator is input. > Maybe we are already doing this. Then we should improve the error > message to give the user easy options for fixing it. We only notice it at a higher level. The smoother could check whether its action is SPD with respect to each vector it is applied to at the cost of a dot product (check that x^T S x > 0). Actually, the problem is more subtle because if one wants an SPD multigrid cycle, they only need that the post-smoother is the transpose of the pre-smoother. This could also be checked, but I don't know how to do it without significant wasted work.

1 0

Re: [petsc-dev] [petsc-users] unreliable AMG in PETSc
by Matthew Knepley 27 Oct '14

27 Oct '14

On Mon, Oct 27, 2014 at 4:53 PM, Jed Brown <jed(a)jedbrown.org> wrote: > Barry Smith <bsmith(a)mcs.anl.gov> writes: > > How can we insure that that the smoother is SPD? Which are known to > be. In his case it was Richardson + SSOR on a bunch of processes. > > The Richardson scaling cannot be too ambitious and the SSOR needs to be > an SPD operator. I think this is true any time the forward and back > sweep is stable, which is guaranteed for an M-matrix, for example. > > I'm not aware of an inexpensive way to guarantee stability/SPD smoother > for a general matrix, and even if you could choose parameters to make it > so, you might end up with such aggressive under-relaxation that the > solver configuration is not useful. > > Focus on debuggability or automation? > Definitely debuggability. It would be great if we automatically diagnose the failure of definiteness (I assume all these retain symmetry). Maybe we are already doing this. Then we should improve the error message to give the user easy options for fixing it. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

GAMG documentation very thin?
by Barry Smith 27 Oct '14

27 Oct '14

The manual page for PCGAMG and the users manual seem very very thin are how to use PGGAMG, no mention of various useful options and no seeming mention of near null spaces etc that, as I understand it, care important for getting good convergence. Remember this is a library, not a framework? Barry Library, something you get and use. Framework, something you write a proposal with the authors of and then they may have some junior person help you use it until the grant runs low or they are working on another proposal.

1 0

Re: [petsc-dev] [petsc-users] unreliable AMG in PETSc
by Jed Brown 27 Oct '14

27 Oct '14

Barry Smith <bsmith(a)mcs.anl.gov> writes: > How can we insure that that the smoother is SPD? Which are known to be. In his case it was Richardson + SSOR on a bunch of processes. The Richardson scaling cannot be too ambitious and the SSOR needs to be an SPD operator. I think this is true any time the forward and back sweep is stable, which is guaranteed for an M-matrix, for example. I'm not aware of an inexpensive way to guarantee stability/SPD smoother for a general matrix, and even if you could choose parameters to make it so, you might end up with such aggressive under-relaxation that the solver configuration is not useful. Focus on debuggability or automation?

1 0

Re: [petsc-dev] [petsc-users] unreliable AMG in PETSc
by Barry Smith 27 Oct '14

27 Oct '14

> On Oct 27, 2014, at 1:09 PM, Mark Adams <mfadams(a)lbl.gov> wrote: > > Just to reiterate,add, emphasize what what was said. AMG solvers are very tricky to make robust to all -- even reasonable -- inputs. To say nothing of all inputs, but there are a finite number of ways the code can segv and we want fix them -- one by one. This code is in fact sable, for 2-3 year now but users find valid ways to create a segv or mysterious failure and we fix these. One at a time, but there is a finite number. > > Many PC can be unsymmetric (ILU, MG with F-cycles) and in the case of ILU indefinite to boot (thus the fix that Barry talked about). And as Jed says CG seems to be pretty robust to this. I've seen PETSc CG return negative eigenvalues for large deformation elasticity that do in fact go indefinite (buckling) but it worked fine and Newton eventually converged to an SPD system. > > AMG (MG in general) should not be indefinite with valid inputs, for scalar Laplacian at least. So a V-cycle preconditioner for an SPD matrix, with an SPD smoother should stay SPD. Mark, How can we insure that that the smoother is SPD? Which are known to be. In his case it was Richardson + SSOR on a bunch of processes. Barry > It might die but it should not be indefinite. > > Mark > > On Thu, Oct 23, 2014 at 2:12 PM, Jed Brown <jed(a)jedbrown.org> wrote: > Barry Smith <bsmith(a)mcs.anl.gov> writes: > > It is any interesting question if there is anything one can “do” to > > a preconditioner to insure that it is SPD or detect if it may not > > be BEFORE actually running the CG with that preconditioned. > > You can use an eigensolver for the extreme eigenvalues of the iteration > matrix for the smoother. And if it's unstable, report that. This is > one thing that the petsc-eig plugin (which I'll update) can do, but it's > too expensive to do by default. >

1 0

Re: [petsc-dev] Backend independent VecGetArray for GPUs
by Matthew Knepley 27 Oct '14

27 Oct '14

On Mon, Oct 27, 2014 at 7:48 AM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: > On 27 October 2014 15:06, Matthew Knepley <knepley(a)gmail.com> wrote: > > On Mon, Oct 27, 2014 at 6:30 AM, Lisandro Dalcin <dalcinl(a)gmail.com> > wrote: > >> > >> On 27 October 2014 13:49, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: > >> > 1) petsc4py will provide a method Vec.getCUDAAddress() that will > >> > return a plain Python integer (and a corresponding > >> > restoreCUDAAddress()). Any other name to suggest? Using "CUDAArray" > >> > would be a bit misleading in Python land. > >> > >> Perhaps "getCUDAHandle()" is better? Please help me to decide, I'm > >> really bad at naming things. > > > > > > I like handle. That is the traditional name. > > > > Thanks! Take a look at branch dalcinl/cuda-handle > > https://bitbucket.org/petsc/petsc4py/src/cec7da13bbb407a1b376f8e6b46b1ef31c… Great! Thanks, Matt > > PS: Ashwin, you could use contextmanager > https://docs.python.org/2/library/contextlib.html#contextlib.contextmanager > to handle the get/restore using Python's with statement, eg. > > with gpuarray(vec) as array: > launch_cuda_kernel(array) > > -- > Lisandro Dalcin > ============ > Research Scientist > Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) > Numerical Porous Media Center (NumPor) > King Abdullah University of Science and Technology (KAUST) > http://numpor.kaust.edu.sa/ > > 4700 King Abdullah University of Science and Technology > al-Khawarizmi Bldg (Bldg 1), Office # 4332 > Thuwal 23955-6900, Kingdom of Saudi Arabia > http://www.kaust.edu.sa > > Office Phone: +966 12 808-0459 > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Backend independent VecGetArray for GPUs
by Lisandro Dalcin 27 Oct '14

27 Oct '14

On 27 October 2014 15:06, Matthew Knepley <knepley(a)gmail.com> wrote: > On Mon, Oct 27, 2014 at 6:30 AM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >> >> On 27 October 2014 13:49, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >> > 1) petsc4py will provide a method Vec.getCUDAAddress() that will >> > return a plain Python integer (and a corresponding >> > restoreCUDAAddress()). Any other name to suggest? Using "CUDAArray" >> > would be a bit misleading in Python land. >> >> Perhaps "getCUDAHandle()" is better? Please help me to decide, I'm >> really bad at naming things. > > > I like handle. That is the traditional name. > Thanks! Take a look at branch dalcinl/cuda-handle https://bitbucket.org/petsc/petsc4py/src/cec7da13bbb407a1b376f8e6b46b1ef31c… PS: Ashwin, you could use contextmanager https://docs.python.org/2/library/contextlib.html#contextlib.contextmanager to handle the get/restore using Python's with statement, eg. with gpuarray(vec) as array: launch_cuda_kernel(array) -- Lisandro Dalcin ============ Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459

1 0