February 2012 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] SNES, Coloring, DM, and TS
by Matthew Knepley 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 11:31 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Mon, Feb 27, 2012 at 11:28, Matthew Knepley <knepley(a)gmail.com> wrote: > >> PetscErrorCode SNESDMComputeJacobian(SNES snes,Vec X,Mat *J,Mat >>> *B,MatStructure *flag,void *ptr) >>> { >>> PetscErrorCode ierr; >>> DM dm; >>> >>> PetscFunctionBegin; >>> ierr = SNESGetDM(snes,&dm);CHKERRQ(ierr); >>> ierr = DMComputeJacobian(dm,X,*J,*B,flag);CHKERRQ(ierr); >>> PetscFunctionReturn(0); >>> } >>> >> >> I thought the idea was for the solver to come in through the context >> argument of the user callback. Maybe there should be >> a structured way to for SNES to modify the DM context which it passes to >> the user callback. >> > > The way the code is written, there is no user context. The user can call > DMGetApplicationContext() if they want one, _but_ that should not contain a > SNES or TS. The application context should only contain > resolution-independent parameters. > ? My signature for FormJacobianLocal(DM dm, Vec X, Mat Jac, AppCtx *user) definitely has a context, which is required to have DM as the first member. We could require the solver as the second member. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] SNES, Coloring, DM, and TS
by Jed Brown 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 11:28, Matthew Knepley <knepley(a)gmail.com> wrote: > PetscErrorCode SNESDMComputeJacobian(SNES snes,Vec X,Mat *J,Mat >> *B,MatStructure *flag,void *ptr) >> { >> PetscErrorCode ierr; >> DM dm; >> >> PetscFunctionBegin; >> ierr = SNESGetDM(snes,&dm);CHKERRQ(ierr); >> ierr = DMComputeJacobian(dm,X,*J,*B,flag);CHKERRQ(ierr); >> PetscFunctionReturn(0); >> } >> > > I thought the idea was for the solver to come in through the context > argument of the user callback. Maybe there should be > a structured way to for SNES to modify the DM context which it passes to > the user callback. > The way the code is written, there is no user context. The user can call DMGetApplicationContext() if they want one, _but_ that should not contain a SNES or TS. The application context should only contain resolution-independent parameters.

1 0

Re: [petsc-dev] SNES, Coloring, DM, and TS
by Matthew Knepley 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 11:24 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Mon, Feb 27, 2012 at 11:21, Matthew Knepley <knepley(a)gmail.com> wrote: > >> Having DM hold the callbacks is a problem. But don't we have them in the >> context arg? >> > > PetscErrorCode SNESDMComputeJacobian(SNES snes,Vec X,Mat *J,Mat > *B,MatStructure *flag,void *ptr) > { > PetscErrorCode ierr; > DM dm; > > PetscFunctionBegin; > ierr = SNESGetDM(snes,&dm);CHKERRQ(ierr); > ierr = DMComputeJacobian(dm,X,*J,*B,flag);CHKERRQ(ierr); > PetscFunctionReturn(0); > } > I thought the idea was for the solver to come in through the context argument of the user callback. Maybe there should be a structured way to for SNES to modify the DM context which it passes to the user callback. Matt Matt > The SNES is not available from the callback. > > >> Aren't they just passed in? Or do you mean auxiliary vectors? > > > Auxiliary vectors. In this case, they are used by TS to convert the > semi-discrete form provided by the user into the fully discrete form being > solved by SNES. > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] SNES, Coloring, DM, and TS
by Jed Brown 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 11:21, Matthew Knepley <knepley(a)gmail.com> wrote: > Having DM hold the callbacks is a problem. But don't we have them in the > context arg? > PetscErrorCode SNESDMComputeJacobian(SNES snes,Vec X,Mat *J,Mat *B,MatStructure *flag,void *ptr) { PetscErrorCode ierr; DM dm; PetscFunctionBegin; ierr = SNESGetDM(snes,&dm);CHKERRQ(ierr); ierr = DMComputeJacobian(dm,X,*J,*B,flag);CHKERRQ(ierr); PetscFunctionReturn(0); } The SNES is not available from the callback. > Aren't they just passed in? Or do you mean auxiliary vectors? Auxiliary vectors. In this case, they are used by TS to convert the semi-discrete form provided by the user into the fully discrete form being solved by SNES.

1 0

Re: [petsc-dev] SNES, Coloring, DM, and TS
by Dmitry Karpeev 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 11:11 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Mon, Feb 27, 2012 at 10:55, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > >> Why is a nonlinear problem not part of the DM interface? In my mind a DM >> encapsulates a problem, that also knows >> how to linearize/split/coarsen/refine itself by returning the >> corresponding subDMs. >> > > SNES depends on DM, not the other way around. > > In some cases, the callback needs access to the TS or SNES because it is > doing some configuration or wants to hack in some logic. The DM callbacks > don't pass that parameter on. If they did pass it on, that part of the > interface could not sit in src/dm. > I agree that DM shouldn't depend on SNES or TS -- there is some awkwardness of a similar kind in libMesh. One way around this ould be to have SNESSetUp proess the (possibly changed) DM settings before every solve. Dmitry.

1 0

Re: [petsc-dev] SNES, Coloring, DM, and TS
by Matthew Knepley 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 11:16 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > On Mon, Feb 27, 2012 at 11:00, Matthew Knepley <knepley(a)gmail.com> wrote: > >> I don't see why this is a dependency loop. >> > > 1. The callback interface is duplicated. > > 2. DM callbacks should provide a way for the called code to get access to > the TS or DM so that users aren't prevented from doing hacky things. > Having DM hold the callbacks is a problem. But don't we have them in the context arg? > > >> >> >>> As a related matter, I'm stuck with a reference counting loop where I >>> want to attach special Vecs to a DM. I can wrap them in a PetscContainer to >>> circumvent the code that checks for reference cycles, but that makes >>> freeing them delicate. We could make a named cache similar to DM work >>> vectors, but where restored Vecs are not available for reuse. >>> >> >> I know that rationale for having Vecs know about a DM (output), which I >> don't like. What is the rationale for having DMs know about a certain Vec? > > > A TS wants to solve a nonlinear problem using FAS, so it needs to > implement a callback on levels. So it coarsens the grid, but it also needs > to coarsen the affine vectors so that it can define the finite-step problem > on coarse levels. > > Same issue for lots of cases where a user needs to save grid setup, > coefficient fields, etc on levels so that it is available in a callback. > It's natural to compose those vectors with the DM, but that creates a > reference loop. > Aren't they just passed in? Or do you mean auxiliary vectors? Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] the complex dot business
by Matthew Knepley 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 11:16 AM, Alexander Grayver <agrayver(a)gfz-potsdam.de > wrote: > ** > On 27.02.2012 18:01, Matthew Knepley wrote: > > On Mon, Feb 27, 2012 at 10:55 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > >> >> I'm afraid the sudden appearance of the blas dot for complex numbers >> problem is my fault. >> >> Traditionally since 199x we never used the complex dot product from >> blas because of the return complex type problem. VecDot_Seq() used #ifdef >> for complex and a simple loop. >> >> Since some blas norm suck I off-hand told Shri to use dot for norm and >> wasn't thinking the complex case. >> >> Since then, of course, the complex dot problem has been coming up. >> >> Maybe it is best to continue to never use complex blas dot and just >> change the VecNorm_ routines to use norm for complex and not dot.? Just >> throw away this silly problem and time sink? > > > Why is this such a problem? Just use PetscScalar for the BLASdot_() > return type and its fine. > > > For instance MKL's BLAS returns result for zdot in the first argument and > PETSc's BLASdot_() fails there. > Yes, there is already a configure test for that and code that fixes it. This is about interfaces. Matt > Matt > > >> >> Barry >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > > -- > Regards, > Alexander > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] SNES, Coloring, DM, and TS
by Jed Brown 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 11:00, Matthew Knepley <knepley(a)gmail.com> wrote: > I don't see why this is a dependency loop. > 1. The callback interface is duplicated. 2. DM callbacks should provide a way for the called code to get access to the TS or DM so that users aren't prevented from doing hacky things. > > >> As a related matter, I'm stuck with a reference counting loop where I >> want to attach special Vecs to a DM. I can wrap them in a PetscContainer to >> circumvent the code that checks for reference cycles, but that makes >> freeing them delicate. We could make a named cache similar to DM work >> vectors, but where restored Vecs are not available for reuse. >> > > I know that rationale for having Vecs know about a DM (output), which I > don't like. What is the rationale for having DMs know about a certain Vec? A TS wants to solve a nonlinear problem using FAS, so it needs to implement a callback on levels. So it coarsens the grid, but it also needs to coarsen the affine vectors so that it can define the finite-step problem on coarse levels. Same issue for lots of cases where a user needs to save grid setup, coefficient fields, etc on levels so that it is available in a callback. It's natural to compose those vectors with the DM, but that creates a reference loop.

1 0

Re: [petsc-dev] the complex dot business
by Alexander Grayver 27 Feb '12

27 Feb '12

On 27.02.2012 18:01, Matthew Knepley wrote: > On Mon, Feb 27, 2012 at 10:55 AM, Barry Smith <bsmith(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > > I'm afraid the sudden appearance of the blas dot for complex > numbers problem is my fault. > > Traditionally since 199x we never used the complex dot product > from blas because of the return complex type problem. VecDot_Seq() > used #ifdef for complex and a simple loop. > > Since some blas norm suck I off-hand told Shri to use dot for > norm and wasn't thinking the complex case. > > Since then, of course, the complex dot problem has been coming up. > > Maybe it is best to continue to never use complex blas dot and > just change the VecNorm_ routines to use norm for complex and not > dot.? Just throw away this silly problem and time sink? > > > Why is this such a problem? Just use PetscScalar for the BLASdot_() > return type and its fine. For instance MKL's BLAS returns result for zdot in the first argument and PETSc's BLASdot_() fails there. > > Matt > > > Barry > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -- Regards, Alexander

1 0

Re: [petsc-dev] SNES, Coloring, DM, and TS
by Jed Brown 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 10:55, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > Why is a nonlinear problem not part of the DM interface? In my mind a DM > encapsulates a problem, that also knows > how to linearize/split/coarsen/refine itself by returning the > corresponding subDMs. > SNES depends on DM, not the other way around. In some cases, the callback needs access to the TS or SNES because it is doing some configuration or wants to hack in some logic. The DM callbacks don't pass that parameter on. If they did pass it on, that part of the interface could not sit in src/dm.

1 0