February 2013 - petsc-dev - mailman3.cels.anl.gov

Vector operations on ghosted vectors
by John Mousel 27 Feb '13

27 Feb '13

Is there a possibility of adding a wrapper function around a few basic vector operations such as VecCopy, VecAXPY, VECAXPYPZ... to operate on ghosted vectors? I perform a lot of vector operations including the ghost region to avoid communication. It really clutters code to see stuff like CALL VecGhostGetLocalForm(solver%u,uk,ierr) CALL VecGhostGetLocalForm(solver%ukm1,ukm1,ierr) CALL VecGhostGetLocalForm(solver%p,pk,ierr) CALL VecGhostGetLocalForm(solver%pkm1,pkm1,ierr) CALL VecCopy(uk,ukm1,ierr) CALL VecCopy(pk,pkm1,ierr) CALL VecGhostRestoreLocalForm(solver%u,uk,ierr) CALL VecGhostRestoreLocalForm(solver%ukm1,ukm1,ierr) CALL VecGhostRestoreLocalForm(solver%p,pk,ierr) CALL VecGhostRestoreLocalForm(solver%pkm1,pkm1,ierr) Maybe there's a good reason for this not already existing, or maybe it does exist and I haven't found it, but I'd think something like VecCopyGhosted would be fairly useful. John

1 0

Re: [petsc-dev] Deflated Krylov solvers for PETSc
by Barry Smith 27 Feb '13

27 Feb '13

The PETSc Krylov methods are supposed to be implemented using KSP_MatMult(), KSP_PCApply() etc instead of directly using MatMult() and PCApply() etc. This means, in theory, one can add support for techniques that modify the application of the multiply or preconditioner in a single place and does not mean changing code in each method implementation. I've always had trouble wrapping my head around if deflation techniques should be viewed as modifications to the Krylov methods or modification (via composing) of the preconditioners. Having this completely clarified would help us understand how deflation could be added to PETSc in a systematic (and hence maintainable) way. Barry On Feb 26, 2013, at 10:58 PM, Jie Chen <jiechen(a)mcs.anl.gov> wrote: > To whomever is interested in deflated Krylov solvers: > > Deflation is very useful technique for accelerating the convergence of Krylov iterations. It seems to me that deflation is not supported in PETSc, except for the special case DGMRES, where eigenvectors are used to deflate GMRES. As a matter of fact, one does not need to use eigenvectors for deflation, nor deflation is dependent on a specific choice of the iterative method. > > Coincidentally, Kees Vuik pointed to me an earlier paper of his: > > http://ta.twi.tudelft.nl/nw/users/vuik/papers/Vui01F.pdf > > which claims that the convergence of using block Jacobi/ILU(0) [the default preconditioner for PETSc] becomes in general better if deflation is included. In his paper, deflation means "coarse grid acceleration"; see page 935, the paragraph of equation (2). > > So I think (including my personal experience), deflation is a general enough technique that may benefit every user of the KSP solver. Implementation-wise, my personal take is that the library needs to change the A-multiply to (A-AZ(Z'AZ)^{-1}Z'A)-multiply, where Z is the deflation matrix: tall, skinny, and with a small number of columns. PETSc can ask the user to define Z, and it can also predefine several choices of Z, one example choice is from the above mentioned paper. In principle, deflation should be independent of the KSP types, but I don't see how exactly it can be implemented without touching the code of every krylov solvers. Perhaps adding one more layer of abstraction is needed. > > So for anyone who is interested in implementing deflation, I can provide a more detailed tutorial. The design of PETSc is becoming more and more complicated for me to digest, thus I think it may be unrealistic for me to implement deflation myself for the moment. > > Jie > > > > -- > Jie Chen > Mathematics and Computer Science Division > Argonne National Laboratory > Address: 9700 S Cass Ave, Bldg 240, Lemont, IL 60439 > Phone: (630) 252-3313 > Email: jiechen(a)mcs.anl.gov > Homepage: http://www.mcs.anl.gov/~jiechen

1 0

Re: [petsc-dev] Use KSPSetUpNorms_Private() in KSPSetFromOptions()
by Dmitry Karpeev 27 Feb '13

27 Feb '13

On Tue, Feb 26, 2013 at 8:59 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Feb 26, 2013, at 9:54 PM, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > > > Here's what I get with this simple example (source and makefile also > attached): > > > > > > #include <petscksp.h> > > > > #undef __FUNCT__ > > #define __FUNCT__ "main" > > int main(int argc,char **args) > > { > > KSP ksp; > > PetscErrorCode ierr; > > PetscInitialize(&argc,&args,(char*)0,(char*)0); > > ierr = KSPCreate(PETSC_COMM_WORLD, &ksp);CHKERRQ(ierr); > > ierr = KSPSetPCSide(ksp, PC_RIGHT);CHKERRQ(ierr); > > ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr); > > ierr = KSPDestroy(&ksp);CHKERRQ(ierr); > > PetscFinalize(); > > } > > > > > > Then > > > > [hal3 Tue Feb 26 09:34 PM] > ~/petsc/dev/c/src/ksp/ksp/examples/tutorials/tmp >./ex -ksp_type preonly > -ksp_pc_side left > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > > [0]PETSC ERROR: No support for this operation for this object type! > > [0]PETSC ERROR: KSP preonly does not support RIGHT! > > [0]PETSC ERROR: > ------------------------------------------------------------------------ > > [0]PETSC ERROR: Petsc Development HG revision: > 55699a42ce9360fb4641931ab7ec242cba302c58 HG Date: Mon Feb 25 22:15:44 2013 > -0500 > > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > [0]PETSC ERROR: See docs/index.html for manual pages. > > [0]PETSC ERROR: > ------------------------------------------------------------------------ > > [0]PETSC ERROR: ./ex on a arch-gxx-real named hal3 by karpeev Tue Feb 26 > 21:37:13 2013 > > [0]PETSC ERROR: Libraries linked from > /home/karpeev/petsc/dev/c/arch-gxx-real/lib > > [0]PETSC ERROR: Configure run at Tue Feb 26 17:50:35 2013 > > [0]PETSC ERROR: Configure options --PETSC_ARCH=arch-gxx-real > --download-f-blas-lapack=1 --download-mpich=1 --with-cc=gcc > --with-clanguage=C++ --with-cxx=g++ --with-dynamic-loading=1 > --with-python=1 --with-scalar-type=real --with-shared-libraries=1 > > [0]PETSC ERROR: > ------------------------------------------------------------------------ > > [0]PETSC ERROR: KSPSetUpNorms_Private() line 385 in > /home/karpeev/petsc/dev/c/src/ksp/ksp/interface/itcreate.c > > [0]PETSC ERROR: KSPSetFromOptions() line 409 in > /home/karpeev/petsc/dev/c/src/ksp/ksp/interface/itcl.c > > [0]PETSC ERROR: main() line 13 in ex.c > > application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 > > [unset]: aborting job: > > application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 > > [hal3 Tue Feb 26 09:37 PM] > ~/petsc/dev/c/src/ksp/ksp/examples/tutorials/tmp > > > > > The problem seems to be this clause in KSPSetUpNorms_Private(): > > > > > > if (ksp->normtype == KSP_NORM_DEFAULT && i == KSP_NORM_NONE && > ksp->normsupporttable[i][j] <= 1) { > > continue; /* Skip because we don't want to default to no norms > unless set by the KSP (preonly). */ > > > > > > It is hit because of the code in KSPCreate_PREONLY() you quoted: > > > > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_LEFT,2);CHKERRQ(ierr); > /* LEFT/RIGHT is arbitrary, so "support" both */ > > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_RIGHT,1);CHKERRQ(ierr); > > > > The problem is that PC_RIGHT is preferred by KSP_NORM only with priority > 1, which is too weak to overcome > > that "continue" clause above. > > Ok, this is a bug, we'll leave it to Jed to fix properly since he > created this Frankensteinen monster. > Is increasing priority of PC_RIGHT to 2 in KSPPREONLY an option as an interim solution? It would be nice to patch petsc-3.3 with it, since codes like Moose are likely to keep using it for a while. Dmitry. > > > > > One way to fix it is to jack up the priority of PC_RIGHT, but I still > don't understand why this combination > > of norms and pc sides is being figured out in KSPSetFromOptions() > (twice) with only partial KSP data. > > Because it want's to give as reasonable a possible solution to the user > before the user gets to override with something else. > > > And then again, this time "for real", in KSPSetUp(). > > > > Dmitry. > > > > On Wed, Feb 27, 2013 at 2:52 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > > > On Feb 26, 2013, at 4:44 PM, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > > > > > These calls compute default values for normtype and pcside based on > partial KSP options, > > > which can cause an unwarranted error to be thrown. > > > For example, in code there may be > > > > > > KSPSetPCSide(ksp, PC_RIGHT); /* hardwired default, e.g., in Moose. */ > > > > > > Now, when I'm using command-line arguments > > > -ksp_type preonly > > > an error is thrown, since KSPPREONLY doesn't work with PC_RIGHT. > > > > Huh? > > > > EXTERN_C_BEGIN > > #undef __FUNCT__ > > #define __FUNCT__ "KSPCreate_PREONLY" > > PetscErrorCode KSPCreate_PREONLY(KSP ksp) > > { > > PetscErrorCode ierr; > > > > PetscFunctionBegin; > > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_LEFT,2);CHKERRQ(ierr); > /* LEFT/RIGHT is arbitrary, so "support" both */ > > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_RIGHT,1);CHKERRQ(ierr); > > > > Is it because MOOSE also hardwires the norm type and preonly doesn't > support it? What is the full and complete error message. > > > > > Fine, I try > > > -ksp_type preonly -ksp_pc_side left > > > but still get the same error, since KSPSetUpNorms_Private() is called > after KSP type has been set to KSPPREONLY, > > > but BEFORE there was a chance to set the PC side! > > > > > > Is there any reason to keep these calls to KSPSetUpNorms_Private() in > KSPSetFromOptions()? > > > > I think they are there so that users can overwrite them in the rest > of KSPSetFromOptions(). If you move them to KSPSetUp() then the user cannot > reset them. > > > > Barry > > > > > > > KSPSetUpNorms_Private() will be just about the first thing called in > KSPSetUp() and will compute the appropriate normtype and pcside, > > > but now based on complete KSP options. > > > > > > Not to mention that calling XXXSetUp_YYY() XXXSetFromOptions() seems > generally contrary to PETSc's design > > > philosophy. > > > > > > Dmitry. > > > > > > > > > > > > > > > > <ex.c><makefile> > >

1 0

Re: [petsc-dev] Deflated Krylov solvers for PETSc
by Jie Chen 27 Feb '13

27 Feb '13

My personal experience is to use approximate eigenvectors to deflate conjugate gradient. In that regard, it is kinda like DGMRES, except that the iterative method is different. A lot of situations I have seen use eigenvectors, but just as what Vuik does, you never know when someone will propose something unconventional. Deflation being a general framework, it does not hurt (from a user's point of view) to add something to the library and let the user play. Jie ----- Original Message ----- From: "Matthew Knepley" <knepley(a)gmail.com> To: "For users of the development version of PETSc" <petsc-dev(a)mcs.anl.gov> Sent: Wednesday, February 27, 2013 4:05:47 AM Subject: Re: [petsc-dev] Deflated Krylov solvers for PETSc I think the problem with that approach to deflation is preconditioning, unless you use injection as Vuik does. You can easily get this kind of projection by using MatShell. I tried Ronald Morgan's stuff when I was in grad school and it never made a bit of difference. Is there any situation where deflation is useful and cannot be phrased as our current MG? Matt

1 0

Re: [petsc-dev] Deflated Krylov solvers for PETSc
by Matthew Knepley 27 Feb '13

27 Feb '13

On Wed, Feb 27, 2013 at 12:20 AM, Jed Brown <jedbrown(a)mcs.anl.gov> wrote: > Interesting. I've always thought of deflation as being about the > construction of the deflation vectors rather than application of the > projection, which I think can be implemented by composing PCComposite and > PCGalerkin (or with a degenerate Galerkin two-level multigrid cycle). Note > that the DGMRES construction has some similarities to Bootstrap AMG in that > low-energy eigenvectors are processed incrementally and used to define the > new coarse grid. > I think the problem with that approach to deflation is preconditioning, unless you use injection as Vuik does. You can easily get this kind of projection by using MatShell. I tried Ronald Morgan's stuff when I was in grad school and it never made a bit of difference. Is there any situation where deflation is useful and cannot be phrased as our current MG? Matt > On Tue, Feb 26, 2013 at 11:58 PM, Jie Chen <jiechen(a)mcs.anl.gov> wrote: > >> To whomever is interested in deflated Krylov solvers: >> >> Deflation is very useful technique for accelerating the convergence of >> Krylov iterations. It seems to me that deflation is not supported in PETSc, >> except for the special case DGMRES, where eigenvectors are used to deflate >> GMRES. As a matter of fact, one does not need to use eigenvectors for >> deflation, nor deflation is dependent on a specific choice of the iterative >> method. >> >> Coincidentally, Kees Vuik pointed to me an earlier paper of his: >> >> http://ta.twi.tudelft.nl/nw/users/vuik/papers/Vui01F.pdf >> >> which claims that the convergence of using block Jacobi/ILU(0) [the >> default preconditioner for PETSc] becomes in general better if deflation is >> included. In his paper, deflation means "coarse grid acceleration"; see >> page 935, the paragraph of equation (2). >> >> So I think (including my personal experience), deflation is a general >> enough technique that may benefit every user of the KSP solver. >> Implementation-wise, my personal take is that the library needs to change >> the A-multiply to (A-AZ(Z'AZ)^{-1}Z'A)-multiply, where Z is the deflation >> matrix: tall, skinny, and with a small number of columns. PETSc can ask the >> user to define Z, and it can also predefine several choices of Z, one >> example choice is from the above mentioned paper. In principle, deflation >> should be independent of the KSP types, but I don't see how exactly it can >> be implemented without touching the code of every krylov solvers. Perhaps >> adding one more layer of abstraction is needed. >> >> So for anyone who is interested in implementing deflation, I can provide >> a more detailed tutorial. The design of PETSc is becoming more and more >> complicated for me to digest, thus I think it may be unrealistic for me to >> implement deflation myself for the moment. >> >> Jie >> >> >> >> -- >> Jie Chen >> Mathematics and Computer Science Division >> Argonne National Laboratory >> Address: 9700 S Cass Ave, Bldg 240, Lemont, IL 60439 >> Phone: (630) 252-3313 >> Email: jiechen(a)mcs.anl.gov >> Homepage: http://www.mcs.anl.gov/~jiechen >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener

1 0

Re: [petsc-dev] Deflated Krylov solvers for PETSc
by Jie Chen 26 Feb '13

26 Feb '13

Well not surprising that there are a lot of equivalences between deflation from the Krylov point of view and restriction/prolongation from the multigrid point of view. And of course constructing/refining the deflation matrix can be a central piece of research work. Vuik's paper does something simple, yet effective: the deflation Z is a matrix of 0's and 1's, where the 1's correspond to each subdomain. I do not exactly understand how to use PCComposite and PCGalerkin. But I suppose that you do not want them to appear in user's code (application code), right? Jie ----- Original Message ----- From: "Jed Brown" <jedbrown(a)mcs.anl.gov> To: "For users of the development version of PETSc" <petsc-dev(a)mcs.anl.gov> Sent: Wednesday, February 27, 2013 12:20:49 AM Subject: Re: [petsc-dev] Deflated Krylov solvers for PETSc Interesting. I've always thought of deflation as being about the construction of the deflation vectors rather than application of the projection, which I think can be implemented by composing PCComposite and PCGalerkin (or with a degenerate Galerkin two-level multigrid cycle). Note that the DGMRES construction has some similarities to Bootstrap AMG in that low-energy eigenvectors are processed incrementally and used to define the new coarse grid. On Tue, Feb 26, 2013 at 11:58 PM, Jie Chen < jiechen(a)mcs.anl.gov > wrote: To whomever is interested in deflated Krylov solvers: Deflation is very useful technique for accelerating the convergence of Krylov iterations. It seems to me that deflation is not supported in PETSc, except for the special case DGMRES, where eigenvectors are used to deflate GMRES. As a matter of fact, one does not need to use eigenvectors for deflation, nor deflation is dependent on a specific choice of the iterative method. Coincidentally, Kees Vuik pointed to me an earlier paper of his: http://ta.twi.tudelft.nl/nw/users/vuik/papers/Vui01F.pdf which claims that the convergence of using block Jacobi/ILU(0) [the default preconditioner for PETSc] becomes in general better if deflation is included. In his paper, deflation means "coarse grid acceleration"; see page 935, the paragraph of equation (2). So I think (including my personal experience), deflation is a general enough technique that may benefit every user of the KSP solver. Implementation-wise, my personal take is that the library needs to change the A-multiply to (A-AZ(Z'AZ)^{-1}Z'A)-multiply, where Z is the deflation matrix: tall, skinny, and with a small number of columns. PETSc can ask the user to define Z, and it can also predefine several choices of Z, one example choice is from the above mentioned paper. In principle, deflation should be independent of the KSP types, but I don't see how exactly it can be implemented without touching the code of every krylov solvers. Perhaps adding one more layer of abstraction is needed. So for anyone who is interested in implementing deflation, I can provide a more detailed tutorial. The design of PETSc is becoming more and more complicated for me to digest, thus I think it may be unrealistic for me to implement deflation myself for the moment. Jie -- Jie Chen Mathematics and Computer Science Division Argonne National Laboratory Address: 9700 S Cass Ave, Bldg 240, Lemont, IL 60439 Phone: (630) 252-3313 Email: jiechen(a)mcs.anl.gov Homepage: http://www.mcs.anl.gov/~jiechen

1 0

Re: [petsc-dev] Deflated Krylov solvers for PETSc
by Jed Brown 26 Feb '13

26 Feb '13

Interesting. I've always thought of deflation as being about the construction of the deflation vectors rather than application of the projection, which I think can be implemented by composing PCComposite and PCGalerkin (or with a degenerate Galerkin two-level multigrid cycle). Note that the DGMRES construction has some similarities to Bootstrap AMG in that low-energy eigenvectors are processed incrementally and used to define the new coarse grid. On Tue, Feb 26, 2013 at 11:58 PM, Jie Chen <jiechen(a)mcs.anl.gov> wrote: > To whomever is interested in deflated Krylov solvers: > > Deflation is very useful technique for accelerating the convergence of > Krylov iterations. It seems to me that deflation is not supported in PETSc, > except for the special case DGMRES, where eigenvectors are used to deflate > GMRES. As a matter of fact, one does not need to use eigenvectors for > deflation, nor deflation is dependent on a specific choice of the iterative > method. > > Coincidentally, Kees Vuik pointed to me an earlier paper of his: > > http://ta.twi.tudelft.nl/nw/users/vuik/papers/Vui01F.pdf > > which claims that the convergence of using block Jacobi/ILU(0) [the > default preconditioner for PETSc] becomes in general better if deflation is > included. In his paper, deflation means "coarse grid acceleration"; see > page 935, the paragraph of equation (2). > > So I think (including my personal experience), deflation is a general > enough technique that may benefit every user of the KSP solver. > Implementation-wise, my personal take is that the library needs to change > the A-multiply to (A-AZ(Z'AZ)^{-1}Z'A)-multiply, where Z is the deflation > matrix: tall, skinny, and with a small number of columns. PETSc can ask the > user to define Z, and it can also predefine several choices of Z, one > example choice is from the above mentioned paper. In principle, deflation > should be independent of the KSP types, but I don't see how exactly it can > be implemented without touching the code of every krylov solvers. Perhaps > adding one more layer of abstraction is needed. > > So for anyone who is interested in implementing deflation, I can provide a > more detailed tutorial. The design of PETSc is becoming more and more > complicated for me to digest, thus I think it may be unrealistic for me to > implement deflation myself for the moment. > > Jie > > > > -- > Jie Chen > Mathematics and Computer Science Division > Argonne National Laboratory > Address: 9700 S Cass Ave, Bldg 240, Lemont, IL 60439 > Phone: (630) 252-3313 > Email: jiechen(a)mcs.anl.gov > Homepage: http://www.mcs.anl.gov/~jiechen >

1 0

Deflated Krylov solvers for PETSc
by Jie Chen 26 Feb '13

26 Feb '13

To whomever is interested in deflated Krylov solvers: Deflation is very useful technique for accelerating the convergence of Krylov iterations. It seems to me that deflation is not supported in PETSc, except for the special case DGMRES, where eigenvectors are used to deflate GMRES. As a matter of fact, one does not need to use eigenvectors for deflation, nor deflation is dependent on a specific choice of the iterative method. Coincidentally, Kees Vuik pointed to me an earlier paper of his: http://ta.twi.tudelft.nl/nw/users/vuik/papers/Vui01F.pdf which claims that the convergence of using block Jacobi/ILU(0) [the default preconditioner for PETSc] becomes in general better if deflation is included. In his paper, deflation means "coarse grid acceleration"; see page 935, the paragraph of equation (2). So I think (including my personal experience), deflation is a general enough technique that may benefit every user of the KSP solver. Implementation-wise, my personal take is that the library needs to change the A-multiply to (A-AZ(Z'AZ)^{-1}Z'A)-multiply, where Z is the deflation matrix: tall, skinny, and with a small number of columns. PETSc can ask the user to define Z, and it can also predefine several choices of Z, one example choice is from the above mentioned paper. In principle, deflation should be independent of the KSP types, but I don't see how exactly it can be implemented without touching the code of every krylov solvers. Perhaps adding one more layer of abstraction is needed. So for anyone who is interested in implementing deflation, I can provide a more detailed tutorial. The design of PETSc is becoming more and more complicated for me to digest, thus I think it may be unrealistic for me to implement deflation myself for the moment. Jie -- Jie Chen Mathematics and Computer Science Division Argonne National Laboratory Address: 9700 S Cass Ave, Bldg 240, Lemont, IL 60439 Phone: (630) 252-3313 Email: jiechen(a)mcs.anl.gov Homepage: http://www.mcs.anl.gov/~jiechen

1 0

Re: [petsc-dev] Use KSPSetUpNorms_Private() in KSPSetFromOptions()
by Barry Smith 26 Feb '13

26 Feb '13

On Feb 26, 2013, at 9:54 PM, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > Here's what I get with this simple example (source and makefile also attached): > > > #include <petscksp.h> > > #undef __FUNCT__ > #define __FUNCT__ "main" > int main(int argc,char **args) > { > KSP ksp; > PetscErrorCode ierr; > PetscInitialize(&argc,&args,(char*)0,(char*)0); > ierr = KSPCreate(PETSC_COMM_WORLD, &ksp);CHKERRQ(ierr); > ierr = KSPSetPCSide(ksp, PC_RIGHT);CHKERRQ(ierr); > ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr); > ierr = KSPDestroy(&ksp);CHKERRQ(ierr); > PetscFinalize(); > } > > > Then > > [hal3 Tue Feb 26 09:34 PM] ~/petsc/dev/c/src/ksp/ksp/examples/tutorials/tmp >./ex -ksp_type preonly -ksp_pc_side left > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: No support for this operation for this object type! > [0]PETSC ERROR: KSP preonly does not support RIGHT! > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development HG revision: 55699a42ce9360fb4641931ab7ec242cba302c58 HG Date: Mon Feb 25 22:15:44 2013 -0500 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: ./ex on a arch-gxx-real named hal3 by karpeev Tue Feb 26 21:37:13 2013 > [0]PETSC ERROR: Libraries linked from /home/karpeev/petsc/dev/c/arch-gxx-real/lib > [0]PETSC ERROR: Configure run at Tue Feb 26 17:50:35 2013 > [0]PETSC ERROR: Configure options --PETSC_ARCH=arch-gxx-real --download-f-blas-lapack=1 --download-mpich=1 --with-cc=gcc --with-clanguage=C++ --with-cxx=g++ --with-dynamic-loading=1 --with-python=1 --with-scalar-type=real --with-shared-libraries=1 > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: KSPSetUpNorms_Private() line 385 in /home/karpeev/petsc/dev/c/src/ksp/ksp/interface/itcreate.c > [0]PETSC ERROR: KSPSetFromOptions() line 409 in /home/karpeev/petsc/dev/c/src/ksp/ksp/interface/itcl.c > [0]PETSC ERROR: main() line 13 in ex.c > application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 > [unset]: aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 > [hal3 Tue Feb 26 09:37 PM] ~/petsc/dev/c/src/ksp/ksp/examples/tutorials/tmp > > > The problem seems to be this clause in KSPSetUpNorms_Private(): > > > if (ksp->normtype == KSP_NORM_DEFAULT && i == KSP_NORM_NONE && ksp->normsupporttable[i][j] <= 1) { > continue; /* Skip because we don't want to default to no norms unless set by the KSP (preonly). */ > > > It is hit because of the code in KSPCreate_PREONLY() you quoted: > > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_LEFT,2);CHKERRQ(ierr); /* LEFT/RIGHT is arbitrary, so "support" both */ > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_RIGHT,1);CHKERRQ(ierr); > > The problem is that PC_RIGHT is preferred by KSP_NORM only with priority 1, which is too weak to overcome > that "continue" clause above. Ok, this is a bug, we'll leave it to Jed to fix properly since he created this Frankensteinen monster. > > One way to fix it is to jack up the priority of PC_RIGHT, but I still don't understand why this combination > of norms and pc sides is being figured out in KSPSetFromOptions() (twice) with only partial KSP data. Because it want's to give as reasonable a possible solution to the user before the user gets to override with something else. > And then again, this time "for real", in KSPSetUp(). > > Dmitry. > > On Wed, Feb 27, 2013 at 2:52 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Feb 26, 2013, at 4:44 PM, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > > > These calls compute default values for normtype and pcside based on partial KSP options, > > which can cause an unwarranted error to be thrown. > > For example, in code there may be > > > > KSPSetPCSide(ksp, PC_RIGHT); /* hardwired default, e.g., in Moose. */ > > > > Now, when I'm using command-line arguments > > -ksp_type preonly > > an error is thrown, since KSPPREONLY doesn't work with PC_RIGHT. > > Huh? > > EXTERN_C_BEGIN > #undef __FUNCT__ > #define __FUNCT__ "KSPCreate_PREONLY" > PetscErrorCode KSPCreate_PREONLY(KSP ksp) > { > PetscErrorCode ierr; > > PetscFunctionBegin; > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_LEFT,2);CHKERRQ(ierr); /* LEFT/RIGHT is arbitrary, so "support" both */ > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_RIGHT,1);CHKERRQ(ierr); > > Is it because MOOSE also hardwires the norm type and preonly doesn't support it? What is the full and complete error message. > > > Fine, I try > > -ksp_type preonly -ksp_pc_side left > > but still get the same error, since KSPSetUpNorms_Private() is called after KSP type has been set to KSPPREONLY, > > but BEFORE there was a chance to set the PC side! > > > > Is there any reason to keep these calls to KSPSetUpNorms_Private() in KSPSetFromOptions()? > > I think they are there so that users can overwrite them in the rest of KSPSetFromOptions(). If you move them to KSPSetUp() then the user cannot reset them. > > Barry > > > > KSPSetUpNorms_Private() will be just about the first thing called in KSPSetUp() and will compute the appropriate normtype and pcside, > > but now based on complete KSP options. > > > > Not to mention that calling XXXSetUp_YYY() XXXSetFromOptions() seems generally contrary to PETSc's design > > philosophy. > > > > Dmitry. > > > > > > > > > <ex.c><makefile>

1 0

Re: [petsc-dev] Use KSPSetUpNorms_Private() in KSPSetFromOptions()
by Dmitry Karpeev 26 Feb '13

26 Feb '13

Here's what I get with this simple example (source and makefile also attached): #include <petscksp.h> #undef __FUNCT__ #define __FUNCT__ "main" int main(int argc,char **args) { KSP ksp; PetscErrorCode ierr; PetscInitialize(&argc,&args,(char*)0,(char*)0); ierr = KSPCreate(PETSC_COMM_WORLD, &ksp);CHKERRQ(ierr); ierr = KSPSetPCSide(ksp, PC_RIGHT);CHKERRQ(ierr); ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr); ierr = KSPDestroy(&ksp);CHKERRQ(ierr); PetscFinalize(); } Then [hal3 Tue Feb 26 09:34 PM] ~/petsc/dev/c/src/ksp/ksp/examples/tutorials/tmp >./ex -ksp_type preonly -ksp_pc_side left [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: No support for this operation for this object type! [0]PETSC ERROR: KSP preonly does not support RIGHT! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: 55699a42ce9360fb4641931ab7ec242cba302c58 HG Date: Mon Feb 25 22:15:44 2013 -0500 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./ex on a arch-gxx-real named hal3 by karpeev Tue Feb 26 21:37:13 2013 [0]PETSC ERROR: Libraries linked from /home/karpeev/petsc/dev/c/arch-gxx-real/lib [0]PETSC ERROR: Configure run at Tue Feb 26 17:50:35 2013 [0]PETSC ERROR: Configure options --PETSC_ARCH=arch-gxx-real --download-f-blas-lapack=1 --download-mpich=1 --with-cc=gcc --with-clanguage=C++ --with-cxx=g++ --with-dynamic-loading=1 --with-python=1 --with-scalar-type=real --with-shared-libraries=1 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: KSPSetUpNorms_Private() line 385 in /home/karpeev/petsc/dev/c/src/ksp/ksp/interface/itcreate.c [0]PETSC ERROR: KSPSetFromOptions() line 409 in /home/karpeev/petsc/dev/c/src/ksp/ksp/interface/itcl.c [0]PETSC ERROR: main() line 13 in ex.c application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 [unset]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 [hal3 Tue Feb 26 09:37 PM] ~/petsc/dev/c/src/ksp/ksp/examples/tutorials/tmp > The problem seems to be this clause in KSPSetUpNorms_Private(): if (ksp->normtype == KSP_NORM_DEFAULT && i == KSP_NORM_NONE && ksp->normsupporttable[i][j] <= 1) { continue; /* Skip because we don't want to default to no norms unless set by the KSP (preonly). */ It is hit because of the code in KSPCreate_PREONLY() you quoted: ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_LEFT,2);CHKERRQ(ierr); /* LEFT/RIGHT is arbitrary, so "support" both */ ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_RIGHT,1);CHKERRQ(ierr); The problem is that PC_RIGHT is preferred by KSP_NORM only with priority 1, which is too weak to overcome that "continue" clause above. One way to fix it is to jack up the priority of PC_RIGHT, but I still don't understand why this combination of norms and pc sides is being figured out in KSPSetFromOptions() (twice) with only partial KSP data. And then again, this time "for real", in KSPSetUp(). Dmitry. On Wed, Feb 27, 2013 at 2:52 AM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: > > On Feb 26, 2013, at 4:44 PM, Dmitry Karpeev <karpeev(a)mcs.anl.gov> wrote: > > > These calls compute default values for normtype and pcside based on > partial KSP options, > > which can cause an unwarranted error to be thrown. > > For example, in code there may be > > > > KSPSetPCSide(ksp, PC_RIGHT); /* hardwired default, e.g., in Moose. */ > > > > Now, when I'm using command-line arguments > > -ksp_type preonly > > an error is thrown, since KSPPREONLY doesn't work with PC_RIGHT. > > Huh? > > EXTERN_C_BEGIN > #undef __FUNCT__ > #define __FUNCT__ "KSPCreate_PREONLY" > PetscErrorCode KSPCreate_PREONLY(KSP ksp) > { > PetscErrorCode ierr; > > PetscFunctionBegin; > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_LEFT,2);CHKERRQ(ierr); > /* LEFT/RIGHT is arbitrary, so "support" both */ > ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_RIGHT,1);CHKERRQ(ierr); > > Is it because MOOSE also hardwires the norm type and preonly doesn't > support it? What is the full and complete error message. > > > Fine, I try > > -ksp_type preonly -ksp_pc_side left > > but still get the same error, since KSPSetUpNorms_Private() is called > after KSP type has been set to KSPPREONLY, > > but BEFORE there was a chance to set the PC side! > > > > Is there any reason to keep these calls to KSPSetUpNorms_Private() in > KSPSetFromOptions()? > > I think they are there so that users can overwrite them in the rest of > KSPSetFromOptions(). If you move them to KSPSetUp() then the user cannot > reset them. > > Barry > > > > KSPSetUpNorms_Private() will be just about the first thing called in > KSPSetUp() and will compute the appropriate normtype and pcside, > > but now based on complete KSP options. > > > > Not to mention that calling XXXSetUp_YYY() XXXSetFromOptions() seems > generally contrary to PETSc's design > > philosophy. > > > > Dmitry. > > > > > >

1 0