August 2020 - petsc-dev - mailman3.cels.anl.gov

PetscContainerSetPointer
by Mark Adams 10 Aug '20

10 Aug '20

I had some problems giving a container an PETSc object so I wrapped it in a struct. I forget what problem I had. Maybe ISColoringDestroy can't take a void*. I'd like to check if I was just missing something. Should something like this work? ISColoring c; .... ierr = PetscContainerSetPointer(container,(void*)c);CHKERRQ(ierr); ierr = PetscContainerSetUserDestroy(container, ISColoringDestroy);CHKERRQ(ierr);

1 0

Re: [petsc-dev] Use -Wshadow on pipelines?
by Jed Brown 09 Aug '20

09 Aug '20

After adding support for C99, I'm in favor of using for loop declarations for (PetscInt i=0; i<n; i++) whenever possible. However, I don't think it's worth refactoring existing code, especially if it isn't automated. I can imagine some day clang-tidy might be capable of automating such a transformation across an entire project without breaking other PETSc conventions (especially with respect to CHKERRQ), at which point we could consider whether the modest clarity improvement is worth the code churn. At most, I would enable (and fix) -Wshadow=compatible-local at this time. Jacob Faibussowitsch <jacob.fai(a)gmail.com> writes: > Hello All, > > I discovered a rather obscure set of compiler flags -Wshadow, -Wshadow=local, -Wshadow=compatible-local while trying to track down a bug which warn about local redefinition of a variable. Out of curiosity I also recompiled petsc with this flag and lo and behold the src is littered with shadowed variables. From what I can tell most are likely harmless — defining an iterator variable in the beginning of the function with all the rest of the variables, then redefining it later in some loop — but maybe we should consider adding this as a CI flag as it isn’t activated by -Wall. > > Best regards, > > Jacob Faibussowitsch > (Jacob Fai - booss - oh - vitch) > Cell: (312) 694-3391

1 0

Re: [petsc-dev] Use -Wshadow on pipelines?
by Barry Smith 09 Aug '20

09 Aug '20

Agree > On Aug 9, 2020, at 6:55 PM, Jacob Faibussowitsch <jacob.fai(a)gmail.com> wrote: > > Hello All, > > I discovered a rather obscure set of compiler flags -Wshadow, -Wshadow=local, -Wshadow=compatible-local while trying to track down a bug which warn about local redefinition of a variable. Out of curiosity I also recompiled petsc with this flag and lo and behold the src is littered with shadowed variables. From what I can tell most are likely harmless — defining an iterator variable in the beginning of the function with all the rest of the variables, then redefining it later in some loop — but maybe we should consider adding this as a CI flag as it isn’t activated by -Wall. > > Best regards, > > Jacob Faibussowitsch > (Jacob Fai - booss - oh - vitch) > Cell: (312) 694-3391 >

1 0

Use -Wshadow on pipelines?
by Jacob Faibussowitsch 09 Aug '20

09 Aug '20

Hello All, I discovered a rather obscure set of compiler flags -Wshadow, -Wshadow=local, -Wshadow=compatible-local while trying to track down a bug which warn about local redefinition of a variable. Out of curiosity I also recompiled petsc with this flag and lo and behold the src is littered with shadowed variables. From what I can tell most are likely harmless — defining an iterator variable in the beginning of the function with all the rest of the variables, then redefining it later in some loop — but maybe we should consider adding this as a CI flag as it isn’t activated by -Wall. Best regards, Jacob Faibussowitsch (Jacob Fai - booss - oh - vitch) Cell: (312) 694-3391

1 0

*****SPAM*****https://clementsam.tech/glab/
by Barry Smith 05 Aug '20

05 Aug '20

1 0

Re: [petsc-dev] TAOPDIPM
by Abhyankar, Shrirang G 05 Aug '20

05 Aug '20

Hi Pierre, We have not tested PDIPM without equality (or inequality) constraints. I’ll take a look at this over the weekend. This is the first version of PDIPM and there are many bells and whistles that need to be added before making PDIPM fully functional for solving large problems. We’ve internally discussed the different pieces needed to make it robust, but we haven’t had time to work on it yet. Thanks, Shri From: Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> Date: Wednesday, August 5, 2020 at 2:11 AM To: "Abhyankar, Shrirang G" <shrirang.abhyankar(a)pnnl.gov> Cc: "Dener, Alp" <adener(a)anl.gov>, PETSc Development <petsc-dev(a)mcs.anl.gov> Subject: Re: [petsc-dev] TAOPDIPM Sorry for these questions, I’m trying to help the said user transitioning from IPOPT over to Tao, and since I’m no expert I run into a bunch of errors myself. For example, if I run PDIPM without equality constraint, I now get: [0]PETSC ERROR: VecSet() line 522 in src/vec/vec/interface/rvector.c Null Object: Parameter # 1 Indeed, I can see that lambdae is conditionally created line 815, but always used in VecSet() line 522. I think the method is general enough to handle cases without such constraints, and should not thus fail here, right? You should be able to reproduce this by commenting all DE/De/EqualityConstraints/JacobianEquality code from ex1.c. Thanks, Pierre On 5 Aug 2020, at 5:27 AM, Abhyankar, Shrirang G <shrirang.abhyankar(a)pnnl.gov<mailto:[email protected]>> wrote: Pierre, Thanks for catching this issue. As Alp pointed out, the issue was because the tolerance was not set correctly. @Alp: Let me know if I can help you with the patch. Thanks, Shri From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov<mailto:[email protected]>> on behalf of Pierre Jolivet <pierre.jolivet(a)enseeiht.fr<mailto:[email protected]>> Date: Tuesday, August 4, 2020 at 1:46 PM To: "Dener, Alp" <adener(a)anl.gov<mailto:[email protected]>> Cc: PETSc Development <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>> Subject: Re: [petsc-dev] TAOPDIPM Thanks to you both, I’ll forward this to my user. Pierre On 4 Aug 2020, at 7:30 PM, Dener, Alp <adener(a)anl.gov<mailto:[email protected]>> wrote: Hi Pierre, This is indeed an issue with TAO default tolerances. Specifically it has to do with constraint tolerances. The default value is an exact zero. The problem should still work with line 89 commented out. Our default gradient tolerance is 1e-8. In your case, commenting out line 90 is causing the solver to try to converge constraints to exact zero and it cannot. I will get a patch out for this within the week but in the meantime please ensure that the constraint tolerances are set for any constrained problem. Thank you! — Alp On Aug 4, 2020, at 12:24 PM, Munson, Todd via petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>> wrote: Hi Pierre, I would say the answer to that question is "no, its not expected". We will look into fixing it. It seems like the default tolerances are being set to zero and the result is an inability to satisfy the constraints or gradient of the Lagrangian to that small of a tolerance. Thanks for point this out to us! We will get it resolved. Thanks, Todd. On 8/4/20, 12:12 PM, "petsc-dev on behalf of Pierre Jolivet" <petsc-dev-bounces(a)mcs.anl.gov<mailto:[email protected]> on behalf of pierre.jolivet(a)enseeiht.fr<mailto:[email protected]>> wrote: Hello, If I comment line 89 and 90 of src/tao/constrained/tutorials/ex1.c from master, the example deadlocks with a single process. Is this expected? Thanks, Pierre $ ./ex1 -tao_monitor ---- Constrained Problem ----- Solution should be f(1,1)=-2 0 TAO, Function value: 8., Residual: 8.48528 1 TAO, Function value: 1.28532, Residual: 10.7411 2 TAO, Function value: -2.56703, Residual: 1.87847 3 TAO, Function value: -2.03161, Residual: 0.12881 4 TAO, Function value: -1.99961, Residual: 0.0450855 5 TAO, Function value: -1.99993, Residual: 0.000916939 6 TAO, Function value: -1.99999, Residual: 6.69184e-05 7 TAO, Function value: -2., Residual: 7.15427e-06 8 TAO, Function value: -2., Residual: 7.15779e-07 9 TAO, Function value: -2., Residual: 7.15777e-08 10 TAO, Function value: -2., Residual: 7.15777e-09 11 TAO, Function value: -2., Residual: 7.15779e-10 12 TAO, Function value: -2., Residual: 7.15775e-11 13 TAO, Function value: -2., Residual: 7.1599e-12 14 TAO, Function value: -2., Residual: 7.1599e-13 15 TAO, Function value: -2., Residual: 7.22085e-14 16 TAO, Function value: -2., Residual: 6.44626e-15 17 TAO, Function value: -2., Residual: 1.00751e-15 18 TAO, Function value: -2., Residual: 1.70295e-17 19 TAO, Function value: -2., Residual: 1.70295e-18 20 TAO, Function value: -2., Residual: 1.70295e-19 21 TAO, Function value: -2., Residual: 1.70295e-20 22 TAO, Function value: -2., Residual: 1.70295e-21 23 TAO, Function value: -2., Residual: 1.70295e-22 24 TAO, Function value: -2., Residual: 1.70295e-23 25 TAO, Function value: -2., Residual: 1.70295e-24 26 TAO, Function value: -2., Residual: 1.70295e-25 27 TAO, Function value: -2., Residual: 1.70295e-26 28 TAO, Function value: -2., Residual: 1.70295e-27 29 TAO, Function value: -2., Residual: 1.70295e-28 30 TAO, Function value: -2., Residual: 1.70295e-29 31 TAO, Function value: -2., Residual: 1.70295e-30 32 TAO, Function value: -2., Residual: 1.70295e-31 33 TAO, Function value: -2., Residual: 1.70295e-32 34 TAO, Function value: -2., Residual: 1.70295e-33 35 TAO, Function value: -2., Residual: 1.70295e-34 36 TAO, Function value: -2., Residual: 1.70295e-35 37 TAO, Function value: -2., Residual: 1.70295e-36 38 TAO, Function value: -2., Residual: 1.70295e-37 39 TAO, Function value: -2., Residual: 1.70295e-38 40 TAO, Function value: -2., Residual: 1.70295e-39 41 TAO, Function value: -2., Residual: 1.70295e-40 42 TAO, Function value: -2., Residual: 1.70295e-41 43 TAO, Function value: -2., Residual: 1.70295e-42 44 TAO, Function value: -2., Residual: 1.70295e-43 45 TAO, Function value: -2., Residual: 1.70295e-44 46 TAO, Function value: -2., Residual: 1.70295e-45 47 TAO, Function value: -2., Residual: 1.70295e-46 48 TAO, Function value: -2., Residual: 1.70295e-47 49 TAO, Function value: -2., Residual: 1.70295e-48 50 TAO, Function value: -2., Residual: 1.70295e-49 51 TAO, Function value: -2., Residual: 1.70295e-50 52 TAO, Function value: -2., Residual: 1.70295e-51 ^C

1 0

Re: [petsc-dev] TAOPDIPM
by Pierre Jolivet 05 Aug '20

05 Aug '20

Sorry for these questions, I’m trying to help the said user transitioning from IPOPT over to Tao, and since I’m no expert I run into a bunch of errors myself. For example, if I run PDIPM without equality constraint, I now get: [0]PETSC ERROR: VecSet() line 522 in src/vec/vec/interface/rvector.c Null Object: Parameter # 1 Indeed, I can see that lambdae is conditionally created line 815, but always used in VecSet() line 522. I think the method is general enough to handle cases without such constraints, and should not thus fail here, right? You should be able to reproduce this by commenting all DE/De/EqualityConstraints/JacobianEquality code from ex1.c. Thanks, Pierre > On 5 Aug 2020, at 5:27 AM, Abhyankar, Shrirang G <shrirang.abhyankar(a)pnnl.gov> wrote: > > Pierre, > Thanks for catching this issue. As Alp pointed out, the issue was because the tolerance was not set correctly. > > @Alp: Let me know if I can help you with the patch. > > Thanks, > Shri > > > From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov> on behalf of Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> > Date: Tuesday, August 4, 2020 at 1:46 PM > To: "Dener, Alp" <adener(a)anl.gov> > Cc: PETSc Development <petsc-dev(a)mcs.anl.gov> > Subject: Re: [petsc-dev] TAOPDIPM > > Thanks to you both, I’ll forward this to my user. > Pierre > >> On 4 Aug 2020, at 7:30 PM, Dener, Alp <adener(a)anl.gov <mailto:[email protected]>> wrote: >> >> Hi Pierre, >> >> This is indeed an issue with TAO default tolerances. >> >> Specifically it has to do with constraint tolerances. The default value is an exact zero. >> >> The problem should still work with line 89 commented out. Our default gradient tolerance is 1e-8. In your case, commenting out line 90 is causing the solver to try to converge constraints to exact zero and it cannot. >> >> I will get a patch out for this within the week but in the meantime please ensure that the constraint tolerances are set for any constrained problem. >> >> Thank you! >> — >> Alp >> >>> On Aug 4, 2020, at 12:24 PM, Munson, Todd via petsc-dev <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>> wrote: >>> >>> >>> Hi Pierre, >>> >>> I would say the answer to that question is "no, its not expected". >>> >>> We will look into fixing it. It seems like the default tolerances are being set to zero and the result is an inability to satisfy the constraints or gradient of the Lagrangian to that small of a tolerance. >>> >>> Thanks for point this out to us! We will get it resolved. >>> >>> Thanks, Todd. >>> >>> On 8/4/20, 12:12 PM, "petsc-dev on behalf of Pierre Jolivet" <petsc-dev-bounces(a)mcs.anl.gov <mailto:[email protected]> on behalf of pierre.jolivet(a)enseeiht.fr <mailto:[email protected]>> wrote: >>> >>> Hello, >>> If I comment line 89 and 90 of src/tao/constrained/tutorials/ex1.c from master, the example deadlocks with a single process. >>> Is this expected? >>> >>> Thanks, >>> Pierre >>> >>> $ ./ex1 -tao_monitor >>> ---- Constrained Problem ----- >>> Solution should be f(1,1)=-2 >>> 0 TAO, Function value: 8., Residual: 8.48528 >>> 1 TAO, Function value: 1.28532, Residual: 10.7411 >>> 2 TAO, Function value: -2.56703, Residual: 1.87847 >>> 3 TAO, Function value: -2.03161, Residual: 0.12881 >>> 4 TAO, Function value: -1.99961, Residual: 0.0450855 >>> 5 TAO, Function value: -1.99993, Residual: 0.000916939 >>> 6 TAO, Function value: -1.99999, Residual: 6.69184e-05 >>> 7 TAO, Function value: -2., Residual: 7.15427e-06 >>> 8 TAO, Function value: -2., Residual: 7.15779e-07 >>> 9 TAO, Function value: -2., Residual: 7.15777e-08 >>> 10 TAO, Function value: -2., Residual: 7.15777e-09 >>> 11 TAO, Function value: -2., Residual: 7.15779e-10 >>> 12 TAO, Function value: -2., Residual: 7.15775e-11 >>> 13 TAO, Function value: -2., Residual: 7.1599e-12 >>> 14 TAO, Function value: -2., Residual: 7.1599e-13 >>> 15 TAO, Function value: -2., Residual: 7.22085e-14 >>> 16 TAO, Function value: -2., Residual: 6.44626e-15 >>> 17 TAO, Function value: -2., Residual: 1.00751e-15 >>> 18 TAO, Function value: -2., Residual: 1.70295e-17 >>> 19 TAO, Function value: -2., Residual: 1.70295e-18 >>> 20 TAO, Function value: -2., Residual: 1.70295e-19 >>> 21 TAO, Function value: -2., Residual: 1.70295e-20 >>> 22 TAO, Function value: -2., Residual: 1.70295e-21 >>> 23 TAO, Function value: -2., Residual: 1.70295e-22 >>> 24 TAO, Function value: -2., Residual: 1.70295e-23 >>> 25 TAO, Function value: -2., Residual: 1.70295e-24 >>> 26 TAO, Function value: -2., Residual: 1.70295e-25 >>> 27 TAO, Function value: -2., Residual: 1.70295e-26 >>> 28 TAO, Function value: -2., Residual: 1.70295e-27 >>> 29 TAO, Function value: -2., Residual: 1.70295e-28 >>> 30 TAO, Function value: -2., Residual: 1.70295e-29 >>> 31 TAO, Function value: -2., Residual: 1.70295e-30 >>> 32 TAO, Function value: -2., Residual: 1.70295e-31 >>> 33 TAO, Function value: -2., Residual: 1.70295e-32 >>> 34 TAO, Function value: -2., Residual: 1.70295e-33 >>> 35 TAO, Function value: -2., Residual: 1.70295e-34 >>> 36 TAO, Function value: -2., Residual: 1.70295e-35 >>> 37 TAO, Function value: -2., Residual: 1.70295e-36 >>> 38 TAO, Function value: -2., Residual: 1.70295e-37 >>> 39 TAO, Function value: -2., Residual: 1.70295e-38 >>> 40 TAO, Function value: -2., Residual: 1.70295e-39 >>> 41 TAO, Function value: -2., Residual: 1.70295e-40 >>> 42 TAO, Function value: -2., Residual: 1.70295e-41 >>> 43 TAO, Function value: -2., Residual: 1.70295e-42 >>> 44 TAO, Function value: -2., Residual: 1.70295e-43 >>> 45 TAO, Function value: -2., Residual: 1.70295e-44 >>> 46 TAO, Function value: -2., Residual: 1.70295e-45 >>> 47 TAO, Function value: -2., Residual: 1.70295e-46 >>> 48 TAO, Function value: -2., Residual: 1.70295e-47 >>> 49 TAO, Function value: -2., Residual: 1.70295e-48 >>> 50 TAO, Function value: -2., Residual: 1.70295e-49 >>> 51 TAO, Function value: -2., Residual: 1.70295e-50 >>> 52 TAO, Function value: -2., Residual: 1.70295e-51 >>> ^C

1 0

Re: [petsc-dev] TAOPDIPM
by Abhyankar, Shrirang G 04 Aug '20

04 Aug '20

Pierre, Thanks for catching this issue. As Alp pointed out, the issue was because the tolerance was not set correctly. @Alp: Let me know if I can help you with the patch. Thanks, Shri From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov> on behalf of Pierre Jolivet <pierre.jolivet(a)enseeiht.fr> Date: Tuesday, August 4, 2020 at 1:46 PM To: "Dener, Alp" <adener(a)anl.gov> Cc: PETSc Development <petsc-dev(a)mcs.anl.gov> Subject: Re: [petsc-dev] TAOPDIPM Thanks to you both, I’ll forward this to my user. Pierre On 4 Aug 2020, at 7:30 PM, Dener, Alp <adener(a)anl.gov<mailto:[email protected]>> wrote: Hi Pierre, This is indeed an issue with TAO default tolerances. Specifically it has to do with constraint tolerances. The default value is an exact zero. The problem should still work with line 89 commented out. Our default gradient tolerance is 1e-8. In your case, commenting out line 90 is causing the solver to try to converge constraints to exact zero and it cannot. I will get a patch out for this within the week but in the meantime please ensure that the constraint tolerances are set for any constrained problem. Thank you! — Alp On Aug 4, 2020, at 12:24 PM, Munson, Todd via petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>> wrote: Hi Pierre, I would say the answer to that question is "no, its not expected". We will look into fixing it. It seems like the default tolerances are being set to zero and the result is an inability to satisfy the constraints or gradient of the Lagrangian to that small of a tolerance. Thanks for point this out to us! We will get it resolved. Thanks, Todd. On 8/4/20, 12:12 PM, "petsc-dev on behalf of Pierre Jolivet" <petsc-dev-bounces(a)mcs.anl.gov<mailto:[email protected]> on behalf of pierre.jolivet(a)enseeiht.fr<mailto:[email protected]>> wrote: Hello, If I comment line 89 and 90 of src/tao/constrained/tutorials/ex1.c from master, the example deadlocks with a single process. Is this expected? Thanks, Pierre $ ./ex1 -tao_monitor ---- Constrained Problem ----- Solution should be f(1,1)=-2 0 TAO, Function value: 8., Residual: 8.48528 1 TAO, Function value: 1.28532, Residual: 10.7411 2 TAO, Function value: -2.56703, Residual: 1.87847 3 TAO, Function value: -2.03161, Residual: 0.12881 4 TAO, Function value: -1.99961, Residual: 0.0450855 5 TAO, Function value: -1.99993, Residual: 0.000916939 6 TAO, Function value: -1.99999, Residual: 6.69184e-05 7 TAO, Function value: -2., Residual: 7.15427e-06 8 TAO, Function value: -2., Residual: 7.15779e-07 9 TAO, Function value: -2., Residual: 7.15777e-08 10 TAO, Function value: -2., Residual: 7.15777e-09 11 TAO, Function value: -2., Residual: 7.15779e-10 12 TAO, Function value: -2., Residual: 7.15775e-11 13 TAO, Function value: -2., Residual: 7.1599e-12 14 TAO, Function value: -2., Residual: 7.1599e-13 15 TAO, Function value: -2., Residual: 7.22085e-14 16 TAO, Function value: -2., Residual: 6.44626e-15 17 TAO, Function value: -2., Residual: 1.00751e-15 18 TAO, Function value: -2., Residual: 1.70295e-17 19 TAO, Function value: -2., Residual: 1.70295e-18 20 TAO, Function value: -2., Residual: 1.70295e-19 21 TAO, Function value: -2., Residual: 1.70295e-20 22 TAO, Function value: -2., Residual: 1.70295e-21 23 TAO, Function value: -2., Residual: 1.70295e-22 24 TAO, Function value: -2., Residual: 1.70295e-23 25 TAO, Function value: -2., Residual: 1.70295e-24 26 TAO, Function value: -2., Residual: 1.70295e-25 27 TAO, Function value: -2., Residual: 1.70295e-26 28 TAO, Function value: -2., Residual: 1.70295e-27 29 TAO, Function value: -2., Residual: 1.70295e-28 30 TAO, Function value: -2., Residual: 1.70295e-29 31 TAO, Function value: -2., Residual: 1.70295e-30 32 TAO, Function value: -2., Residual: 1.70295e-31 33 TAO, Function value: -2., Residual: 1.70295e-32 34 TAO, Function value: -2., Residual: 1.70295e-33 35 TAO, Function value: -2., Residual: 1.70295e-34 36 TAO, Function value: -2., Residual: 1.70295e-35 37 TAO, Function value: -2., Residual: 1.70295e-36 38 TAO, Function value: -2., Residual: 1.70295e-37 39 TAO, Function value: -2., Residual: 1.70295e-38 40 TAO, Function value: -2., Residual: 1.70295e-39 41 TAO, Function value: -2., Residual: 1.70295e-40 42 TAO, Function value: -2., Residual: 1.70295e-41 43 TAO, Function value: -2., Residual: 1.70295e-42 44 TAO, Function value: -2., Residual: 1.70295e-43 45 TAO, Function value: -2., Residual: 1.70295e-44 46 TAO, Function value: -2., Residual: 1.70295e-45 47 TAO, Function value: -2., Residual: 1.70295e-46 48 TAO, Function value: -2., Residual: 1.70295e-47 49 TAO, Function value: -2., Residual: 1.70295e-48 50 TAO, Function value: -2., Residual: 1.70295e-49 51 TAO, Function value: -2., Residual: 1.70295e-50 52 TAO, Function value: -2., Residual: 1.70295e-51 ^C

1 0

Re: [petsc-dev] How far should internal error checking go?
by Jed Brown 04 Aug '20

04 Aug '20

There are lots of places where we could use Valgrind client checks to invalidate memory or assert that it's valid, in order to catch bugs sooner and more reliably. On Tue, Aug 4, 2020, at 1:05 PM, Barry Smith wrote: > > So long as the error checking is not much more expensive then the computation than I see no harm in including it with the if (debug) model. For example, checking a sorted array is sorted is less expensive than the sort so can be included at the end of sort algorithm. > > For the numerical solvers things are much trickery because it is difficult to check if the result is "correct", especially given the different criteria possible for "convergence" and norms that might be used. Thus checking the "little" things throughout the code is a good idea, because we can't catch the "big things". > > One check we don't do consistently which requires elbow grease is to use VecGetArrayWrite() instead of VecGetArray() when the routine has to fill ALL the values in the array. Then VecGetArrayWrite() can fill the array with Nan, and VecRestoreArrayWrite() verifies that there are no Nan in the result to do the basic test that the routine did not miss setting some values. > > We could extend this concept to other places where a routine is suppose to "fill up" memory obtained with PetscMalloc(), have something like PetscMallocVerifyFilled() that checks malloced space after it has been filled by the code to make sure no places where missing. Note that valgrind does some of this checking, but not all, valgrind only generates messages when a resulting unset value is used in an if (something) or something that controls program flow etc. It will not detect when a numerical location is not filled in but is used later in a numerical computation that never controls program flow. Hence in debug mode I would like PetscMalloc() to fill all numerical arrays with Nan. > > This is the simplest error checking, not even checking that correct values are used, just checking that something is used and we don't even do this everywhere yet. > > Barry > > > > > > >> On Aug 4, 2020, at 12:24 PM, Jacob Faibussowitsch <jacob.fai(a)gmail.com> wrote: >> >> Hello All, >> >> How far should one go in error checking when using #if defined(PETSC_USE_DEBUG)? So far I have gone with the mantra that internal petsc routines (including ones authored by myself) should be considered infallible and that I should only be checking for garbage *input* from the user. But there is not a person in existence that doesn’t write buggy code, as evidenced by the somewhat routine “fixing bug in XYZ” merge requests one sees. >> >> On the other hand it is also not reasonable to check every single output with a fine-tooth comb because for the majority of cases code written by petsc developers is working as intended. Take for example writing an array sorting algorithm. Since every operation is "performance critical” these are often written in less logical or less readable formats leading to some subtle bugs that the writer doesn’t immediately catch. If these remain uncaught through CI/CD and then bleeds into a user code I see absolutely no chance of the user (or even other devs) ever being able to identify that the sorting algorithm deep in some function stack is the one producing the bug without significant effort. One could include a “dumb” version of the same algorithm that checks a copy of the initial array for missing/misplaced elements but as mentioned above this is a pointless slowdown 99% of the time. >> >> CI/CD -- while excellent at catching a lot of machine-specific bugs -- isn’t bulletproof either. It relies on the assumption that the writer knows all possible sources of bugs in their code and provides a test case for each, but to quote Isaac Newton “what we know is a drop, what we don’t know is an ocean”. I have been mulling over this problem for a while now, and have looked through the user manual/developers manual but have not found a definitive answer. >> >> Best regards, >> >> Jacob Faibussowitsch >> (Jacob Fai - booss - oh - vitch) >> Cell: (312) 694-3391

1 0

Re: [petsc-dev] How far should internal error checking go?
by Barry Smith 04 Aug '20

04 Aug '20

So long as the error checking is not much more expensive then the computation than I see no harm in including it with the if (debug) model. For example, checking a sorted array is sorted is less expensive than the sort so can be included at the end of sort algorithm. For the numerical solvers things are much trickery because it is difficult to check if the result is "correct", especially given the different criteria possible for "convergence" and norms that might be used. Thus checking the "little" things throughout the code is a good idea, because we can't catch the "big things". One check we don't do consistently which requires elbow grease is to use VecGetArrayWrite() instead of VecGetArray() when the routine has to fill ALL the values in the array. Then VecGetArrayWrite() can fill the array with Nan, and VecRestoreArrayWrite() verifies that there are no Nan in the result to do the basic test that the routine did not miss setting some values. We could extend this concept to other places where a routine is suppose to "fill up" memory obtained with PetscMalloc(), have something like PetscMallocVerifyFilled() that checks malloced space after it has been filled by the code to make sure no places where missing. Note that valgrind does some of this checking, but not all, valgrind only generates messages when a resulting unset value is used in an if (something) or something that controls program flow etc. It will not detect when a numerical location is not filled in but is used later in a numerical computation that never controls program flow. Hence in debug mode I would like PetscMalloc() to fill all numerical arrays with Nan. This is the simplest error checking, not even checking that correct values are used, just checking that something is used and we don't even do this everywhere yet. Barry > On Aug 4, 2020, at 12:24 PM, Jacob Faibussowitsch <jacob.fai(a)gmail.com> wrote: > > Hello All, > > How far should one go in error checking when using #if defined(PETSC_USE_DEBUG)? So far I have gone with the mantra that internal petsc routines (including ones authored by myself) should be considered infallible and that I should only be checking for garbage *input* from the user. But there is not a person in existence that doesn’t write buggy code, as evidenced by the somewhat routine “fixing bug in XYZ” merge requests one sees. > > On the other hand it is also not reasonable to check every single output with a fine-tooth comb because for the majority of cases code written by petsc developers is working as intended. Take for example writing an array sorting algorithm. Since every operation is "performance critical” these are often written in less logical or less readable formats leading to some subtle bugs that the writer doesn’t immediately catch. If these remain uncaught through CI/CD and then bleeds into a user code I see absolutely no chance of the user (or even other devs) ever being able to identify that the sorting algorithm deep in some function stack is the one producing the bug without significant effort. One could include a “dumb” version of the same algorithm that checks a copy of the initial array for missing/misplaced elements but as mentioned above this is a pointless slowdown 99% of the time. > > CI/CD -- while excellent at catching a lot of machine-specific bugs -- isn’t bulletproof either. It relies on the assumption that the writer knows all possible sources of bugs in their code and provides a test case for each, but to quote Isaac Newton “what we know is a drop, what we don’t know is an ocean”. I have been mulling over this problem for a while now, and have looked through the user manual/developers manual but have not found a definitive answer. > > Best regards, > > Jacob Faibussowitsch > (Jacob Fai - booss - oh - vitch) > Cell: (312) 694-3391 >

1 0