Re: [petsc-dev] Testing seems broken in master

4 Jan 2021


      Yes.ï¿½ There are test cases where a cast is done so for those.

Scott


On 1/4/21 10:29 AM, Satish Balay wrote:
...
hm - indices are integers  and its not ignored (i.e %d) - only the %f and %e diff is ignored (by default)
Satish
On Mon, 4 Jan 2021, Scott Kruger wrote:
...
Is this just the 3rd problem?
Regarding how you can end up with changes not being caught:
The default (going all the way back to the old harness) is to not check
numbers to avoid round-off errors giving false negatives (failures).
Of course, sometimes you *want* to check the numbers; e.g., for indices.ï¿½ The
solution for this is to add:
ï¿½ï¿½ï¿½ï¿½ diff_args: -j
Scott
On 12/31/20 4:40 PM, Barry Smith wrote:
...
ï¿½ I think I have it "fixed" now in the branch, once it passes the pipeline I
will shepard it through the MR quickly. Sorry about this, even all our CI
testing can miss a great deal.
ï¿½ Barry
...
On Dec 31, 2020, at 2:44 PM, Barry Smith <[email protected]
<mailto:[email protected]>> wrote:
ï¿½ This is a different (3rd) problem. Funny it didn't bother anyone for two
months.
ï¿½ Fix is in barry/2020-12-29/fix-petscdiff-bracket but the pipeline keeps
failing ts_tutorials_advection-diffusion-reaction-ex3_2 fails on different
machines with slightly different counts. I don't see how this change could
cause that! But gets old results on my machine. Very frustrating.
Barry
...
On Dec 31, 2020, at 1:02 PM, Matthew Knepley <[email protected]
<mailto:[email protected]>> wrote:
On Thu, Dec 31, 2020 at 1:48 PM Barry Smith <[email protected]
<mailto:[email protected]>> wrote:
ï¿½ So the programs output changes and should no longer match that
     in the output/* file yet the test harness does not error with a
     statement that the two outputs do not match?
ï¿½ ï¿½I noticed the gmakegentest.py is not being run before it runs
     the test? Does this mean it is just running all the old stuff
     which does match fine?
ï¿½ ï¿½Then either how petscdiff is called by the test harness has
     changed or petscdiff has changed and does not detect changes
     anymore
ï¿½ ï¿½BTW: I always use -f ./gmakefile.test test not just the gmakefile
ï¿½ ï¿½All the PETSc changes are trivial and can be seen with a
     simple diff, it is hard to believe they would cause this
     behavior but I guess they must.
ï¿½ ï¿½You can go to PETSC_ARCH/tests/snes/tests and run the ex13
     shell script directly.
It is the sed problem:
master *$:/PETSc3/petsc/petsc-dev$
/PETSc3/petsc/petsc-dev/lib/petsc/bin/petscdiff
/PETSc3/petsc/petsc-dev/src/snes/tests/output/ex13_bench.out
ex13_bench.tmp
sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])
sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])
The error was getting eaten.
This is in current master. Is it fixed in a branch?
ï¿½ ï¿½Matt
ï¿½ Barry
...
On Dec 31, 2020, at 12:38 PM, Matthew Knepley
     <[email protected] <mailto:[email protected]>> wrote:
I justï¿½pulled master, and simple alterations to tests do not
     produce a failure:
master *$:/PETSc3/petsc/petsc-dev$ PETSC_ARCH=arch-master-debug
     make -f ./gmakefile test search="snes_tests-ex13_bench"
     TIMEOUT=5000 EXTRA_OPTIONS="-dm_
     refine 0"
     Using MAKEFLAGS: EXTRA_OPTIONS=-dm_refine 0 TIMEOUT=5000
     search=snes_tests-ex13_bench
     ï¿½ ï¿½ ï¿½ ï¿½ TEST
     arch-master-debug/tests/counts/snes_tests-ex13_bench.counts
     ï¿½ok snes_tests-ex13_bench
     ï¿½ok diff-snes_tests-ex13_bench
I check that the runs produce different output when done manually.
Scott and Barry, could this be related to changed to testing?
ï¿½ Thanks,
ï¿½ ï¿½ ï¿½Matt
--
     What most experimenters take for granted before they begin
     their experiments is infinitely more interesting than any
     results to which their experiments lead.
     -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
     <http://www.cse.buffalo.edu/%7Eknepley/>
-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/%7Eknepley/>
-- 
Tech-X Corporation               [email protected]
5621 Arapahoe Ave, Suite A       Phone: (720) 466-3196
Boulder, CO 80303                Fax:   (303) 448-7756

Scott Kruger

tags

participants (1)