hm - indices are integers  and its not ignored (i.e %d) - only the %f and %e diff is ignored (by default)

Satish

On Mon, 4 Jan 2021, Scott Kruger wrote:

Is this just the 3rd problem?

Regarding how you can end up with changes not being caught:
The default (going all the way back to the old harness) is to not check
numbers to avoid round-off errors giving false negatives (failures).
Of course, sometimes you *want* to check the numbers; e.g., for indices.� The
solution for this is to add:

���� diff_args: -j

Scott

On 12/31/20 4:40 PM, Barry Smith wrote:

� I think I have it "fixed" now in the branch, once it passes the pipeline I
will shepard it through the MR quickly. Sorry about this, even all our CI
testing can miss a great deal.

� Barry

On Dec 31, 2020, at 2:44 PM, Barry Smith <bsmith@petsc.dev
<mailto:bsmith@petsc.dev>> wrote:


� This is a different (3rd) problem. Funny it didn't bother anyone for two
months.

� Fix is in barry/2020-12-29/fix-petscdiff-bracket but the pipeline keeps
failing ts_tutorials_advection-diffusion-reaction-ex3_2 fails on different
machines with slightly different counts. I don't see how this change could
cause that! But gets old results on my machine. Very frustrating.
Barry

On Dec 31, 2020, at 1:02 PM, Matthew Knepley <knepley@gmail.com
<mailto:knepley@gmail.com>> wrote:

On Thu, Dec 31, 2020 at 1:48 PM Barry Smith <bsmith@petsc.dev
<mailto:bsmith@petsc.dev>> wrote:


    � So the programs output changes and should no longer match that
    in the output/* file yet the test harness does not error with a
    statement that the two outputs do not match?

    � �I noticed the gmakegentest.py is not being run before it runs
    the test? Does this mean it is just running all the old stuff
    which does match fine?

    � �Then either how petscdiff is called by the test harness has
    changed or petscdiff has changed and does not detect changes
    anymore

    � �BTW: I always use -f ./gmakefile.test test not just the gmakefile

    � �All the PETSc changes are trivial and can be seen with a
    simple diff, it is hard to believe they would cause this
    behavior but I guess they must.

    � �You can go to PETSC_ARCH/tests/snes/tests and run the ex13
    shell script directly.


It is the sed problem:

master *$:/PETSc3/petsc/petsc-dev$
/PETSc3/petsc/petsc-dev/lib/petsc/bin/petscdiff
/PETSc3/petsc/petsc-dev/src/snes/tests/output/ex13_bench.out
ex13_bench.tmp

sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])
sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])

The error was getting eaten.

This is in current master. Is it fixed in a branch?

� �Matt

    � Barry

    On Dec 31, 2020, at 12:38 PM, Matthew Knepley
    <knepley@gmail.com <mailto:knepley@gmail.com>> wrote:

    I just�pulled master, and simple alterations to tests do not
    produce a failure:

    master *$:/PETSc3/petsc/petsc-dev$ PETSC_ARCH=arch-master-debug
    make -f ./gmakefile test search="snes_tests-ex13_bench"
    TIMEOUT=5000 EXTRA_OPTIONS="-dm_
    refine 0"
    Using MAKEFLAGS: EXTRA_OPTIONS=-dm_refine 0 TIMEOUT=5000
    search=snes_tests-ex13_bench
    � � � � TEST
    arch-master-debug/tests/counts/snes_tests-ex13_bench.counts
    �ok snes_tests-ex13_bench
    �ok diff-snes_tests-ex13_bench

    I check that the runs produce different output when done manually.

    Scott and Barry, could this be related to changed to testing?

    � Thanks,

    � � �Matt

    -- 
    What most experimenters take for granted before they begin
    their experiments is infinitely more interesting than any
    results to which their experiments lead.
    -- Norbert Wiener

    https://www.cse.buffalo.edu/~knepley/
    <http://www.cse.buffalo.edu/%7Eknepley/>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/%7Eknepley/>

-- 
Tech-X Corporation               kruger@txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 466-3196
Boulder, CO 80303                Fax:   (303) 448-7756