On Thu, Sep 23, 2021 at 5:07 AM Pierre Jolivet <pierre@joliv.et> wrote:
Hello,
When a tests fails because the timeout limit is reached, check-test-errors still returns a 0 exit code.
Is this intended? Any way to bypass this and return something different than 0, as it could return false negative in CI tests?

This is the intended behavior. We ignore timeouts, but report them.

I would not have a problem with a flag to convert these to errors, but I would leave this default.

  Thanks,

     Matt
 
Thanks,
Pierre

$ make -f gmakefile test s='ksp_ksp_tutorials-ex2_1' EXTRA_OPTIONS="-m 200 -n 200" check-test-errors 2>&1 > /dev/null; echo $?
make: *** [check-test-errors] Error 1
2
$ make -f gmakefile test s='ksp_ksp_tutorials-ex2_1' EXTRA_OPTIONS="-m 200 -n 200" check-test-errors TIMEOUT=1 2>&1 > /dev/null; echo $?
0


--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener