The function tolerances are usually something like |f(x_k) - f_(x_{k-1})| <= fatol

But that is *not* what they are in TAO, this has always been a source of confusion. The one upside of the f-tolerances is that they give the users a reasonable level of confidence that the objective function value achieved is within ftol of the true minimum, but it just happens to be redundant because f(x)-f(X*) is estimated using ||g(x)||^2 (How? I thought this made sense to me once, but now I don't see it anymore) and we also have a convergence criteria for ||g(x)||. On Wed, Sep 9, 2015 at 10:02 AM, Munson, Todd S. <[email protected]> wrote:

I do not see a need to keep them around. The function tolerances are usually something like |f(x_k) - f_(x_{k-1})| <= fatol and then you use |f(x_0)| in the denominator for the relative tolerance. There are pretty useless tests though and give many false positives.

Todd.

...

On Sep 9, 2015, at 9:26 AM, Jason Sarich <[email protected]> wrote:

Todd, can you come up with a reason why we might need to keep these fatol, frtol tests around? I would prefer to take them out.

Jason

On Tue, Sep 8, 2015 at 9:54 PM, Smith, Barry F. <[email protected]> wrote:

Jason,

Perhaps it is just documentation.

/*@ TaoSetTolerances - Sets parameters used in TAO convergence tests

Logically collective on Tao

Input Parameters: + tao - the Tao context . fatol - absolute convergence tolerance . frtol - relative convergence tolerance . gatol - stop if norm of gradient is less than this . grtol - stop if relative norm of gradient is less than this - gttol - stop if norm of gradient is reduced by this factor

Options Database Keys: + -tao_fatol <fatol> - Sets fatol . -tao_frtol <frtol> - Sets frtol . -tao_gatol <gatol> - Sets gatol . -tao_grtol <grtol> - Sets grtol - -tao_gttol <gttol> - Sets gttol

Stopping Criteria: $ f(X) - f(X*) (estimated) <= fatol $ |f(X) - f(X*)| (estimated) / |f(X)| <= frtol $ ||g(X)|| <= gatol $ ||g(X)|| / |f(X)| <= grtol $ ||g(X)|| / ||g(X0)|| <= gttol

How is f(x) - f(X*) (estimated) and how come the second definition has absolute values but the first does not?

From below it looks like f(x) - f(X*) is estimated as gnorm*gnorm?

} else if (gnorm2 <= fatol && cnorm <=catol) { ierr = PetscInfo2(tao,"Converged due to estimated f(X) - f(X*) = %g < %g\n",(double)gnorm2,(double)fatol);CHKERRQ(ierr); reason = TAO_CONVERGED_FATOL; } else if (f != 0 && gnorm2 / PetscAbsReal(f)<= frtol && cnorm/PetscMax(cnorm0,1.0) <= crtol) { ierr = PetscInfo2(tao,"Converged due to estimated |f(X)-f(X*)|/f(X) = %g < %g\n",(double)(gnorm2/PetscAbsReal(f)),(double)frtol);CHKERRQ(ierr); reason = TAO_CONVERGED_FRTOL;

It seems the next test is the same as the first I list above exact gnorm is not squared? Why is it worth having this duplicate test?

} else if (gnorm<= gatol && cnorm <=catol) { ierr = PetscInfo2(tao,"Converged due to residual norm ||g(X)||=%g < %g\n",(double)gnorm,(double)gatol);CHKERRQ(ierr); reason = TAO_CONVERGED_GATOL;

At a minimum the manual page should document what f(X) - f(X*) (estimated) means.

Barry

} else if ( f!=0 && PetscAbsReal(gnorm/f) <= grtol && cnorm <= crtol) { ierr = PetscInfo2(tao,"Converged due to residual ||g(X)||/|f(X)| =%g < %g\n",(double)(gnorm/f),(double)grtol);CHKERRQ(ierr);