Hi, I did not parse your particular needs correctly with regard to complex step derivatives. I do not see any huge barriers, but it needs to be made clear the optimization methods only use the real parts of both the inputs and outputs (and if the user functions use the imaginary parts when computing their objective, that is, they really have a problems with complex variables, then they will get incorrect behavior). I am sure they will be other issues that crop up, such as only performing the KSP solves using the real part of the matrices. Maybe someone has a better idea about using complex step for derivatives of what is fundamentally a real valued problem. Todd.
On Dec 12, 2018, at 11:04 AM, Stefano Zampini <[email protected]> wrote:
My problems are all real.
I was just building a complex version of petsc and noticed these issue. As far as it is made clear (as it is) that Tao only supports real valued objective functions, there's no need to disable the solvers suite when petsc is compiled with complexes, since all the vectors and matrices used in TAO solvers will be zero in their imaginary part. But probably there are other issues that I'm not considering?
Il giorno Mer 12 Dic 2018, 18:46 Munson, Todd <[email protected]> ha scritto:
Yes; the optimization problems do not make sense if the objective function is from C^n to C, as there is not a natural ordering for complex numbers. If your objective is from C^n to R (and the constraints are from C^n to R^m), then the problem can be well defined. Then we need to get into what bound constraints mean, etc.
One challenge from the optimization perspective is dealing with the complex vectors and creating complex Hessian approximations. I think its possible, but has never been a priority. If the bounds are on the magnitude of the complex values, then we need appropriate nonlinear constraints to enforce them.
Todd.
On Dec 12, 2018, at 9:18 AM, Stefano Zampini via petsc-dev <[email protected]> wrote:
Il giorno mer 12 dic 2018 alle ore 18:06 Dener, Alp <[email protected]> ha scritto: Hi Stefano,
I can’t really speak to why the entirety of TAO is disabled in complex builds. Todd can probably offer historical insight to the reason(s).
However, we do have some methods in TAO that are not tested or guaranteed to be correct for complex-valued objective functions.
As far as I understands, TAO is clear in requiring real objective functions, having PetscReal on the prototype for the callback
The limited-memory quasi-Newton codes, for instance, currently only construct the approximate Hessian for the real part of the function and do not generate step directions in the complex part. I also don’t know whether any of the CG methods or line searches require any modifications to be correct in complex builds either.
In your itemized list, I agree that at minimum we should be supporting #3 even if TAO itself does not work correctly for complex-valued functions.
In the meantime, TAO can test real gradients or Hessians internally with `-tao_test_gradient` or `-tao_test_hessian`. Or you can also call `TaoTestHessian()` too. It’s a finite-difference test though, and not a complex-step test.
—— Alp Dener Argonne National Laboratory https://www.anl.gov/profile/alp-dener
On Dec 12, 2018, at 8:23 AM, Stefano Zampini via petsc-dev <[email protected]> wrote:
If I compile PETSc with complex support, all of the TAO methods are not registered. Fare enough? not really, I can imagine at least three valid reasons to use those
1) I want to check that my code works for complex builds, even if I have pure real function evaluations. I may need complex support in other parts or the code (not when using TAO), or I just want to be a diligent PETSc developer 2) I want to test REAL gradients or Hessians using built-in capability, instead of writing my own 3) I want to use the complex differentiation trick to compute REAL gradients
Last time I checked, at least option 2) was available; However, from commit 5921d7004dad83427f45468906667b100d2e2b6e, no Tao methods are registered when PETSC_USE_COMPLEX is defined, and code like the one below fail at runtime:
TaoCreate() TaoSetFromOption() TaoSetObjectiveRoutine() TaoTestGradient()
[0]PETSC ERROR: Unknown type. Check for miss-spelling or missing package: http://www.mcs.anl.gov/petsc/documentation/installation.html#external [0]PETSC ERROR: Unable to find requested Tao type lmvm
-- Stefano
-- Stefano