Highly Asymmetric Hessian after optimization

[ChrisRackauckas]

@Tamas_Papp

I think that a Symmetric wrapper would be preferable (because of what I said above: you can enforce symmetry in your output, but it can get lost easily again).

Yeah, good point. Output a Symmetric from the Hessians by ChrisRackauckas · Pull Request #68 · JuliaDiff/FiniteDiff.jl · GitHub

[Ross_Boylan]

This is a side-issue, but I don’t see copytri! in the LinearAlgebra documentation at Linear Algebra · The Julia Language, or anywhere in the docs according to the search web interface. Does that mean anything?

[ChrisRackauckas]

It means it copies the triangular part to the other triangle.

[Ross_Boylan]

I meant does the fact that copytri! isn’t documented mean anything?

[Tamas_Papp]

It is not part of the official API and could change without warning.

But given that new releases are tested against registered packages, this is not a major concern.

[Ross_Boylan]

@pkofod

Since the hessian evaluation often shares considerable calculations with the gradient and value, it’s almost always better to evaluate them together (and with automatic differentiation you can actually get the gradient and value for free given you’ve decided to evaluate the hessian, so…). You only save some calculations in the last iteration, but all other iterations become more expensive - seems like stupid trade-off.

I’m not sure if those remarks were intended to cover Newton only, or all the optimization methods. What about methods that don’t use the 2nd derivative? The trade-off seems different there, since the optimization never uses the Hessian but the consumer of the result may want it aftewords.

[pkofod]

For BFGS we don’t assume that the hessian is available, so in that case it’s up to the user to do whatever they want.