Optim.jl: bracketing with Newton()

Is it possible to use Newton() methods with limiting some variables? Like FMinbox?

please start with the manual, eg


Thank you, I read the manual. I think I know how Fminbox() works, but it doesn’t work with Newton(). (L)BFGS() works fine with Fminbox(), but I hope to have some solution for Newton().

We have a Newton method for bound-constrained problems here: https://juliasmoothoptimizers.github.io/JSOSolvers.jl/latest/solvers/#JSOSolvers.tron

See this issue you may be running into, and particularly this comment:

Thanks, i will try!

Thanks, this issue is wery close to my project. I also need to maximize likelihood function and one of parameter of covariance matrix - rho - changing in limits 0 - 1.0, and when i use LBFGS - it’s works fine, but to slow. Newton() works fast, but i need to do things like this: if rho > 1 rho = 1 end and this is not good, because last step of optimization some time > 1.0 end it can leads to “stuck” in value 1.0 when real value close to 1.0 ( f.e. 0.988). I try to use “link function” like sigmoid, but at marginal values it works not good: 1/(1+exp(x)) not works correctly, 1/(1+x^4) works nearly good, but not not better when rho “force limited”. Now i use LBFGS till rho is stable and then use Newton(), or Newton() at first step and then LBFGS, but i think there must be a better way.

It is hard to say more about this without an MWE (eg do you provide derivatives manually or use AD, what is rho [stepsize correction?], …).

Hi! I use AD, and likelihood function based on derivation of variance-covariance matrix. For example 2x2 matrix (ρ, σ₁, σ₂ - should be found)

σ²₁             ρσ₁σ₂ 
ρσ₁σ₂           σ²₂

where ρ covariation coefficient that should be also found - and it can be in the range 0 - 1.

You might have better luck transforming your variables, as done here: https://julianlsolvers.github.io/Optim.jl/stable/#examples/generated/maxlikenlm/ . They work with the log variance which can take on any value. For ρ you could use tanh and atanh to go back and forth between (-1, 1) and (-inf, inf)

1 Like