In my recent question (Optim finite differences use a vectorized version) the following “sphere trick” was suggested to me. If I use it here, I get results that are fine:
using Random, LinearAlgebra, BenchmarkTools
using Optim, ForwardDiff
Random.seed!(42)
norm_c!(c, x) = (c[1] = sum(y -> y^2, x); c)
constraints(d) = TwiceDifferentiableConstraints(norm_c!, fill(-1., d), fill(1.,d), [-Inf], [1.], :forward)
function sphere_trick(a::Vector{T}) where {T<:Real}
a_coords = view(a, 1:length(a)-1)
if all(x -> x == 0, a_coords)
return a_coords
else
return a_coords * a[end] / (sqrt(sum(y -> y^2, a_coords)))
end
end
constraints_alt(d) = TwiceDifferentiableConstraints(vcat(fill(-1., d), [0.]), vcat(fill(1.,d), [1.]))
function check_optim(d)
f_sum = x -> ones(d) ⋅ x
cons = constraints(d)
init = [0. for _ in 1:d]
foc_sum = Optim.optimize(f_sum, cons, init, IPNewton(); autodiff = :forward)
display(foc_sum)
display(foc_sum.minimizer)
display(norm_c!([NaN], foc_sum.minimizer))
f_sum_alt = x -> ones(d) ⋅ sphere_trick(x)
cons_alt = constraints_alt(d)
init_alt = vcat(init, 0.5)
foc_alt = Optim.optimize(f_sum_alt, cons_alt, init_alt, IPNewton(); autodiff = :forward)
display(foc_alt)
display(foc_alt.minimizer)
min_true = foc_alt.minimizer[1:end-1] ./ sqrt(sum(y -> y^2, foc_alt.minimizer[1:end-1]))*foc_alt.minimizer[end]
display(norm_c!([NaN], foc_alt.minimizer[1:end-1]))
display(norm_c!([NaN], min_true))
end
gives
* Convergence measures
|x - x'| = 8.45e-14 ≰ 0.0e+00
|x - x'|/|x'| = 8.45e-14 ≰ 0.0e+00
|f(x) - f(x')| = 0.00e+00 ≤ 0.0e+00
|f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 0.0e+00
|g(x)| = 1.41e+00 ≰ 1.0e-08
* Work counters
Seconds run: 0 (vs limit Inf)
Iterations: 39
f(x) calls: 62
∇f(x) calls: 62
3-element Vector{Float64}:
-0.01523778048002576
-0.015237780479863912
1.0
1-element Vector{Float64}:
0.0004643799079099757
1-element Vector{Float64}:
1.0000000000000002
In my current problem I can probably get away with simply doing this, but it does feel a bit suboptimal if I’d have to keep doing this (as I now need to differentiate different optim
calls based on the nature of the constraints, where first these were simply reflected in the TwiceDifferentiableConstraints
instances…