Optim.jl : user defined Manifold, error in the definition of the retraction

Specific Domains Optimization (Mathematical)
1

Hi, I try to optimize a simple quadratic function on a user-defined manifold (the hyperplan orthogonal to ones(n)) with the following code

using Optim
const c = ones(2) ./ sqrt(2)
f(x) = 0.5 * sum((x - c).^2)
g!(g,x)  = g[:] = x - c

struct HypOne <: Manifold end
retract!(S::HypOne, x) = (x ./= sum(x))
function project_tangent!(S::HypOne, g, x)
    α = 1. / size(x,1) * sum(g)
    g .-= α
end
x0 = [1., 0.]
Optim.optimize(f, g!, x0, Optim.ConjugateGradient(manifold=HypOne()))

apparently, my way to define the manifold and the retraction seems to be not good, as the following error message says

ERROR: LoadError: MethodError: no method matching retract!(::HypOne, ::Vector{Float64})
The function `retract!` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  retract!(::Optim.ProductManifold, ::Any)
   @ Optim ~/.julia/packages/Optim/7krni/src/Manifolds.jl:144
  retract!(::Optim.PowerManifold, ::Any)
   @ Optim ~/.julia/packages/Optim/7krni/src/Manifolds.jl:113
  retract!(::Optim.Stiefel_CholQR, ::Any)
   @ Optim ~/.julia/packages/Optim/7krni/src/Manifolds.jl:91
  ...
2

The error reports that it does not “find” your defined methods, not that they are not good. The reason is, that you define them in your main scope, which in short is a different namespace than the one from Optim. One could in short say that is for safety reasons.

To actually add methods to Optim.retract! you have to import that methods with import Optim: retract!, project_tangent!. Not that I did not check your formulas in general, but the following code at least runs without errors.

using Optim
import Optim: retract!, project_tangent! # This is the only new line
const c = ones(2) ./ sqrt(2)
f(x) = 0.5 * sum((x - c).^2)
g!(g,x)  = g[:] = x - c

struct HypOne <: Manifold end
retract!(S::HypOne, x) = (x ./= sum(x))
function project_tangent!(S::HypOne, g, x)
    α = 1. / size(x,1) * sum(g)
    g .-= α
end
x0 = [1., 0.]
Optim.optimize(f, g!, x0, Optim.ConjugateGradient(manifold=HypOne()))
2 Likes
3

Thanks, it solved my problem.

1 Like