Reduce memory allocation for repeatedly solving problem using Optim.jl

Specific Domains Optimization (Mathematical)
, ,
1

I’m trying to re-solve a problem with different parameter values many times using Optim.jl. My current code allocates a lot of memory because a new TwiceDifferentiable and TwiceDifferentiableConstraints is created for every new set of parameter values. Is there a way to cache the problem structure to prevent the memory allocation? Perhaps someone with knowledge of the internal workings of Optim.jl knows how to do this. Here’s a simple MWE:

using Random
using Optim, NLSolversBase
using BenchmarkTools

function fun(x, w)
    mapreduce((xi, wi) -> xi * wi, +, x, w)
end

function fun_grad!(g, x, w)
    g .= w
end

function fun_hess!(h, x, w)
    h .= 0.0
end

const n = 10
const x0 = zeros(n)
const lx = ones(n) * -1.0
const ux = ones(n)

function solve_subproblem(w)
    f = (x) -> fun(x, w)
    g! = (g, x) -> fun_grad!(g, x, w)
    h! = (h, x) -> fun_hess!(h, x, w)

    df = TwiceDifferentiable(f, g!, h!, x0)
    dfc = TwiceDifferentiableConstraints(lx, ux)
    
    return optimize(df, dfc, x0, IPNewton())
end

function solve(m)
    for i = 1:m
        w = rand(n)
        solve_subproblem(w)
    end
end

@benchmark solve(10000)


BenchmarkTools.Trial: 
  memory estimate:  9.01 GiB
  allocs estimate:  73275049
  --------------
  minimum time:     6.539 s (5.82% GC)
  median time:      6.539 s (5.82% GC)
  mean time:        6.539 s (5.82% GC)
  maximum time:     6.539 s (5.82% GC)
  --------------
  samples:          1
  evals/sample:     1
2 Likes