Avoid allocating an anonymous function

[jacob-roth]

I have an application where the package I am using requires the user to provide a function phi with calling statement like phi(a::Real)=.... I would like to pass additional function arguments args to phi. The way I have taken to get around this is to create an anonymous function a->myphi(a,args) that behaves as phi (in particular, the line involving the call to “linesearch” in the code below). The function phi gets called in an inner-loop in my program, so args is frequently changing, and I have to allocate for a new function a->myphi(a,args) each time. Is there a better alternative to recreating the anonymous function each time?

This question is a slight modification of an issue I raised, since perhaps there is a more general answer to this.

Here is the example. It modifies some code provided by the LineSearches.jl package.

using LinearAlgebra: norm, dot

mutable struct myargs
  v :: Float64
  i :: Int64
end

function gdoptimize(f, g!, fg!, x0::AbstractArray{T}, linesearch,
                    maxiter::Int = 10000,
                    g_rtol::T = sqrt(eps(T)), g_atol::T = eps(T)) where T <: Number
    x = copy(x0)
    gvec = similar(x)
    args = myargs(0.0, 0)
    g!(gvec, x, args)
    fx = f(x, args)

    gnorm = norm(gvec)
    gtol = max(g_rtol*gnorm, g_atol)

    # Univariate line search functions
    ϕ(α, args) = f(x .+ α.*s, args)
    function dϕ(α, args)
        g!(gvec, x .+ α.*s, args)
        return dot(gvec, s)
    end
    function ϕdϕ(α, args)
        phi = fg!(gvec, x .+ α.*s, args)
        dphi = dot(gvec, s)
        return (phi, dphi)
    end

    s = similar(gvec) # Step direction

    iter = 0
    while iter < maxiter && gnorm > gtol
        iter += 1
        args.i = iter
        s .= -gvec

        dϕ_0 = dot(s, gvec)
        α, fx = linesearch(a -> ϕ(a, args), a -> dϕ(a, args), a -> ϕdϕ(a, args), 1.0, fx, dϕ_0)
        
        @. x = x + α*s
        g!(gvec, x, args)
        gnorm = norm(gvec)
    end

    return (fx, x, iter)
end

f(x, args) = begin
  println("f: args.i = $(args.i)")
  (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2 + args.v
end

function g!(gvec, x, args)
    println("g!: args.i = $(args.i)")
    gvec[1] = -2.0 * (1.0 - x[1]) - 400.0 * (x[2] - x[1]^2) * x[1]
    gvec[2] = 200.0 * (x[2] - x[1]^2)
    gvec
end

function fg!(gvec, x, args)
    println("fg!: args.i = $(args.i)")
    g!(gvec, x, args)
    f(x, args)
end

x0 = [-1., 1.0]

using LineSearches
ls = BackTracking(order=3)
fx_bt3, x_bt3, iter_bt3 = gdoptimize(f, g!, fg!, x0, ls)

ls = StrongWolfe()
fx_sw, x_sw, iter_sw = gdoptimize(f, g!, fg!, x0, ls)

[lmiq]

Are you sure allocations are happening at every loop iteration? I don’t think that should happen unless the types of the arguments change.

[cgeoga]

Jake Roth! Maybe I’m missing something here, but a callable struct might be useful:

mutable struct MyPhi <: Function # subtyping might be necessary
  stuff :: [...]
end

function (phi::MyPhi)(a)
  [...]
end

And then in your code you would create phi = MyPhi(other_args...), which you could then pass to your linesearch function.

[GHTaarn]

Would you be able to simplify your code and still illustrate your problem? In item 5 in Please read: make it easier to help you this is suggested as one of the ways to make it easier for people to answer your question.

[jacob-roth]

Ah, I wasn’t aware of this! Thanks Chris!

[jacob-roth]

I think that you’re right! The allocations in --track-allocation=all don’t seem to show any. I’d originally found allocations when using @allocated on the single linesearch line of code.