Metaheuristics.jl ECA() through Optimization.jl does not work with EnsembleProblems

Specific Domains Optimization (Mathematical)
1

I’m performing an optimization where the number of parameters used is varied across the ensemble. (e.g. prob 1 uses 1 parameter, prob 2 uses 2 parameters) Unfortunately, the algorithm that seems to work best with my specific problem appears to be incompatible with ensemble solves.

In my actual problem, the issue doesn’t happen every time, but in this example, it seems like every problem in the ensemble has the issue, which is usually good for figuring out what’s going wrong, but I haven’t been able to so far.

I’ve attached a short example below. Any advice on how to fix this, or if I’m doing something silly would be very helpful.

using LinearAlgebra, ForwardDiff, Optim
using Optimization, OptimizationMetaheuristics, OptimizationOptimJL

xdata = collect(LinRange(0, 10, 100));
ydata = rand(100);

function loss(u, p)
    x, y, n = p
    z = zeros(size(x,1))
    for i = 1:n
        z1 = u[i].*x
        z = z + z1
    end
    return sum((z .- y).^2)
end


u0 = rand(10)
optf = OptimizationFunction(loss, Optimization.AutoForwardDiff())
prob = OptimizationProblem(optf, u0, (xdata, ydata, 3), lb = zeros(10), ub = 10*ones(10))
sol = solve(prob, ECA(), maxtime = 3)
#single solve works fine
@show loss(sol.u, sol.prob.p) == sol.objective


function prob_func(prob, i, repeat)
    ps = prob.p
    newps = (ps[1], ps[2], i)
    prob = remake(prob, u0 = u0, p = newps)
    return prob
end

#verifies that stored solution objective matches actual loss when 
#recalculated using optimized parameters
function solution_checker(ensemblesol)
    for sol in ensemblesol
        listed_loss = sol.objective
        actual_loss = loss(sol.u, sol.prob.p)
        @show listed_loss == actual_loss
    end
end

ensembleprob = Optimization.EnsembleProblem(prob; prob_func)

#Metaheuristics ECA does not work properly
#using EnsembleSerial() to avoid any threading issues, but it's the same with threads
ensemblesol1 = solve(ensembleprob, ECA(), EnsembleSerial(), maxtime = 5, trajectories = 10);
solution_checker(ensemblesol1)
#returns 1x true followed by 9x false

#LBFGS does not have any issues
ensemblesol2 = solve(ensembleprob, LBFGS(), EnsembleSerial(), maxtime = 5, trajectories = 10);
solution_checker(ensemblesol2)
#returns 10x true