Memory Problem with JuMP and pmap

Specific Domains Optimization (Mathematical)
,
1

Hallo,

i want to solve some JuMP models in parallel. For this i tryed using pmap from the Distributed libarary. Here you can see a simple example:

using JuMP, SCIP
using Distributed


addprocs(4)

@everywhere begin
    using Pkg
    Pkg.activate(".")
    using JuMP, SCIP
end
@everywhere function run_problem(model::JuMP.Model ,weight::Float64)

    set_objective_sense(model, JuMP.MIN_SENSE)
    set_objective_function(model, weight * model[:obj])

    optimize!(model)

    return JuMP.value(model[:obj])
end

model = JuMP.Model()

@variable(model, x[1:2], Int)
@expression(model, obj, -2 * x[1] - 3 * x[2])
@constraint(model, 4 * x[1] - 2 * x[2] >= 0)
@constraint(model, 2 * x[2] + 2 * x[1] ≤ 30)
@constraint(model, x[2] >= 0)
@constraint(model, 0.5 * x[1] + 3.75 - x[2] >= 0)

set_optimizer(model, SCIP.Optimizer)
set_silent(model)
@time begin
    for k in 1:100
        sol = pmap(run_problem, repeat([model], 100), ones(100))
    end
end

The problem is that the memory is not freed after calling pmap. If I increase the number of iterations the whole memory fills up.
Thanks for your help and feedback

2

Hi there, welcome to the forum.

Take a read of this tutorial: Parallelism · JuMP

When working with JuMP models in parallel, you need to construct a model on each worker, rather than creating one and then passing it to the worker.

1 Like
3

It still needs more memory

using JuMP, SCIP
using Distributed
using BenchmarkTools

addprocs(8)

@everywhere begin
    using Pkg
    Pkg.activate(".")
    using JuMP, SCIP
end
@everywhere function run_problem(weight::Float64)

    model = JuMP.Model()

    @variable(model, x[1:2], Int)
    @expression(model, obj, -2 * x[1] - 3 * x[2])
    @constraint(model, 4 * x[1] - 2 * x[2] >= 0)
    @constraint(model, 2 * x[2] + 2 * x[1] ≤ 30)
    @constraint(model, x[2] >= 0)
    @constraint(model, 0.5 * x[1] + 3.75 - x[2] >= 0)

    set_optimizer(model, SCIP.Optimizer)
    set_silent(model)

    set_objective_sense(model, JuMP.MIN_SENSE)
    set_objective_function(model, weight * model[:obj])

    optimize!(model)

    sol = JuMP.value(model[:obj])
    # model =nothing
    # GC.gc()
    return sol
end



@benchmark begin
    kappa = 100
    local p = 1
    while p < kappa
        sol = pmap(run_problem, ones(100))
        sol = Nothing
        p += 1
    end
end