Resolve models in parallel within a loop

Specific Domains Optimization (Mathematical)
1

Hi all, I am trying to implement an algorithm in JuMP. In each iteration, I will need to solve several models in parallel, update some data and then resolve these models in the next iteration with different objectives (constraints and variables are the same). What I am thinking is that I set up models without objectives beforehand, and then in each iteration I use pmap to set new objectives and resolve. I don’t want to rebuild models from scratch in each iteration. I am wondering is this doable and supported in JuMP? Thanks.

2

I am wondering is this doable and supported in JuMP?

There is nothing “in” JuMP that supports this. However, it does support changing the objective of a model and efficiently resolving the problem.

However, you can use Julia’s parallel features to parallelize the solves. It is worth having a read of some related posts as there are quite a few traps you can run into:
https://discourse.julialang.org/search?q=jump%20parallel

Have a read, try some things out, and then come back if you run into problems.

3

It sounds pretty similar to what I used in Juniper.jl

The function itself

So you basically load the basis model to all processors and then use remotecall_fetch to call the processors with information like the new objective and get that back. All in a pmap function (@sync, @async loop :wink: )

Don’t hesitate to comment here if you need some more information.
More about the pmap function

4

One constraint to be aware of is that JuMP Model objects aren’t designed to support serialization. pmapping over a list of models to solve generally won’t work, because that involves sending the model across processes. You should try to build a model locally once per process.

5

Thanks, I am looking into this!

6

Thanks! That’s helpful!

7

Thanks for the reply. Can you please specify how to

Say I have 3 models and 3 processes, can I do something like:

@everywhere using JuMP
@everywhere using Ipopt

function generateModel()
    mList = Array(JuMP.Model,3)
    for i in 1:3
        m = Model(solver = IpoptSolver(print_level = 0))
        @variable(m, x[1:2], lowerbound = -1, upperbound = 1)
        a = rand(2)
        @constraint(m, dot(a, x) == 0)
        mList[i] = m
    end
    return mList
end

@everywhere function solveModel(m::JuMP.Model)
    c = rand(2)
    @objective(m, Min, dot(c,m[:x]))
    s = solve(m)
    return getobjectivevalue(m)
end

mList = generateModel()

addprocs(2)

for i in 1:3
    rr = RemoteChannel(i)
    put!(rr, mList[i])
end

bList = Any[]

@time for k in 1:100
    b = Dict(i=>0.0 for i in 1:3)
    @sync begin
        for i in 1:3
            b[i] = solveModel(mList[i])
           
        end
    end
    push!(bList, b)
end

When I tried to put each model into a process, I had

LoadError: On worker 2:
UndefVarError: JuMP not defined

In addition, can this make sure in every iteration(1:100), the model i will be solved in process i? I am also a little confused about when to use @parallel after @sync.

Thank you!

8

Hi. Thank you for the reply. I am trying to workout a simple example:

addprocs(2)

@everywhere using JuMP
@everywhere using Ipopt

@everywhere function generateModel()
    m = Model(solver = IpoptSolver(print_level = 0))
    @variable(m, x[1:2], lowerbound = -1, upperbound = 1)
    a = rand(2)
    @constraint(m, dot(a,x) == 0)
    return m
end

@everywhere function solveModel(m::JuMP.Model)
    c = rand(2)
    @objective(m, Min, dot(c, m[:x]))
    solve(m)
    return getobjectivevalue(m)
end

mList = Array{JuMP.Model}(3)
@sync for i in 1:3
    mList[i] = remotecall_fetch(generateModel, i)
end

I think I am generating models in each process, and my first question is : am I generating models in a parallel way?

In addition, I am a little confused about the following different approaches to solve the model
(a)

sol = Array{Float64}(3)
@time @sync for i in 1:3
    sol[i] = remotecall_fetch(solveModel,i,mList[i])
end

(b)

sol = Array{Float64}(3)
@time @sync @parallel for i in 1:3
    sol[i] = remotecall_fetch(solveModel,i,mList[i])
end

(c)

sol = Array{Float64}(3)
@time @sync  for i in 1:3
    sol[i] = remote_do(solveModel,i,mList[i])
end

(d)

sol = pmap(solveModel, mList)

Anyone can help explain the difference of the four approaches? Thanks!

9

It looks like you’re adding procs after using JuMP and declaring functions. The procs you add are not able to use anything declared with @everywhere before the proc is added.

10

Thanks for the reply. Yes you are right. I just realized that. I changed the code in the previous reply.

11

Hi. Could you please explain what is happening in the while loop marked by the marco @async in the map function? Thanks

12

You have basically the @sync loop over the processors and the @async loop over the tasks. At the beginning you get the new task (for you a new objective from a list) and break if the list is empty. This is all done by the master processor. Then use remotecall_fetch function to send the task to the processor. Do you need more information?

13

I see. So assume I have equal number (n) of models and processes, and I want each process to take care of one model. Can I do something like

@sync for i in 1:n
    @async result[i] = remotecall_fetch(solveModel, i, modelList[i])
end

where modelList is an array of n JuMP models.

In addition, what I need is that in each iteration of my algorithm, I need to implement the above code (in parallel) to resolve n models. But each process will always handle the same model. Is it necessary to build and store each model in a process (say with the same id), or is it find to create all models in the main process (with id 1)?

Thanks.