# Implement an MPS-based MOI solver

**URL:** <https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218>\
**Category:** Optimization (Mathematical)\
**Tags:** question\
**Created:** [September 27, 2019, 7:10am UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218 "2019-09-27T07:10:21Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ohmsweetohm1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ohmsweetohm1/32/49126_2.png) [@ohmsweetohm1](https://discourse.julialang.org/u/ohmsweetohm1)\
**Post date:** [September 27, 2019, 7:10am UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218/1 "2019-09-27T07:10:22Z")

</div>

I want to implement an MPS-based solver interface.  
The actual solver is on a server and I have to send an MPS file, get the solution back and update the JuMP model with the solution.  
Are there any existing packages that are a good starting point? I have seen NEOS.jl but it is pretty outdated.  
So far I know how to get an MPS file from a JuMP model using MathOptFormat.jl.  
Do I have to define the rest of the MOI wrapper functions as in e.g. CPLEX.jl?

---

<div class="post-metadata">

**Author:** ![leethargo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leethargo/32/6004_2.png) [@leethargo](https://discourse.julialang.org/u/leethargo)\
**Post date:** [September 27, 2019, 11:51am UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218/2 "2019-09-27T11:51:04Z")

</div>

Yes, I believe you have to implement _some_ MOI methods, e.g., for querying the status and solution values etc. But you don’t have to accept all different kinds of function/set pairs for constraints, if they can be handled by bridges. And you also don’t need to support incremental model building or model updates.

The [MOI docs](http://www.juliaopt.org/MathOptInterface.jl/stable/apimanual/#Implementing-copy-1) have a section about implementing solver interfaces for solvers that only accept full models. This would be most appropriate for your case, I guess.

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [September 27, 2019, 12:19pm UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218/3 "2019-09-27T12:19:05Z")

</div>

If you plan to use [MathOptFormat](https://github.com/odow/MathOptFormat.jl/blob/master/src/MPS/MPS.jl) to write the file, you will need to load the problem into `MathOptFormat.MPS.InnerModel` so you might as well do

```julia
struct Optimizer
    mps_model::MathOptFormat.MPS.InnerModel{Float64}
end

```

and redirect all model building methods to `mps_model`. Then in `MOI.optimize!` you can call `write_to_file`, get the result and stor it in `Optimizer`.  
To redirect model building methods you can do for instance

```julia
MOI.supports_constraint(optimizer::Optimizer, F, S) = MOI.supports_constraint(optimizer.mps_model, F, S)
MOI.add_constraint(optimizer::Optimizer, func, set) = MOI.add_constraint(optimizer.mps_model, func, set)
MOI.Utilities.supports_default_copy_to(::Optimizer, ::Bool) = true
...

```

Alternatively, you can only implement `MOI.copy_to` and create the `MathOptFormat.MPS.InnerModel` inside this function as @leethargo suggested but that means that your solver won’t be usable with JuMP direct mode and will use an addictional cache for the bridged model in JuMP automatic and manual model.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [September 27, 2019, 1:22pm UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218/4 "2019-09-27T13:22:47Z")

</div>

@blegat’s suggestion is good. A work-flow like the following?

```nohighlight
struct MyOptimizer <: MOI.AbstractOptimizer
    inner::MathOptFormat.MPS.Model
    solution_cache # You will need to sort this out
end
MOI.supports_constraint(optimizer::Optimizer, F, S) = MOI.supports_constraint(optimizer.mps_model, F, S)
MOI.add_constraint(optimizer::Optimizer, func, set) = MOI.add_constraint(optimizer.mps_model, func, set)
MOI.Utilities.supports_default_copy_to(::Optimizer, ::Bool) = true

function MOI.optimize!(model::MyOptimizer)
    MOI.write_to_file(model.inner, "model.mps")
    send_model_to_server("model.mps")
    sol_file = get_sol_file_from_server()
    model.solution_cache = parse_sol_file(model.inner, sol_file)
    return
end

function MOI.get(model::MyOptimizer, ::MOI.VariablePrimal(), x::MOI.VariableIndex)
    return model.solution_cache[x] # Or something
end

```

This is very close to what I’ll do with NEOS.jl. Is it possible to make the `send_file_to_server` modular so that we can swap out NEOS for your server? That would save some code.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [September 27, 2019, 1:54pm UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218/5 "2019-09-27T13:54:03Z")

</div>

You will probably need to define lots more methods, like `add_variable` and some `supports`. So anytime you see a `MethodError` it probably means to need to forward that method from `MyOptimizer` to the inner `MPS.Model`.

---

<div class="post-metadata">

**Author:** ![agcm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/agcm/32/24304_2.png) [@agcm](https://discourse.julialang.org/u/agcm)\
**Post date:** [October 31, 2019, 11:31am UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218/6 "2019-10-31T11:31:45Z")

</div>

Thanks to @odow, @blegat and leethargo in the end we came up with the following solution:

```julia
mutable struct Optimizer <: MOI.AbstractOptimizer
    mps_model::MathOptFormat.MPS.InnerModel{Float64} # default: MathOptFormat.MPS.Model()
    solution_cache::Dict{MOI.VariableIndex,Float64} # default: Dict{MOI.VariableIndex,Float64}()
    obj_value::Float64 # default: 0.0
    solver_status::MOI.TerminationStatusCode # default: MOI.OPTIMIZE_NOT_CALLED
    solve_time::Float64 # [s] default: -1
    solve_time_max::Int # [s] default: 5 min (300 seconds)
    MIP_gap::Float64 # desired value, default: 0.1 (10 per cent)
    actual_MIP_gap::Float64 # default: -1 (optimize not called)

    function Optimizer(;solve_time_max = 300, MIP_gap = 0.1)
        mps_model = MathOptFormat.MPS.Model()
        solution_cache = Dict{MOI.VariableIndex,Float64}()
        obj_value = 0.0
        solver_status = MOI.OPTIMIZE_NOT_CALLED
        solve_time = -1
        return new(mps_model, solution_cache, obj_value, solver_status, solve_time, solve_time_max, MIP_gap, -1)
    end
end

############################################################
# MOI interface/general
############################################################

# forward add_constraint, add_variable is_empty function to MOI interface of MathOptFormat MPS model
MOI.add_constraint(optimizer::Optimizer, func::MathOptInterface.AbstractFunction, set::MathOptInterface.AbstractSet) = MOI.add_constraint(optimizer.mps_model, func, set)
MOI.add_constraint(optimizer::Optimizer, func::Vector{MathOptInterface.AbstractFunction}, set::Vector{MathOptInterface.AbstractSet}) = MOI.add_constraint(optimizer.mps_model, func, set)
MOI.add_constraints(optimizer::Optimizer, func::Vector{MathOptInterface.AbstractFunction}, set::Vector{MathOptInterface.AbstractSet}) = MOI.add_constraints(optimizer.mps_model, func, set)
MOI.add_variable(optimizer::Optimizer) = MOI.add_variable(optimizer.mps_model)
MOI.add_variables(optimizer::Optimizer, n::Int) = MOI.add_variables(optimizer.mps_model, n)
MOI.is_empty(optimizer::Optimizer) = MOI.is_empty(optimizer.mps_model)

function MOI.empty!(optimizer::Optimizer)
    optimizer.mps_model = MathOptFormat.MPS.Model()
    optimizer.solution_cache = Dict{MOI.VariableIndex,Float64}()
    optimizer.obj_value = 0.0
    optimizer.solver_status = MOI.OPTIMIZE_NOT_CALLED
    optimizer.solve_time = -1
    optimizer.actual_MIP_gap = -1
    return
end

############################################################
# MOI interface/supports
############################################################

function MOI.supports_constraint(::Optimizer, ::Type{MOI.SingleVariable}, ::Type{<:Union{
        MOI.EqualTo{Float64}, MOI.LessThan{Float64},
        MOI.GreaterThan{Float64}, MOI.Interval{Float64},
        MOI.ZeroOne, MOI.Integer}})
    return true
end
function MOI.supports_constraint(::Optimizer, ::Type{MOI.ScalarAffineFunction{Float64}}, ::Type{<:Union{
        MOI.EqualTo{Float64}, MOI.LessThan{Float64},
        MOI.GreaterThan{Float64}, MOI.Interval{Float64}}})
    return true
end
MOI.supports_constraint(::Optimizer, ::Type{MOI.VectorOfVariables}, ::Type{MOI.SOS1{Float64}}) = true
MOI.supports_constraint(::Optimizer, ::Type{MOI.VectorOfVariables}, ::Type{MOI.SOS2{Float64}}) = true
MOI.supports(::Optimizer, ::MOI.ObjectiveFunction{<:Union{MOI.ScalarAffineFunction{Float64}, MOI.SingleVariable}}) = true
MOI.supports(::Optimizer, ::MOI.ObjectiveSense) = true
MOI.supports(::Optimizer, ::MOI.Silent) = true
MOI.supports(::Optimizer, ::MOI.TimeLimitSec) = true
MOI.Utilities.supports_default_copy_to(::Optimizer, ::Bool) = true

############################################################
# MOI interface/set functions
############################################################
function MOI.set(optimizer::Optimizer, ::MOI.ObjectiveSense, sense::MOI.OptimizationSense)
    if sense == MOI.MAX_SENSE
        MOI.set(optimizer.mps_model, MOI.ObjectiveSense(), MOI.MAX_SENSE)
    else ## Other senses are set as minimization (cbc default)
        MOI.set(optimizer.mps_model, MOI.ObjectiveSense(), MOI.MIN_SENSE)
    end
end

MOI.set(optimizer::Optimizer, ::MOI.TimeLimitSec, limit::Real) = optimizer.solve_time_max = convert(Float64,limit)

function MOI.set(optimizer::Optimizer, ::MOI.ObjectiveFunction{F}, f::F) where {F <: MOI.SingleVariable}
    MOI.set(optimizer.mps_model, MOI.ObjectiveFunction{MOI.ScalarAffineFunction{Float64}}(),convert(MOI.ScalarAffineFunction{Float64}, f))
    return
end

function MOI.set(optimizer::Optimizer, ::MOI.ObjectiveFunction{F}, f::F) where {F <: MOI.ScalarAffineFunction{Float64}}
    MOI.set(optimizer.mps_model, MOI.ObjectiveFunction{MOI.ScalarAffineFunction{Float64}}(),f)
    return
end

############################################################
# MOI interface/get functions
############################################################
MOI.get(optimizer::Optimizer, ::MOI.TerminationStatus) = optimizer.solver_status
MOI.get(optimizer::Optimizer, ::MOI.VariablePrimal, idx::MOI.VariableIndex) = optimizer.solution_cache[idx]
MOI.get(optimizer::Optimizer, ::MOI.SolveTime) = optimizer.solve_time
MOI.get(optimizer::Optimizer, ::MOI.TimeLimitSec) = optimizer.solve_time_max
MOI.get(optimizer::Optimizer, ::MOI.RelativeGap) = optimizer.actual_MIP_gap
MOI.get(optimizer::Optimizer, ::MOI.ObjectiveValue) = optimizer.obj_value
MOI.get(::Optimizer, ::MOI.SolverName) = "MPS-based MOI Solver for JuMP"

############################################################
# MOI interface/show
############################################################
Base.show(io::IO, optimizer::Optimizer) = print(io, MOI.get(optimizer,MOI.SolverName()))

############################################################
# MOI interface/optimize
############################################################

function MOI.optimize!(optimizer::Optimizer)    
    io = IOBuffer()
    MOI.write_to_file(optimizer.mps_model, io)
    mps_str = String(take!(io))
 
    result = send_to_server(mps_str)

    optimizer.solution_cache = parse_result(result,"solution_cache")
    optimizer.obj_value = parse_result(result,"obj_value")
    optimizer.solve_time = parse_result(result,"solve_time")
    optimizer.actual_MIP_gap = parse_result(result,"actual_MIP_gap")
    optimizer.solver_status = parse_result(result,"solver_status") # e.g. MOI.OPTIMAL

    return
end

```

Maybe you can use the code for `NEOS.jl` for MPS-based solver as the package is not working any more.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 31, 2019, 12:22pm UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218/7 "2019-10-31T12:22:53Z")

</div>

This looks perfect! Please use any and all code you need from NEOS.jl. it would be great to revive it!

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [October 31, 2019, 12:55pm UTC](https://discourse.julialang.org/t/implement-an-mps-based-moi-solver/29218/8 "2019-10-31T12:55:44Z")

</div>

Looks good! Note that you should support `MOI.RawParameter` and load the parameter given in the constructor with `MOI.RawParameter` as we do in solver interfaces updated for MOI v0.9. The reason is that we will probably stop giving parameters in the constructor in JuMP v0.21: [https://github.com/JuliaOpt/JuMP.jl/pull/2090](https://github.com/JuliaOpt/JuMP.jl/pull/2090)