Implement an MPS-based MOI solver

Specific Domains Optimization (Mathematical)
1

I want to implement an MPS-based solver interface.
The actual solver is on a server and I have to send an MPS file, get the solution back and update the JuMP model with the solution.
Are there any existing packages that are a good starting point? I have seen NEOS.jl but it is pretty outdated.
So far I know how to get an MPS file from a JuMP model using MathOptFormat.jl.
Do I have to define the rest of the MOI wrapper functions as in e.g. CPLEX.jl?

3 Likes
2

Yes, I believe you have to implement some MOI methods, e.g., for querying the status and solution values etc. But you don’t have to accept all different kinds of function/set pairs for constraints, if they can be handled by bridges. And you also don’t need to support incremental model building or model updates.

The MOI docs have a section about implementing solver interfaces for solvers that only accept full models. This would be most appropriate for your case, I guess.

1 Like
3

If you plan to use MathOptFormat to write the file, you will need to load the problem into MathOptFormat.MPS.InnerModel so you might as well do

struct Optimizer
    mps_model::MathOptFormat.MPS.InnerModel{Float64}
end

and redirect all model building methods to mps_model. Then in MOI.optimize! you can call write_to_file, get the result and stor it in Optimizer.
To redirect model building methods you can do for instance

MOI.supports_constraint(optimizer::Optimizer, F, S) = MOI.supports_constraint(optimizer.mps_model, F, S)
MOI.add_constraint(optimizer::Optimizer, func, set) = MOI.add_constraint(optimizer.mps_model, func, set)
MOI.Utilities.supports_default_copy_to(::Optimizer, ::Bool) = true
...

Alternatively, you can only implement MOI.copy_to and create the MathOptFormat.MPS.InnerModel inside this function as @leethargo suggested but that means that your solver won’t be usable with JuMP direct mode and will use an addictional cache for the bridged model in JuMP automatic and manual model.

1 Like
4

@blegat’s suggestion is good. A work-flow like the following?

struct MyOptimizer <: MOI.AbstractOptimizer
    inner::MathOptFormat.MPS.Model
    solution_cache # You will need to sort this out
end
MOI.supports_constraint(optimizer::Optimizer, F, S) = MOI.supports_constraint(optimizer.mps_model, F, S)
MOI.add_constraint(optimizer::Optimizer, func, set) = MOI.add_constraint(optimizer.mps_model, func, set)
MOI.Utilities.supports_default_copy_to(::Optimizer, ::Bool) = true

function MOI.optimize!(model::MyOptimizer)
    MOI.write_to_file(model.inner, "model.mps")
    send_model_to_server("model.mps")
    sol_file = get_sol_file_from_server()
    model.solution_cache = parse_sol_file(model.inner, sol_file)
    return
end

function MOI.get(model::MyOptimizer, ::MOI.VariablePrimal(), x::MOI.VariableIndex)
    return model.solution_cache[x] # Or something
end

This is very close to what I’ll do with NEOS.jl. Is it possible to make the send_file_to_server modular so that we can swap out NEOS for your server? That would save some code.

2 Likes
5

You will probably need to define lots more methods, like add_variable and some supports. So anytime you see a MethodError it probably means to need to forward that method from MyOptimizer to the inner MPS.Model.

1 Like
6

Thanks to @odow, @blegat and leethargo in the end we came up with the following solution:

mutable struct Optimizer <: MOI.AbstractOptimizer
    mps_model::MathOptFormat.MPS.InnerModel{Float64} # default: MathOptFormat.MPS.Model()
    solution_cache::Dict{MOI.VariableIndex,Float64} # default: Dict{MOI.VariableIndex,Float64}()
    obj_value::Float64 # default: 0.0
    solver_status::MOI.TerminationStatusCode # default: MOI.OPTIMIZE_NOT_CALLED
    solve_time::Float64 # [s] default: -1
    solve_time_max::Int # [s] default: 5 min (300 seconds)
    MIP_gap::Float64 # desired value, default: 0.1 (10 per cent)
    actual_MIP_gap::Float64 # default: -1 (optimize not called)

    function Optimizer(;solve_time_max = 300, MIP_gap = 0.1)
        mps_model = MathOptFormat.MPS.Model()
        solution_cache = Dict{MOI.VariableIndex,Float64}()
        obj_value = 0.0
        solver_status = MOI.OPTIMIZE_NOT_CALLED
        solve_time = -1
        return new(mps_model, solution_cache, obj_value, solver_status, solve_time, solve_time_max, MIP_gap, -1)
    end
end

############################################################
# MOI interface/general
############################################################

# forward add_constraint, add_variable is_empty function to MOI interface of MathOptFormat MPS model
MOI.add_constraint(optimizer::Optimizer, func::MathOptInterface.AbstractFunction, set::MathOptInterface.AbstractSet) = MOI.add_constraint(optimizer.mps_model, func, set)
MOI.add_constraint(optimizer::Optimizer, func::Vector{MathOptInterface.AbstractFunction}, set::Vector{MathOptInterface.AbstractSet}) = MOI.add_constraint(optimizer.mps_model, func, set)
MOI.add_constraints(optimizer::Optimizer, func::Vector{MathOptInterface.AbstractFunction}, set::Vector{MathOptInterface.AbstractSet}) = MOI.add_constraints(optimizer.mps_model, func, set)
MOI.add_variable(optimizer::Optimizer) = MOI.add_variable(optimizer.mps_model)
MOI.add_variables(optimizer::Optimizer, n::Int) = MOI.add_variables(optimizer.mps_model, n)
MOI.is_empty(optimizer::Optimizer) = MOI.is_empty(optimizer.mps_model)

function MOI.empty!(optimizer::Optimizer)
    optimizer.mps_model = MathOptFormat.MPS.Model()
    optimizer.solution_cache = Dict{MOI.VariableIndex,Float64}()
    optimizer.obj_value = 0.0
    optimizer.solver_status = MOI.OPTIMIZE_NOT_CALLED
    optimizer.solve_time = -1
    optimizer.actual_MIP_gap = -1
    return
end

############################################################
# MOI interface/supports
############################################################

function MOI.supports_constraint(::Optimizer, ::Type{MOI.SingleVariable}, ::Type{<:Union{
        MOI.EqualTo{Float64}, MOI.LessThan{Float64},
        MOI.GreaterThan{Float64}, MOI.Interval{Float64},
        MOI.ZeroOne, MOI.Integer}})
    return true
end
function MOI.supports_constraint(::Optimizer, ::Type{MOI.ScalarAffineFunction{Float64}}, ::Type{<:Union{
        MOI.EqualTo{Float64}, MOI.LessThan{Float64},
        MOI.GreaterThan{Float64}, MOI.Interval{Float64}}})
    return true
end
MOI.supports_constraint(::Optimizer, ::Type{MOI.VectorOfVariables}, ::Type{MOI.SOS1{Float64}}) = true
MOI.supports_constraint(::Optimizer, ::Type{MOI.VectorOfVariables}, ::Type{MOI.SOS2{Float64}}) = true
MOI.supports(::Optimizer, ::MOI.ObjectiveFunction{<:Union{MOI.ScalarAffineFunction{Float64}, MOI.SingleVariable}}) = true
MOI.supports(::Optimizer, ::MOI.ObjectiveSense) = true
MOI.supports(::Optimizer, ::MOI.Silent) = true
MOI.supports(::Optimizer, ::MOI.TimeLimitSec) = true
MOI.Utilities.supports_default_copy_to(::Optimizer, ::Bool) = true

############################################################
# MOI interface/set functions
############################################################
function MOI.set(optimizer::Optimizer, ::MOI.ObjectiveSense, sense::MOI.OptimizationSense)
    if sense == MOI.MAX_SENSE
        MOI.set(optimizer.mps_model, MOI.ObjectiveSense(), MOI.MAX_SENSE)
    else ## Other senses are set as minimization (cbc default)
        MOI.set(optimizer.mps_model, MOI.ObjectiveSense(), MOI.MIN_SENSE)
    end
end

MOI.set(optimizer::Optimizer, ::MOI.TimeLimitSec, limit::Real) = optimizer.solve_time_max = convert(Float64,limit)

function MOI.set(optimizer::Optimizer, ::MOI.ObjectiveFunction{F}, f::F) where {F <: MOI.SingleVariable}
    MOI.set(optimizer.mps_model, MOI.ObjectiveFunction{MOI.ScalarAffineFunction{Float64}}(),convert(MOI.ScalarAffineFunction{Float64}, f))
    return
end

function MOI.set(optimizer::Optimizer, ::MOI.ObjectiveFunction{F}, f::F) where {F <: MOI.ScalarAffineFunction{Float64}}
    MOI.set(optimizer.mps_model, MOI.ObjectiveFunction{MOI.ScalarAffineFunction{Float64}}(),f)
    return
end

############################################################
# MOI interface/get functions
############################################################
MOI.get(optimizer::Optimizer, ::MOI.TerminationStatus) = optimizer.solver_status
MOI.get(optimizer::Optimizer, ::MOI.VariablePrimal, idx::MOI.VariableIndex) = optimizer.solution_cache[idx]
MOI.get(optimizer::Optimizer, ::MOI.SolveTime) = optimizer.solve_time
MOI.get(optimizer::Optimizer, ::MOI.TimeLimitSec) = optimizer.solve_time_max
MOI.get(optimizer::Optimizer, ::MOI.RelativeGap) = optimizer.actual_MIP_gap
MOI.get(optimizer::Optimizer, ::MOI.ObjectiveValue) = optimizer.obj_value
MOI.get(::Optimizer, ::MOI.SolverName) = "MPS-based MOI Solver for JuMP"

############################################################
# MOI interface/show
############################################################
Base.show(io::IO, optimizer::Optimizer) = print(io, MOI.get(optimizer,MOI.SolverName()))

############################################################
# MOI interface/optimize
############################################################

function MOI.optimize!(optimizer::Optimizer)    
    io = IOBuffer()
    MOI.write_to_file(optimizer.mps_model, io)
    mps_str = String(take!(io))
 
    result = send_to_server(mps_str)

    optimizer.solution_cache = parse_result(result,"solution_cache")
    optimizer.obj_value = parse_result(result,"obj_value")
    optimizer.solve_time = parse_result(result,"solve_time")
    optimizer.actual_MIP_gap = parse_result(result,"actual_MIP_gap")
    optimizer.solver_status =  parse_result(result,"solver_status") # e.g. MOI.OPTIMAL

    return
end

Maybe you can use the code for NEOS.jl for MPS-based solver as the package is not working any more.

7

This looks perfect! Please use any and all code you need from NEOS.jl. it would be great to revive it!

8

Looks good! Note that you should support MOI.RawParameter and load the parameter given in the constructor with MOI.RawParameter as we do in solver interfaces updated for MOI v0.9. The reason is that we will probably stop giving parameters in the constructor in JuMP v0.21: https://github.com/JuliaOpt/JuMP.jl/pull/2090