Ways to save and load JuMP models from/to disk or memory

Specific Domains Optimization (Mathematical)
,
1

Dear team,

Is there a way to save JuMP models to disk and then load them back and then just perform some modifications?

Thank you!

2

In short (and in general), no. Because of the reliance on the underlying solver which is often not written in Julia.

However, in the next JuMP release (which will be 0.21) (or now if you run ] add JuMP#master), support is added for reading and writing to a variety of file formats.

See Solvers ¡ JuMP

In particular, if you want to read/write MPS files, you will be able to go:

model = Model()
# ...
write_to_file(model, "my_file.mps")

# Then, in a new session
new_model = read_from_file("my_file.mps")

If you have a model with more general constraints (e.g., complementarity, or cones + quadratics), you can try our new “MathOptFormat” file type:

model = Model()
# ...
write_to_file(model, "my_file.mof.json"; format = MOI.FORMAT_MOF)

# Then, in a new session
new_model = read_from_file("my_file.mof.json"; format = MOI.FORMAT_MOF)
5 Likes
3

There is apparently something wrong when installing MathOptFormat. See below for a computer where neither HTTP.jl nor JuMP.jl are installed…

julia> Pkg.add(“MathOptFormat”)
Resolving package versions…
ERROR: Unsatisfiable requirements detected for package HTTP [cd3eb016]:
HTTP [cd3eb016] log:
├─possible versions are: 0.8.0 or uninstalled
└─found to have no compatible versions left with MathOptFormat [f4570300]
└─MathOptFormat [f4570300] log:
├─possible versions are: [0.1.0-0.1.1, 0.2.0-0.2.2] or uninstalled
└─restricted to versions * by an explicit requirement, leaving only versions [0.1.0-0.1.1, 0.2.0-0.2.2]
Stacktrace:
[1] #propagate_constraints!#61(::Bool, ::typeof(Pkg.GraphType.propagate_constraints!), ::Pkg.GraphType.Graph, ::Set{Int64}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1005
[2] propagate_constraints! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:946 [inlined]
[3] #simplify_graph!#121(::Bool, ::typeof(Pkg.GraphType.simplify_graph!), ::Pkg.GraphType.Graph, ::Set{Int64}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1460
[4] simplify_graph! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1460 [inlined] (repeats 2 times)
[5] resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Nothing) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:388
[6] resolve_versions! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:328 [inlined]
[7] #add_or_develop#63(::Array{Base.UUID,1}, ::Symbol, ::typeof(Pkg.Operations.add_or_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:1235
[8] #add_or_develop at .\none:0 [inlined]
[9] #add_or_develop#17(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add_or_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:59
[10] #add_or_develop at .\none:0 [inlined]
[11] #add_or_develop#16 at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:36 [inlined]
[12] #add_or_develop at .\none:0 [inlined]
[13] #add_or_develop#13 at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:34 [inlined]
[14] #add_or_develop at .\none:0 [inlined]
[15] #add_or_develop#12(::Base.Iterators.Pairs{Symbol,Symbol,Tuple{Symbol},NamedTuple{(:mode,),Tuple{Symbol}}}, ::typeof(Pkg.API.add_or_develop), ::String) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:33
[16] #add_or_develop at .\none:0 [inlined]
[17] #add#22 at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:64 [inlined]
[18] add(::String) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:64
[19] top-level scope at none:0

4

You have a conflict between HTTP and another package. What’s the output of

import Pkg
Pkg.status()
5

[ Info: Loading DataFrames support into Gadfly.jl
Status C:\Users\DiegoLucadeTena\.juliapro\JuliaPro_v1.2.0-1\environments\v1.2\Project.toml
[c52e3926] Atom v0.11.3
[336ed68f] CSV v0.5.18
[a93c6f00] DataFrames v0.20.0
[c91e804a] Gadfly v1.0.1
[7073ff75] IJulia v1.19.0
[c601a237] Interact v0.10.3
[4076af6c] JuMP v0.19.2
[e5e0dc1b] Juno v0.7.1
[f8899e07] LinQuadOptInterface v0.6.0
[b8f27783] MathOptInterface v0.8.4
[9e70acf3] Xpress v0.9.1
[ade2ca70] Dates
[8bb1440f] DelimitedFiles
[8ba89e20] Distributed
[cd3eb016] HTTP
julia>

6

Unfortunately you cannot use MathOptFormat and Xpress at the same time.

Xpress is being updated (https://github.com/JuliaOpt/Xpress.jl/pull/44). Once that PR is merged, you will be able to.

7

Great to know Oscar, thank you for the prompt response.
D

8

By the way Oscar, are other commertial solvers, such as Gurobi, compatible with this feature?

9

Yes, Gurobi.jl and CPLEX.jl should both work.

10

I guess serialize should do the trick now, for anyone that might be still wondering. With Julia 1.7 and JuMP v0.22.1 works fine.

2 Likes
11

But that is super brittle isn’t it? Looks like it’s only guaranteed to work on the same Julia installation (same version, same computer, same image).

2 Likes
12

Yes, serializing should not be used as a long term storage solution.

1 Like
13

Sorry to hijack the thread, but when serializing and deserializing AffExpr, I got this error when using isapprox to compare:

ERROR: MethodError: no method matching rtoldefault(::Type{AffExpr}, ::Type{AffExpr}, ::Int64)
rtoldefault(::Union{Type{T}, T}, ::Union{Type{S}, S}, ::Real) where {T<:Number, S<:Number} at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/floatfuncs.jl:315

@odow could you help with a possible definition where coefficients of AffExpr are Float64?

Base.rtoldefault(::Type{AffExpr}, ::Type{AffExpr}, ::Int64) =

Edit: I am not sure if the deserialized AffExpr can be used to resume formulating the model as VariableRef could change…

14

Do you have a reproducible example?

15

Here is a MWE, but I dont think serialization/deserialization would work because I no longer own the old variables when re-creating the model. Perform the following in the same Julia session:

using JuMP, Serialization
m = JuMP.Model()
@variable(m, x[1:2])
@expression(m, ex, 1.0x[1]-1.0x[2])
serialize(“ex”, ex)
ex_de = deserialize(“ex”)
ex ≈ ex_de # ERROR
m2 = JuMP.Model() # Try to recreate model using deserialization (maybe use a new Julia session?)
pointer(x) # check what is currently bound to x
pointer_from_objref(x) # check what is currently bound to x
@variable(m2, x[1:2])
pointer(x) # x refers to new variableref of m2
pointer_from_objref(x) # changed compared to before
@expression(m2, ex2, ex_de)
m2[:ex2] # wrong expression

For these reasons, I dont think its a good idea to save part of the data used in constructing a model to file and use deserialization to recreate a model starting from a middle point (either in same Julia session or in a fresh one).

My current workaround is via JuMP.coefficient which can be saved and retrieved faithfully using JLD2.

Your advice is greatly appreciated @odow.

16

Unfortunately, you can’t use serialization to serialize an expression like this because each variable stores a reference to the model. When you serialize and then deserialize the expression, the model m is different to the model inside the expression.

What you can do is serialize the entire model:

julia> using JuMP, Serialization

julia> m = JuMP.Model()
A JuMP Model
Feasibility problem with:
Variables: 0
Model mode: AUTOMATIC
CachingOptimizer state: NO_OPTIMIZER
Solver name: No optimizer attached.

julia> @variable(m, x[1:2])
2-element Vector{VariableRef}:
 x[1]
 x[2]

julia> @expression(m, ex, 1.0x[1]-1.0x[2])
x[1] - x[2]

julia> serialize("ex", m)

julia> ex_de = deserialize("ex")
A JuMP Model
Feasibility problem with:
Variables: 2
Model mode: AUTOMATIC
CachingOptimizer state: NO_OPTIMIZER
Solver name: No optimizer attached.
Names registered in the model: ex, x

julia> ex_de[:ex]
x[1] - x[2]

What are you trying to achieve?

17

This looks good actually. Can I deserialize a model and continue adding expressions, constraints, etc? I am doing a lot of experimentation with heavy computation and would like so “save” stages of my model’s formulation.

18

Can I deserialize a model and continue adding expressions, constraints, etc

Sure. But a few things:

  • You need to use the correct variables, so make sure to look up x = model_ex[:x] and so on.
  • Serializing after you have called optimize! can lead to problems. If you load a model that has called optimize!, run set_optimizer(model, HiGHS.Optimizer) (or similar).

I am doing a lot of experimentation with heavy computation and would like so “save” stages of my model’s formulation.

You’re probably better off saving the data associated with the model, rather than the JuMP model itself. I don’t see much point in repeatedly serializing the entire model to disk.

2 Likes
19

@odow I have encountered a situation where I need to manually edit some data in an mps file. So far I have been able to read the original mps file as an Array{UInt8,1} and apply the desired changes to that array. However, I cannot seem to save it back as another mps file (I am using FileIO for saving) and I get the error “No applicable_savers found for UNKNOWN”. Do you have any fixes for this problem? Thank you in advance.

20

I am using FileIO for saving

Why is this necessary? Just read and write the file normally.

model = read("model.mps", String)
# Replace all `x` with `y`
model = replace(model, "x" => "y")
write("model.mps", model)
2 Likes