What is the ideal way to handle CPLEX errors in JuMP and Julia? For example, sometimes I run optimize!(m) and it prints CPLEX Error 1292: Invalid choice of optimization method. However, if I use a try/catch and catch the error, the only information I get is the error message. There’s no code or anything else. Does JuMP provide error handling? Should I still use try/catch? What is the designed way to use handle these types of errors?
1 Like
Can you provide a reproducible example?
If it is an error you should be concerned about, Julia will throw an error you can catch. Sometimes, the internal C API prints error messages that you can safely ignore. Here are some examples:
https://github.com/jump-dev/CPLEX.jl/issues/355
https://github.com/jump-dev/CPLEX.jl/issues/356
2 Likes
I think the issue I just opened may be relevant here too. IMHO there should exist an interface like is_solver_error and error_code to query Exceptions originally thrown by solvers and re-thrown (inside Julia) by the solver wrappers.
The case of that issue will be addressed without needing changes in the exception system, however I would see it as a nice improvement. As a scientific user of JuMP, I often want to ignore some exceptions, and send a notification to myself for the really unexpected ones.
1 Like
I’m having trouble reproducing it myself honestly. I run tens of thousands optimization calls and 99.999% of them work perfectly. The error I am getting is:
CPLEX Error 1292: Invalid choice of optimization method.
The wrapper does throw the error again but since it doesn’t have any other fields other than the msg I was curious what the best way to test which error it was. I am worried that my error handling will catch a different error I am not expecting since I can’t test for CPLEX error 1292
Please provide the full stack trace.
Are you modifying the model and resolving? Adding/removing integrality/constraints/quadratic objectives?
Sure. I am running a bunch of the optimize calls in hopes of getting the error again and when I do I will post the full stack trace. It is a very rare occurrence though.
I am running essentially linear regression. I am minimizing the sum of errors from a linear dot product and with the sum I have several other terms that can be thought of as l1 and l2 regularizes. So the objective function in addition to squared sum of errors, has several sums of squared weight coefficients and priored weight coefficients. Additionally it has several sums of absolute values of weight and priored weight coefficients which introduces slack variable constraints. The only real change I am making to the objective function is the trade-off variables. So each sum has a separate coefficient it is multiplied with to weight it differently.
From that perspective, it is weird that I am getting this particular error as the structure of the objective function and constraints stays the same
Try solving linear → quadratic → linear → quadratic. It may be that one of your weights went to 0.
I got the error again. Here is the full stack trace.
CPLEX Error 1292: Invalid choice of optimization method.
ERROR: LoadError: CPLEX Error 1292: Invalid choice of optimization method.
Stacktrace:
[1] _check_ret(::CPLEX.Env, ::Int32) at /home/pmcvay/.julia/packages/CPLEX/uUjlQ/src/MOI/MOI_wrapper.jl:126
[2] _check_ret at /home/pmcvay/.julia/packages/CPLEX/uUjlQ/src/MOI/MOI_wrapper.jl:243 [inlined]
[3] optimize!(::CPLEX.Optimizer) at /home/pmcvay/.julia/packages/CPLEX/uUjlQ/src/MOI/MOI_wrapper.jl:2499
[4] optimize!(::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}) at /home/pmcvay/.julia/packages/MathOptInterface/ZJFKw/src/Bridges/bridge_optimizer.jl:264
[5] optimize!(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.AbstractOptimizer,MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}) at /home/pmcvay/.julia/packages/MathOptInterface/ZJFKw/src/Utilities/cachingoptimizer.jl:215
[6] optimize!(::Model, ::Nothing; bridge_constraints::Bool, ignore_optimize_hook::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/pmcvay/.julia/packages/JuMP/y5vgk/src/optimizer_interface.jl:139
[7] optimize! at /home/pmcvay/.julia/packages/JuMP/y5vgk/src/optimizer_interface.jl:115 [inlined] (repeats 2 times)
[8] (::var"#76#81"{Int64,Array{GenericQuadExpr{Float64,VariableRef},1},Array{var"#15#18"{Int64,DataFrame,DataFrame},1}})(::Array{Float64,1}) at /home/pmcvay/lc.jl:33
[9] bestRegParams2(::var"#76#81"{Int64,Array{GenericQuadExpr{Float64,VariableRef},1},Array{var"#15#18"{Int64,DataFrame,DataFrame},1}}) at /home/pmcvay/lc.jl:152
[10] top-level scope at /home/pmcvay/lc.jl:222
in expression starting at /home/pmcvay/lc.jl:221
I do keep the trade-off weights on a log scale. So when used in the objective call, it is exp(w), and w is bounded in range (-10, 10)
Unfortunately, we’re going to need a reproducible example to debug this further. It could be a bug in CPLEX.jl, or an internal bug in CPLEX.
Can you create model and print list_of_constraint_types(model)?
Tuple{DataType,DataType}[(GenericAffExpr{Float64,VariableRef}, MathOptInterface.GreaterThan{Float64}), (VariableRef, MathOptInterface.GreaterThan{Float64})]
I’m also trying to run a bunch of calls and see what the trade off weights are when it fails so see if there is any cause from that or if I can come up with a reproducible example from that.
Is it possible to use the try/catch to save the model objective, variables and constraints to share for others to recreate? I’m not finding any pattern in when it fails based on the tradeoff parameters and so am failing creating a reproducible example
Is it possible to use the try/catch to save the model objective, variables and constraints
for p in params
try
run_model(p)
catch
write_to_file(model, "model_$(p).mof.json")
end
end
Or add lots of println statements and then look at the last thing printed.
I know from experience that debugging these errors can be tedious! (I once found an bug in Gurobi that arose once every 100,000 solves or so…)
1 Like
That’s crazy… Thank you for doing that!
I finally was able to save a model that fails, load it back, and rerun it seeing the failure. However, it appears that CPLEX works fine. All the normal output from the CPLEX optimizer appears like it always does, the total time prints, and then I see the failure. This makes me think it may be a communication issue between CPLEX and JuMP. Any advice how to dig into this in more detail?
1 Like
Just to confirm, this errors?
using JuMP, CPLEX
model = read_from_file("model_p.mof.json")
set_optimizer(model, CPLEX.Optimizer)
optimize!(model)
If so, please share the .mof.json file.
Yes, that is correct. Here is the model - it is too large for github and has an unsupported type here. It should be available (if I did the settings correctly) here Dropbox - Model - Simplify your life
I’ll take a look. (I forgot to mention, you can use write_to_file(model, "model_fail.mof.json.gz") to compress the file so it’s not so large.
I can’t reproduce:
julia> using JuMP, CPLEX
[ Info: Precompiling CPLEX [a076750e-1247-5638-91d2-ce28b192dca0]
julia> model = read_from_file("model_fail.mof.json")
A JuMP Model
Minimization problem with:
Variables: 8761
Objective function type: GenericQuadExpr{Float64,VariableRef}
`GenericAffExpr{Float64,VariableRef}`-in-`MathOptInterface.GreaterThan{Float64}`: 10104 constraints
`VariableRef`-in-`MathOptInterface.GreaterThan{Float64}`: 5052 constraints
Model mode: AUTOMATIC
CachingOptimizer state: NO_OPTIMIZER
Solver name: No optimizer attached.
julia> set_optimizer(model, CPLEX.Optimizer)
julia> optimize!(model)
Version identifier: 12.10.0.0 | 2019-11-26 | 843d4de
Number of nonzeros in lower triangle of Q = 305647
Using Nested Dissection ordering
Total time for automatic ordering = 0.19 sec. (229.12 ticks)
Summary statistics for factor of Q:
Rows in Factor = 2378
Integer space required = 35380
Total non-zeros in factor = 846370
Total FP ops to factor = 442792770
Tried aggregator 1 time.
QP Presolve eliminated 8729 rows and 6045 columns.
QP Presolve added 0 rows and 2378 columns.
Reduced QP has 3753 rows, 5094 columns, and 812264 nonzeros.
Reduced QP objective Q matrix has 2145 nonzeros.
Presolve time = 0.35 sec. (444.14 ticks)
Parallel mode: using up to 8 threads for barrier.
Number of nonzeros in lower triangle of A*A' = 2280234
Using Nested Dissection ordering
Total time for automatic ordering = 3.37 sec. (6219.80 ticks)
Summary statistics for Cholesky factor:
Threads = 8
Rows in Factor = 3753
Integer space required = 329016
Total non-zeros in factor = 3206921
Total FP ops to factor = 3411521617
Itn Primal Obj Dual Obj Prim Inf Upper Inf Dual Inf
0 2.4435860e+02 2.2068845e+02 0.00e+00 0.00e+00 2.42e+06
1 2.8436515e+01 7.1281975e+00 9.54e-10 0.00e+00 1.21e+04
2 2.8329481e+01 4.7712020e+00 8.37e-07 0.00e+00 6.22e+01
3 1.7288859e+01 4.7413238e+00 6.47e-13 0.00e+00 2.45e-01
4 5.1080725e+00 4.7476922e+00 3.52e-13 0.00e+00 2.57e-02
5 4.7755915e+00 4.7693769e+00 2.46e-13 0.00e+00 8.85e-01
6 4.7602983e+00 4.7621630e+00 2.09e-13 0.00e+00 2.97e-01
7 4.7598154e+00 4.7599562e+00 2.10e-13 0.00e+00 8.63e-02
8 4.7598015e+00 4.7597704e+00 2.13e-13 0.00e+00 3.89e-02
9 4.7597995e+00 4.7597605e+00 2.17e-13 0.00e+00 2.67e-02
10 4.7597958e+00 4.7597957e+00 1.21e-08 0.00e+00 1.36e-04
11 4.7597958e+00 4.7597958e+00 8.56e-11 0.00e+00 6.81e-07
Barrier time = 6.28 sec. (11443.70 ticks)
Total time on 8 threads = 6.28 sec. (11443.70 ticks)
That is strange. This is my output
julia> m = read_from_file("model_fail.mof.json")
A JuMP Model
Minimization problem with:
Variables: 8761
Objective function type: GenericQuadExpr{Float64,VariableRef}
`GenericAffExpr{Float64,VariableRef}`-in-`MathOptInterface.GreaterThan{Float64}`: 10104 constraints
`VariableRef`-in-`MathOptInterface.GreaterThan{Float64}`: 5052 constraints
Model mode: AUTOMATIC
CachingOptimizer state: NO_OPTIMIZER
Solver name: No optimizer attached.
julia> set_optimizer(m, CPLEX.Optimizer)
julia> optimize!(m)
Version identifier: 12.10.0.0 | 2019-11-26 | 843d4de
Number of nonzeros in lower triangle of Q = 305647
Using Nested Dissection ordering
Total time for automatic ordering = 0.26 sec. (229.12 ticks)
Summary statistics for factor of Q:
Rows in Factor = 2378
Integer space required = 35380
Total non-zeros in factor = 846370
Total FP ops to factor = 442792770
Tried aggregator 1 time.
QP Presolve eliminated 8729 rows and 6045 columns.
QP Presolve added 0 rows and 2378 columns.
Reduced QP has 3753 rows, 5094 columns, and 812264 nonzeros.
Reduced QP objective Q matrix has 2145 nonzeros.
Presolve time = 0.43 sec. (444.14 ticks)
Parallel mode: using up to 20 threads for barrier.
Number of nonzeros in lower triangle of A*A' = 2280234
Using Nested Dissection ordering
Total time for automatic ordering = 3.78 sec. (6219.80 ticks)
Summary statistics for Cholesky factor:
Threads = 20
Rows in Factor = 3753
Integer space required = 329016
Total non-zeros in factor = 3206921
Total FP ops to factor = 3411521617
Itn Primal Obj Dual Obj Prim Inf Upper Inf Dual Inf
0 2.4435860e+02 2.2068845e+02 0.00e+00 0.00e+00 2.42e+06
1 2.8436518e+01 7.1285879e+00 9.36e-10 0.00e+00 1.21e+04
2 2.8329506e+01 4.7714215e+00 8.45e-07 0.00e+00 6.33e+01
3 1.7403077e+01 4.7412113e+00 6.50e-13 0.00e+00 2.46e-01
4 5.2928646e+00 4.7464611e+00 2.87e-13 0.00e+00 1.59e-02
5 4.8240506e+00 4.7671989e+00 1.81e-13 0.00e+00 7.62e-01
6 4.7651333e+00 4.7641820e+00 1.51e-13 0.00e+00 4.56e-01
7 4.7599375e+00 4.7605438e+00 1.41e-13 0.00e+00 1.47e-01
8 4.7598052e+00 4.7598371e+00 1.44e-13 0.00e+00 5.21e-02
9 4.7598001e+00 4.7597857e+00 1.45e-13 0.00e+00 2.96e-02
10 4.7597993e+00 4.7597809e+00 1.48e-13 0.00e+00 2.46e-02
* 4.7598001e+00 4.7597857e+00 1.45e-13 0.00e+00 2.96e-02
Barrier time = 6.42 sec. (9577.86 ticks)
Total time on 20 threads = 6.42 sec. (9577.86 ticks)
CPLEX Error 1292: Invalid choice of optimization method.
ERROR: CPLEX Error 1292: Invalid choice of optimization method.
Stacktrace:
[1] _check_ret(::CPLEX.Env, ::Int32) at /home/pmcvay/.julia/packages/CPLEX/uUjlQ/src/MOI/MOI_wrapper.jl:126
[2] _check_ret at /home/pmcvay/.julia/packages/CPLEX/uUjlQ/src/MOI/MOI_wrapper.jl:243 [inlined]
[3] optimize!(::CPLEX.Optimizer) at /home/pmcvay/.julia/packages/CPLEX/uUjlQ/src/MOI/MOI_wrapper.jl:2499
[4] optimize!(::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}) at /home/pmcvay/.julia/packages/MathOptInterface/ZJFKw/src/Bridges/bridge_optimizer.jl:264
[5] optimize!(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.AbstractOptimizer,MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}) at /home/pmcvay/.julia/packages/MathOptInterface/ZJFKw/src/Utilities/cachingoptimizer.jl:215
[6] optimize!(::Model, ::Nothing; bridge_constraints::Bool, ignore_optimize_hook::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/pmcvay/.julia/packages/JuMP/y5vgk/src/optimizer_interface.jl:139
[7] optimize! at /home/pmcvay/.julia/packages/JuMP/y5vgk/src/optimizer_interface.jl:115 [inlined] (repeats 2 times)
[8] top-level scope at REPL[6]:1
[9] run_repl(::REPL.AbstractREPL, ::Any) at /build/julia/src/julia-1.5.3/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
Perhaps it is a version issue? I know julia 1.5.4 came out recently.
$ julia --version
julia version 1.5.3
(@v1.5) pkg> status CPLEX
Status `~/.julia/environments/v1.5/Project.toml`
[a076750e] CPLEX v0.7.6
(@v1.5) pkg> status JuMP
Status `~/.julia/environments/v1.5/Project.toml`
[4076af6c] JuMP v0.21.6
I think I see the problem. Line 2499 isa coming from here:
https://github.com/jump-dev/CPLEX.jl/blob/71e64ea78fea5dc83c556ee76df2b5ea1df124e4/src/MOI/MOI_wrapper.jl#L2496-L2499
It seems like it reports the solution issue infeasible, and then when we query the infeasibility certificate, it can’t do it.
Can you check what termination_status(model) and raw_status_string(model) are?
julia> primal_status(m)
INFEASIBILITY_CERTIFICATE::ResultStatusCode = 4
julia> dual_status(m)
NO_SOLUTION::ResultStatusCode = 0
julia> termination_status(m)
DUAL_INFEASIBLE::TerminationStatusCode = 3
So when you said it did not reproduce on your machine, I tried a different computer I have and I did not get the error. Additionally, the termination status on that machine is
julia> termination_status(m)
OPTIMAL::TerminationStatusCode = 1
The CPLEX versions are the same. Why would the termination status be different?