Updating to MosekToolsv0.15.3 breaks many JuMP SDP codes

I updated MosekTools to v0.15.3 today (from 0.15.1) and it breaks my existing SDP codes with the following error when calling optimize!:

MSK_RES_ERR_SYM_MAT_INVALID_ROW_INDEX): The row index 11 specified for sparse symmetric is invalid. The valid range is [0,10[

or another error:

MOSEK error 3944 (MSK_RES_ERR_SYM_MAT_DUPLICATE): The value (4,4) in a sparse symmetric has been specified twice at position 0 and position 3.

(The numbers 11, (4,4), 0, 3 etc change from instance to instance)

No idea what is going on, it also breaks so many of my previous SDP codes as well (some of them were written in 2022-23 and ran without any issue).

After downgrading MosekTools to 0.15.1 my previous code works without any problem. I am using macOS and the latest version of JuMP 1.23.5 and Julia (1.11.2+0.aarch64.apple.darwin14) and here are my currently installed packages (only MosekTools is downgraded, rest are updated):

  [6e4b80f9] BenchmarkTools v1.5.0
  [31c24e10] Distributions v0.25.114
  [2e9cd046] Gurobi v1.6.0
  [b6b21f68] Ipopt v1.7.0
  [033835bb] JLD2 v0.5.10
  [4076af6c] JuMP v1.23.5
  [67920dd8] KNITRO v0.14.4
  [6405355b] Mosek v10.2.0
⌃ [1ec41992] MosekTools v0.15.1
  [6fe1bfb0] OffsetArrays v1.14.2
  [bac558e1] OrderedCollections v1.7.0
  [37e2e46d] LinearAlgebra v1.11.0

I was able to reproduce it on a Windows computer as well. Not sure if this is useful: I am using OffsetArrays in creating some of the data matrices for the SDPs in consideration and maybe the newly added VariableBasisStatus since MosekTools 0.15.2 is somehow responsible for displaying this error. Any help is much appreciated.

Please open an issue with a reproducible example. It looks like @blegat made some changes that seem related: Comparing v0.15.1...v0.15.3 · jump-dev/MosekTools.jl · GitHub

Thanks for your response @odow! Here is a reproducible example along with output with both MosekTools 0.15.3 and MosekTools 0.15.1. I have opened an issue on Github as well. Any tips to fix this will be much appreciated (for now I am just using 0.15.1) because this ends up breaking many of my previous SDP codes in existing research projects.

## Load the packages
# -------------------------

using JuMP, MosekTools, Mosek, LinearAlgebra, OffsetArrays

## Data matrices in OffsetArray format 
# ------------------------------------------------

A = [0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0; 0.0 0.0 0.0 -0.0 0.0 0.0 0.0 0.0 -0.0 -0.5 0.0 -0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 -0.0 0.0 0.0 0.0 0.0 -0.0 -0.5 0.0 -0.0 0.0; 0.0 0.0 0.5 -0.5 0.0 0.0 0.0 0.0 -0.5 0.0 0.0 -0.5 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 -0.0 0.0 0.0 0.0 0.0 -0.0 -0.5 0.0 -0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0];

A_Offset = OffsetArray(A, 1:13, 1:13)

B = [0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0; 0.0 0.0 0.0 -0.0 -0.453602203155747 0.0 0.0 0.0; 0.0 0.0 0.5 -0.453602203155747 0.0 0.0 0.0 0.453602; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0; 0.0 0.0 0.0 0.0 0.453602 0.0 0.0 0.0];

B_Offset = OffsetArray(B, 1:8, 1:8)

## JuMP model
# -----------------

model_test = Model(Mosek.Optimizer)

@variable(model_test, X_var[1:13, 1:13], PSD)

@variable(model_test, Y_var[1:8, 1:8], PSD)

@objective(model_test, Min, tr(X_var))

@constraint(model_test,tr(X_var * A_Offset) - tr(Y_var * B_Offset) == 0) # This constraint causes the issue

@constraint(model_test, tr(X_var) <= 1)

optimize!(model_test)

OUTPUT with MosekTools v0.15.3:

(Please see output with MosekTools v0.15.1 below, which runs fine)

MOSEK error 3940 (MSK_RES_ERR_SYM_MAT_INVALID_ROW_INDEX): The row index 9 specified for sparse symmetric is invalid. The valid range is [0,8[.
MOSEK error 3940 (MSK_RES_ERR_SYM_MAT_INVALID_ROW_INDEX): The row index 9 specified for sparse symmetric is invalid. The valid range is [0,8[.
MOSEK error 3940 (MSK_RES_ERR_SYM_MAT_INVALID_ROW_INDEX): The row index 9 specified for sparse symmetric is invalid. The valid range is [0,8[.
MOSEK error 3940 (MSK_RES_ERR_SYM_MAT_INVALID_ROW_INDEX): The row index 11 specified for sparse symmetric is invalid. The valid range is [0,8[.
ERROR: MosekError(3940, "The row index 11 specified for sparse symmetric is invalid. The valid range is [0,8[.\n")
Stacktrace:
  [1] appendsparsesymmat(task::Mosek.Task, dim::Int32, subi::Vector{Int32}, subj::Vector{Int32}, valij::Vector{Float64})
    @ Mosek ~/.julia/packages/Mosek/cMwUF/src/msk_functions.jl:3707
  [2] appendsparsesymmat(task::Mosek.Task, dim::Int64, subi::Vector{Int32}, subj::Vector{Int32}, valij::Vector{Float64})
    @ Mosek ~/.julia/packages/Mosek/cMwUF/src/msk_functions.jl:16628
  [3] (::MosekTools.var"#add_sd#24"{MosekTools.Optimizer, MosekTools.var"#25#26"{…}, Vector{…}, Vector{…}, Vector{…}})()
    @ MosekTools ~/.julia/packages/MosekTools/uICAS/src/constraint.jl:69
  [4] split_scalar_matrix(m::MosekTools.Optimizer, terms::Vector{…}, set_sd::MosekTools.var"#25#26"{…})
    @ MosekTools ~/.julia/packages/MosekTools/uICAS/src/constraint.jl:93
  [5] set_row
    @ ~/.julia/packages/MosekTools/uICAS/src/constraint.jl:103 [inlined]
  [6] add_constraint(m::MosekTools.Optimizer, axb::MathOptInterface.ScalarAffineFunction{…}, dom::MathOptInterface.EqualTo{…})
    @ MosekTools ~/.julia/packages/MosekTools/uICAS/src/constraint.jl:437
  [7] _copy_constraints(dest::MosekTools.Optimizer, src::MathOptInterface.Utilities.UniversalFallback{…}, index_map::MathOptInterface.Utilities.IndexMap, index_map_FS::MathOptInterface.Utilities.DoubleDicts.IndexDoubleDictInner{…}, cis_src::Vector{…})
    @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/gLl4d/src/Utilities/copy.jl:180
  [8] _copy_constraints(dest::MosekTools.Optimizer, src::MathOptInterface.Utilities.UniversalFallback{…}, index_map::MathOptInterface.Utilities.IndexMap, cis_src::Vector{…})
    @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/gLl4d/src/Utilities/copy.jl:192
  [9] pass_nonvariable_constraints_fallback(dest::MosekTools.Optimizer, src::MathOptInterface.Utilities.UniversalFallback{…}, index_map::MathOptInterface.Utilities.IndexMap, constraint_types::Vector{…})
    @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/gLl4d/src/Utilities/copy.jl:203
 [10] pass_nonvariable_constraints
    @ ~/.julia/packages/MathOptInterface/gLl4d/src/Utilities/copy.jl:229 [inlined]
 [11] pass_nonvariable_constraints(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{…}, src::MathOptInterface.Utilities.UniversalFallback{…}, idxmap::MathOptInterface.Utilities.IndexMap, constraint_types::Vector{…})
    @ MathOptInterface.Bridges ~/.julia/packages/MathOptInterface/gLl4d/src/Bridges/bridge_optimizer.jl:426
 [12] _pass_constraints(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{…}, src::MathOptInterface.Utilities.UniversalFallback{…}, index_map::MathOptInterface.Utilities.IndexMap, variable_constraints_not_added::Vector{…})
    @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/gLl4d/src/Utilities/copy.jl:251
 [13] default_copy_to(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{…}, src::MathOptInterface.Utilities.UniversalFallback{…})
    @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/gLl4d/src/Utilities/copy.jl:393
 [14] copy_to
    @ ~/.julia/packages/MathOptInterface/gLl4d/src/Bridges/bridge_optimizer.jl:442 [inlined]
 [15] optimize!
    @ ~/.julia/packages/MathOptInterface/gLl4d/src/MathOptInterface.jl:121 [inlined]
 [16] optimize!(m::MathOptInterface.Utilities.CachingOptimizer{…})
    @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/gLl4d/src/Utilities/cachingoptimizer.jl:321
 [17] optimize!(model::Model; ignore_optimize_hook::Bool, _differentiation_backend::MathOptInterface.Nonlinear.SparseReverseMode, kwargs::@Kwargs{})
    @ JuMP ~/.julia/packages/JuMP/i68GU/src/optimizer_interface.jl:595
 [18] optimize!(model::Model)
    @ JuMP ~/.julia/packages/JuMP/i68GU/src/optimizer_interface.jl:546
 [19] top-level scope
    @ ~/Library/MyDrive/junk_3.jl:35

Output with MosekTools v0.15.1:

Problem
  Name                   :                 
  Objective sense        : minimize        
  Type                   : CONIC (conic optimization problem)
  Constraints            : 2               
  Affine conic cons.     : 0               
  Disjunctive cons.      : 0               
  Cones                  : 0               
  Scalar variables       : 0               
  Matrix variables       : 2 (scalarized: 127)
  Integer variables      : 0               

Optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator started.
Freed constraints in eliminator : 0
Eliminator terminated.
Eliminator started.
Freed constraints in eliminator : 0
Eliminator terminated.
Eliminator - tries                  : 2                 time                   : 0.00            
Lin. dep.  - tries                  : 1                 time                   : 0.00            
Lin. dep.  - primal attempts        : 1                 successes              : 1               
Lin. dep.  - dual attempts          : 0                 successes              : 0               
Lin. dep.  - primal deps.           : 0                 dual deps.             : 0               
Presolve terminated. Time: 0.00    
Optimizer  - threads                : 20              
Optimizer  - solved problem         : the primal      
Optimizer  - Constraints            : 2               
Optimizer  - Cones                  : 0               
Optimizer  - Scalar variables       : 1                 conic                  : 0               
Optimizer  - Semi-definite variables: 2                 scalarized             : 127             
Factor     - setup time             : 0.00            
Factor     - dense det. time        : 0.00              GP order time          : 0.00            
Factor     - nonzeros before factor : 3                 after factor           : 3               
Factor     - dense dim.             : 0                 flops                  : 1.20e+03        
ITE PFEAS    DFEAS    GFEAS    PRSTATUS   POBJ              DOBJ              MU       TIME  
0   1.2e+01  1.0e+00  1.4e+01  0.00e+00   1.300000000e+01   0.000000000e+00   1.0e+00  0.00  
1   0.0e+00  0.0e+00  0.0e+00  1.00e+00   0.000000000e+00   0.000000000e+00   0.0e+00  0.01  
Optimizer terminated. Time: 0.01

There’s no work-around. This just looks like a bug in Improve performance of split_scalar_matrix by blegat · Pull Request #136 · jump-dev/MosekTools.jl · GitHub

1 Like

@Shuvomoy_Das_Gupta Thanks for reporting it ! It should be fixed in v0.15.4 of MosekTools

2 Likes

Great, thanks @blegat and @odow !