MWE for directly calling libhighs (using HiGHS_jll)

Specific Domains Optimization (Mathematical)
1

Hi everyone, I want to directly call into libhighs in HiGHS_jll.jl, bypassing JuMP.jl and HiGHS.jl.

First was to come up with a simple toy MILP-problem, to test if it basically works. The idea is to minimize 0.3x + 0.5y, whereas both variables are binary (0 or 1) and the constraint x + y = 1 has to hold.

Loading the library and getting its version works fine:

using HiGHS_jll

highs_ver = @ccall libhighs.Highs_version()::Cstring
println("HiGHS version: $(unsafe_string(highs_ver))")

This prints

HiGHS version: 1.11.0

So, next I try to formulate and solve the toy problem, using Highs_mipCall for convenience:

HighsInt = Cint

col_value = zeros(2)
row_value = zeros(1)
model_status_ref = Ref{HighsInt}(0)

mipcall_ret = @ccall libhighs.Highs_mipCall(
    2::HighsInt, # num_col
    1::HighsInt, # num_row
    2::HighsInt, # num_nz
    1::HighsInt, # a_format = kHighsMatrixFormatColwise = 1
    1::HighsInt, # sense = kHighsObjSenseMinimize = 1
    0.0::Cdouble, # offset
    [0.2, 0.5]::Ptr{Cdouble}, # col_cost
    [-Inf, -Inf]::Ptr{Cdouble}, # col_lower
    [Inf, Inf]::Ptr{Cdouble}, # col_upper
    [1.0]::Ptr{Cdouble}, # row_lower
    [1.0]::Ptr{Cdouble}, # row_upper
    [1,2,3]::Ptr{HighsInt}, # a_start
    [1, 1]::Ptr{HighsInt}, # a_index
    [1.0, 1.0]::Ptr{Cdouble}, # a_value
    [1, 1]::Ptr{HighsInt}, # integrality, kHighsVarTypeInteger = 1
    col_value::Ptr{Cdouble}, # output: col_value
    row_value::Ptr{Cdouble}, # output: row_value
    model_status_ref::Ptr{HighsInt} # output: model_status
)::HighsInt

I just get the return value -1, which indicates an error by HiGHS. But no other output to the console, so it’s really hard to debug anything.

Is the problem formulation wrong? (Although I checked on a slightly more complex problem, formulating it in JuMP as well, and then looking at lp_matrix_data(model), which looks exactly the same as my manual formulation.)

Is there something wrong about my @ccall, the arguments, how I hand over the pointers to be filled by the call (last three arguments)?

BTW, there is no difference if I explicitly set col_lower to [0, 0] and col_upper to [1, 1].

Any pointers on what I’m doing wrong very welcome :wink:.

2

I would use HiGHS.jl (or even JuMP.jl), formulate the MILP and then use the debugger to reach the actual HiGHS ccall site and check what is the correct way to formulate the problem.

3

Hi @asprionj,

First: I strongly encourage you to use HiGHS.jl instead, and preferably via JuMP, but the MOI interface also works. Using the C API requires care.

Here’s code that works:

julia> import HiGHS_jll: libhighs

julia> HighsInt = Cint
Int32

julia> col_value = zeros(2)
2-element Vector{Float64}:
 0.0
 0.0

julia> row_value = zeros(1)
1-element Vector{Float64}:
 0.0

julia> p_model_status = Ref{HighsInt}(0)
Base.RefValue{Int32}(0)

julia> ret = @ccall libhighs.Highs_mipCall(
           2::HighsInt,                   # num_col
           1::HighsInt,                   # num_row
           2::HighsInt,                   # num_nz
           1::HighsInt,                   # a_format = kHighsMatrixFormatColwise = 1
           1::HighsInt,                   # sense = kHighsObjSenseMinimize = 1
           0.0::Cdouble,                  # offset
           [0.2, 0.5]::Ptr{Cdouble},      # col_cost
           [0.0, 0.0]::Ptr{Cdouble},      # col_lower
           [Inf, Inf]::Ptr{Cdouble},      # col_upper
           [1.0]::Ptr{Cdouble},           # row_lower
           [1.0]::Ptr{Cdouble},           # row_upper
           HighsInt[0, 1]::Ptr{HighsInt}, # a_start
           HighsInt[0, 0]::Ptr{HighsInt}, # a_index
           [1.0, 1.0]::Ptr{Cdouble},      # a_value
           HighsInt[1, 1]::Ptr{HighsInt}, # integrality, kHighsVarTypeInteger = 1
           col_value::Ptr{Cdouble},       # output: col_value
           row_value::Ptr{Cdouble},       # output: row_value
           p_model_status::Ptr{HighsInt}  # output: model_status
       )::HighsInt
0

julia> col_value
2-element Vector{Float64}:
 1.0
 0.0

julia> row_value
1-element Vector{Float64}:
 1.0

julia> p_model_status
Base.RefValue{Int32}(7)

The main change is:

HighsInt[0, 1]::Ptr{HighsInt}, # a_start
HighsInt[0, 0]::Ptr{HighsInt}, # a_index

Differences are:

  • Rows and columns are 0-indexed, not 1-indexed. So the first column is index 0.
  • You need to ensure that the HighsInt vectors are of type HighsInt and not Int. Your a_start is a Vector{Int64}, which gets converted to a Ptr{Int32}, but not in the way you think; instead of converting the element values, we just re-interpret the pointer. So it actually gets passed to HiGHS as the equivalent of Int32[0, 1, 0, 2, 0, 3].
2 Likes
4

Thanks a lot! Had the feeling it would be such details. Did actually not know about the exact workings of @ccall, i.e. how it converts stuff. In my “original” example I anyway generate the data in the correct types beforehand and not define the problem “in-line” in the HiGHS-call.

The 0-based indexing I just forgot about :see_no_evil_monkey:.

The thing is that already with HiGHS.jl, many recursive dependencies are introduced, preventing --trim for binary generation, thus leading to a huge package and no “guarantee” that nothing is still JIT-compiled (or at least disptached/decided) dynamically at runtime. The first is “not so nice”, the second is a killer for my application.

Edit: BTW, thank you for all the work you’re putting into JuMP and the entire optimisation ecosystem, that’s just awesome! I think I might follow a pattern where I prototype / experiment using JuMP, then once I approach a final implementation, I look at lp_matrix_data to quickly guide me to a “manual” implementation. Having a look at the HiGHS-calls in the debugger probably is less “enlightening”, since you seem to use the “incremental” way of building up a model, not the one-off Highs_*Call calls…

2 Likes
5

Improving this is not on my priority list while it remains an experimental feature in an unreleased version of Julia.

Once it stabilises though, I expect that there will be persistent demand that forces me to take a look. It seems reasonable that the MOI interface could be made completely inferrable. But JuMP likely never so because of some design choices.