# Multiplying by a variable in a constraint causing type instability in @code\_warntype

**URL:** <https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239>\
**Category:** New to Julia\
**Created:** [December 13, 2019, 6:29pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239 "2019-12-13T18:29:47Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![stop\_movn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stop_movn/32/11828_2.png) [@stop\_movn](https://discourse.julialang.org/u/stop_movn)\
**Post date:** [December 13, 2019, 6:29pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/1 "2019-12-13T18:29:47Z")

</div>

I am trying to figure out how to avoid type instability in my model that uses some constraints and objectives that include a variable. I can’t seem to avoid a “::Union{Val{false}, GenericAffExpr{Float64,VariableRef}}” even if I assert a type or use an expression. Am I missing an easy way to avoid this type instability?

Here is a simplified version of my code that should simply be returning the rows of the input array with the three highest values in the second column:

_using GLPKMathProgInterface_  
_using JuMP_  
_using GLPK_  
_using BenchmarkTools_

_function test\_warntype(values, num\_values)_

_m = Model(with\_optimizer(GLPK.Optimizer))_

_@variable(m, used\_values[i=1:num\_values], Bin)_

_@constraint(m, sum(used\_values[i] for i in 1:num\_values) == 3)_

_@objective(m, Max, sum(values[i,2]\*used\_values[i] for i in 1:num\_values))_

_optimize!(m)_

_return(used\_values)_

_end_

_values = [1.5 4.6_  
_2.6 7.3_  
_1.1 3_  
_3.7 0.3_  
_4.4 6]_

_num\_values = size(values)[2]_

_@code\_warntype test\_warntype(values, num\_values)_

And here is the relevant part of the code\_warntype output:

_Variables_  
_#self#::Core.Compiler.Const(test\_warntype, false)_  
_values::Array{Float64,2}_  
_num\_values::Int64_  
_m::Model_  
_##76793::Array{VariableRef,1}_  
_used\_values::Array{VariableRef,1}_  
_@\_7::Union{Nothing, Tuple{Int64,Int64}}_  
_i@\_8::Int64_  
_q@\_9::Union{Val{false}, GenericAffExpr{Float64,VariableRef}, VariableRef}_  
_@\_10::Union{Nothing, Tuple{Int64,Int64}}_  
_##76798::Union{Val{false}, GenericAffExpr{Float64,VariableRef}, VariableRef}_  
_##76797::Union{Float64, GenericAffExpr{Float64,VariableRef}}_  
_##76796::Union{Float64, GenericAffExpr{Float64,VariableRef}}_  
_##76795::ConstraintRef{Model,\_A,ScalarShape} where \_A_  
_i@\_15::Int64_  
_q@\_16::Union{Val{false}, GenericAffExpr{Float64,VariableRef}}_  
_@\_17::Union{Nothing, Tuple{Int64,Int64}}_  
_##76799::Union{Val{false}, GenericAffExpr{Float64,VariableRef}}_  
_i@\_19::Int64_

_Body::Array{VariableRef,1}_  
_…_  
_…_  
_5 ┄─ %52 = @\_10::Tuple{Int64,Int64}::Tuple{Int64,Int64}_  
_│ (i@\_15 = Core.getfield(%52, 1))_  
_│ %54 = Core.getfield(%52, 2)::Int64_  
_│ %55 = q@\_9::Union{Val{false}, GenericAffExpr{Float64,VariableRef}, VariableRef}_  
_│ %56 = Base.getindex(used\_values, i@\_15)::VariableRef_  
_│ (q@\_9 = JuMP.\_destructive\_add\_with\_reorder!(%55, %56))_  
_│ (@\_10 = Base.iterate(%47, %54))_  
_│ %59 = (@\_10 === nothing)::Bool_  
_│ %60 = Base.not\_int(%59)::Bool_  
_└─── goto #7 if not %60_  
_6 ── goto #5_  
_7 ┄─ (##76798 = q@\_9)_  
_│ (##76797 = JuMP.\_destructive\_add\_with\_reorder!(##76798, -1.0, 3))_  
_│ (##76796 = JuMP.\_destructive\_add\_with\_reorder!(##76797, 0))_  
_│ %66 = m::Model_  
_│ %67 = JuMP.build\_constraint(getfield(JuMP, Symbol(“#\_error#55”)){Symbol}(Core.Box(Any[:m, :(sum((used\_values[i] for i = 1:num\_values)) == 3)]), :constraint), ##76796, MathOptInterface.EqualTo{Float64}(0.0))::ScalarConstraint{GenericAffExpr{Float64,VariableRef},MathOptInterface.EqualTo{Float64}}_  
_│ (##76795 = JuMP.add\_constraint(%66, %67, “”))_  
_│ ##76795_  
_│ JuMP.\_valid\_model(m, :m)_  
_│ Core.NewvarNode(:(##76799))_  
_│ %72 = Core.apply\_type(JuMP.Val, false)::Core.Compiler.Const(Val{false}, false)_  
_│ (q@\_16 = (%72)())_  
_│ %74 = (1:num\_values)::Core.Compiler.PartialStruct(UnitRange{Int64}, Any[Core.Compiler.Const(1, false), Int64])_  
_│ (@\_17 = Base.iterate(%74))_  
_│ %76 = (@\_17 === nothing)::Bool_  
_│ %77 = Base.not\_int(%76)::Bool_  
_└─── goto #10 if not %77_  
_8 ┄─ %79 = @\_17::Tuple{Int64,Int64}::Tuple{Int64,Int64}_  
_│ (i@\_19 = Core.getfield(%79, 1))_  
_│ %81 = Core.getfield(%79, 2)::Int64_  
_│ %82 = q@\_16::Union{Val{false}, GenericAffExpr{Float64,VariableRef}}_  
_│ %83 = Base.getindex(values, i@\_19, 2)::Float64_  
_│ %84 = Base.getindex(used\_values, i@\_19)::VariableRef_  
_│ (q@\_16 = JuMP.\_destructive\_add\_with\_reorder!(%82, %83, %84))_  
_│ (@\_17 = Base.iterate(%74, %81))_  
_│ %87 = (@\_17 === nothing)::Bool_  
_│ %88 = Base.not\_int(%87)::Bool_  
_└─── goto #10 if not %88_  
_9 ── goto #8_  
_10 ┄ (##76799 = q@\_16)_  
_│ %92 = m::Model_  
_│ %93 = JuMP.\_throw\_error\_for\_invalid\_sense(getfield(JuMP, Symbol(“#\_error#57”)){Symbol,Tuple{Symbol,Expr}}(:m, (:Max, :(sum(($(Expr(:escape, :(values[i, 2]))) \* $(Expr(:escape, :(used\_values[i]))) for i = 1:num\_values))))), MathOptInterface.MAX\_SENSE)::Core.Compiler.Const(MathOptInterface.MAX\_SENSE, false)_  
_│ JuMP.set\_objective(%92, %93, ##76799)_  
_│ ##76799_  
_│ Main.optimize!(m)_  
_└─── return used\_values_

Edited for formatting

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [December 13, 2019, 9:25pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/2 "2019-12-13T21:25:38Z")

</div>

Is this a bottleneck in your code? Or are you just trying to eliminate all type instabilities?

Here’s a simpler reproducer:

```julia
using JuMP
model = Model()
@variable(model, x)
test_warntype(model, x) = @expression(model, sum(x for i = 1:2))
@code_warntype test_warntype(model, x)

```

Digging in, the `@expression` expands to:

```julia
function foo(x)
    a = JuMP.Val{false}()
    for i = 1:2
        a = JuMP._destructive_add_with_reorder!(a, x)
    end
    return a
end
@code_warntype foo(x)

```

The issue is that `foo(x)` infers as `Union{GenericAffExpr{Float64,VariableRef}, VariableRef}`.

```nohighlight
julia> @code_warntype foo(x)
Body::Union{GenericAffExpr{Float64,VariableRef}, VariableRef}
1 ── goto #26 if not true
2 ┄─ %2 = φ (#1 => $(QuoteNode(Val{false}())), #25 => %57)::Union{Val{false}, GenericAffExpr{Float64,VariableRef}, VariableRef}
│ %3 = φ (#1 => 1, #25 => %63)::Int64
│ %4 = (isa)(%2, Val{false})::Bool
└─── goto #4 if not %4
3 ── %6 = (Base.getfield)(x, :model)::Model
│ %7 = (Base.getfield)(x, :index)::MathOptInterface.VariableIndex
│ %8 = %new(JuMP.VariableRef, %6, %7)::VariableRef
└─── goto #21
4 ── %10 = (isa)(%2, GenericAffExpr{Float64,VariableRef})::Bool
└─── goto #18 if not %10
5 ── %12 = π (%2, GenericAffExpr{Float64,VariableRef})
│ %13 = (Base.getfield)(%12, :terms)::OrderedCollections.OrderedDict{VariableRef,Float64}
│ %14 = (Base.getfield)(x, :model)::Model
│ %15 = (Base.getfield)(x, :model)::Model
│ %16 = (%14 === %15)::Bool
└─── goto #7 if not %16
6 ── %18 = (Base.getfield)(x, :index)::MathOptInterface.VariableIndex
│ %19 = (Base.getfield)(x, :index)::MathOptInterface.VariableIndex
│ %20 = (%18 === %19)::Bool
└─── goto #8
7 ── goto #8
8 ┄─ %23 = φ (#6 => %20, #7 => false)::Bool
│ %24 = (Base.not_int)(%23)::Bool
└─── goto #10 if not %24
9 ── %26 = invoke Base.print_to_string(_2::VariableRef, " is not a valid key for type "::Vararg{Any,N} where N, VariableRef)::String
│ %27 = %new(Core.ArgumentError, %26)::ArgumentError
│ (OrderedCollections.throw)(%27)
└─── $(Expr(:unreachable))
10 ┄ %30 = invoke OrderedCollections.ht_keyindex2(%13::OrderedCollections.OrderedDict{VariableRef,Float64}, _2::VariableRef)::Int64
│ %31 = (Base.slt_int)(0, %30)::Bool
└─── goto #12 if not %31
11 ─ %33 = (Base.getfield)(%13, :vals)::Array{Float64,1}
│ %34 = (Base.arrayref)(true, %33, %30)::Float64
└─── goto #13
12 ─ %36 = (Base.neg_int)(%30)::Int64
│ invoke OrderedCollections._setindex!(%13::OrderedCollections.OrderedDict{VariableRef,Float64}, 0.0::Float64, _2::VariableRef, %36::Int64)
└─── goto #13
13 ┄ %39 = φ (#11 => %34, #12 => 0.0)::Float64
│ %40 = (Base.add_float)(%39, 1.0)::Float64
│ invoke Base.setindex!(%13::OrderedCollections.OrderedDict{VariableRef,Float64}, %40::Float64, _2::VariableRef)
└─── goto #14
14 ─ goto #15
15 ─ goto #16
16 ─ goto #17
17 ─ goto #21
18 ─ %47 = (isa)(%2, VariableRef)::Bool
└─── goto #20 if not %47
19 ─ %49 = π (%2, VariableRef)
│ %50 = %new(Pair{VariableRef,Float64}, %49, 1.0)::Pair{VariableRef,Float64}
│ %51 = %new(Pair{VariableRef,Float64}, x, 1.0)::Pair{VariableRef,Float64}
│ %52 = invoke JuMP._new_ordered_dict(VariableRef::Type{VariableRef}, Float64::Type{Float64}, %50::Pair{VariableRef,Float64}, %51::Pair{VariableRef,Float64})::OrderedCollections.OrderedDict{VariableRef,Float64}
│ %53 = %new(GenericAffExpr{Float64,VariableRef}, 0.0, %52)::GenericAffExpr{Float64,VariableRef}
└─── goto #21
20 ─ (Core.throw)(ErrorException("fatal error in type inference (type bound)"))
└─── $(Expr(:unreachable))
21 ┄ %57 = φ (#3 => %8, #17 => %12, #19 => %53)::Union{GenericAffExpr{Float64,VariableRef}, VariableRef}
│ %58 = (%3 === 2)::Bool
└─── goto #23 if not %58
22 ─ goto #24
23 ─ %61 = (Base.add_int)(%3, 1)::Int64
└─── goto #24
24 ┄ %63 = φ (#23 => %61)::Int64
│ %64 = φ (#22 => true, #23 => false)::Bool
│ %65 = (Base.not_int)(%64)::Bool
└─── goto #26 if not %65
25 ─ goto #2
26 ┄ %68 = φ (#24 => %57, #1 => $(QuoteNode(Val{false}())))::Union{Val{false}, GenericAffExpr{Float64,VariableRef}, VariableRef}
│ %69 = π (%68, Union{GenericAffExpr{Float64,VariableRef}, VariableRef})
└─── return %69

```

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [December 13, 2019, 9:28pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/3 "2019-12-13T21:28:51Z")

</div>

What’s weird is that

```nohighlight
function bar(x)
    a = JuMP.Val{false}()
    a = JuMP._destructive_add_with_reorder!(a, x)
    a = JuMP._destructive_add_with_reorder!(a, x)
    return a
end
@code_warntype bar(x)

```

infers fine:

```nohighlight
Body::GenericAffExpr{Float64,VariableRef}
1 ─ %1 = (Base.getfield)(x, :model)::Model
│ %2 = (Base.getfield)(x, :index)::MathOptInterface.VariableIndex
│ %3 = %new(JuMP.VariableRef, %1, %2)::VariableRef
│ %4 = %new(Pair{VariableRef,Float64}, %3, 1.0)::Pair{VariableRef,Float64}
│ %5 = %new(Pair{VariableRef,Float64}, x, 1.0)::Pair{VariableRef,Float64}
│ %6 = invoke JuMP._new_ordered_dict(VariableRef::Type{VariableRef}, Float64::Type{Float64}, %4::Pair{VariableRef,Float64}, %5::Pair{VariableRef,Float64})::OrderedCollections.OrderedDict{VariableRef,Float64}
│ %7 = %new(GenericAffExpr{Float64,VariableRef}, 0.0, %6)::GenericAffExpr{Float64,VariableRef}
└── return %7

```

cc @blegat

---

<div class="post-metadata">

**Author:** ![stop\_movn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stop_movn/32/11828_2.png) [@stop\_movn](https://discourse.julialang.org/u/stop_movn)\
**Post date:** [December 13, 2019, 9:31pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/4 "2019-12-13T21:31:02Z")

</div>

Thanks for your help simplifying it further. I don’t know if this is a bottleneck or how to check exactly. I just know I have a more complicated function with multiple of these constraints that is running slower than I would like. I have already eliminated some other type instabilities without seeing much improvement in my benchmarking so I am trying to eliminate as many as I can.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [December 13, 2019, 9:58pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/5 "2019-12-13T21:58:43Z")

</div>

> I have already eliminated some other type instabilities without seeing much improvement in my benchmarking so I am trying to eliminate as many as I can.

Type-instabilities don’t automatically lead to slow code. It’s _really_ worth spending time figuring out what the problem is before you try to optimize the run time.

It’s hard to offer other advice without the full code.

---

<div class="post-metadata">

**Author:** ![stop\_movn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stop_movn/32/11828_2.png) [@stop\_movn](https://discourse.julialang.org/u/stop_movn)\
**Post date:** [December 14, 2019, 4:07am UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/6 "2019-12-14T04:07:13Z")

</div>

The full code is fairly long and complex and might not be easy to follow without a lot of context. The basic principle though is running a more complicated version of this function, saving its results to an array storing the solutions, and then feeding those solutions back in and running it again and using a constraint to make sure no two solutions use more than a specified number of the same rows from the “values” array. Basically we want 100 solutions (sets of rows from the values array) with a maximum overlap.

Is there a way to write the constraint/expression to sum without expanding to a loop of “JuMP.\_destructive\_add\_with\_reorder!” ? How did you find what that expression expands to? Is there an easier way to go about tracking what part of the code is slow? Thanks.

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [December 15, 2019, 9:04am UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/7 "2019-12-15T09:04:25Z")

</div>

> Is there a way to write the constraint/expression to sum without expanding

Use `sum(used_values)`. We expand it to a for when we see `sum(... for ...)`.  
We might use a functional approach instead of for loops to avoid issue with type instability: see [Replace for loops by functional approach · Issue #26 · jump-dev/MutableArithmetics.jl · GitHub](https://github.com/JuliaOpt/MutableArithmetics.jl/issues/26)

> How did you find what that expression expands to?

You can see how it expands to using `@macroexpand`. Julia v1.0 can handle small unions without much overhead (this is the reason they chose `findfirst` to return a `Union{Nothing, Int}`).

> Is there an easier way to go about tracking what part of the code is slow?

You can use [ProfileView](https://github.com/timholy/ProfileView.jl)

---

<div class="post-metadata">

**Author:** ![stop\_movn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stop_movn/32/11828_2.png) [@stop\_movn](https://discourse.julialang.org/u/stop_movn)\
**Post date:** [December 16, 2019, 8:48pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/8 "2019-12-16T20:48:04Z")

</div>

Interestingly, converting the constraints and objectives to use dot products instead of the for notation does get rid of the “::Union{Val{false}, GenericAffExpr{Float64,VariableRef}}” but it replaces some of them with ::Any (specifically those where I am multiplying by the variable) and in my actual code it increases the memory allocation by about 4 times. It has no noticeable impact on overall runtime. I also tried adding @views before the constraints that sum slices and it did not seem to have any impact either. This may be a case where that union isn’t so bad for real performance. The profileview for the actual code is hard to read but I don’t see a lot of red and none at the top so I think this could just be as good as it is going to get.

_#@constraint(m, sum(used\_values[i] for i in 1:num\_values) == 3)_  
_@constraint(m, sum(used\_values) == 3)_

_#@objective(m, Max, sum(values[i,2]\*used\_values[i] for i in 1:num\_values))_  
_@views @objective(m, Max, sum(values[:,2].\*used\_values))_

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [December 16, 2019, 10:11pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/9 "2019-12-16T22:11:03Z")

</div>

What about `dot(values[:, 2], used_values)` instead of `sum(values[:,2] .* used_values))` ?

---

<div class="post-metadata">

**Author:** ![stop\_movn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stop_movn/32/11828_2.png) [@stop\_movn](https://discourse.julialang.org/u/stop_movn)\
**Post date:** [December 16, 2019, 10:48pm UTC](https://discourse.julialang.org/t/multiplying-by-a-variable-in-a-constraint-causing-type-instability-in-code-warntype/32239/10 "2019-12-16T22:48:16Z")

</div>

That gives a similar result in runtime but increase the memory allocation a bit compared to the original.