# Efficient way to get the JuMP.value of all variables in a JuMP.Model?

**URL:** <https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539>\
**Category:** Optimization (Mathematical)\
**Tags:** jump\
**Created:** [October 10, 2022, 10:48pm UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539 "2022-10-10T22:48:17Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![nlaws](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nlaws/32/22205_2.png) [@nlaws](https://discourse.julialang.org/u/nlaws)\
**Post date:** [October 10, 2022, 10:48pm UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539/1 "2022-10-10T22:48:17Z")

</div>

I wrote a method to get all the variable values in a JuMP Model:

```julia

function get_results_dict(m::JuMP.Model)
    av = JuMP.all_variables(m);
    d = Dict()
    for v in av
        # special handling of time series, sub-indices, etc.
        # e.g. a variable could have three sub-indices in a JuMP.Containers
        v = string(v) # e.g. "x[a,2,3]"
        k = Symbol(v[1:prevind(v, findfirst('[', v))[1]]) # :x
        vm = value.(m[k])
        d[k] = vm.data
        # e.g. typeof(vm) is JuMP.Containers.SparseAxisArray{Float64, 3, Tuple{SubString{String}, Int64, Int64}}
    end
    return d
end

```

However, this is extremely slow because

```julia
julia> length(av)
1349040

```

Is there more efficient way to get all the variable values? Other than only getting some variable values? And if the latter is the only solution, what is the most efficient way to get a group of variable values with different indexing structures in `JuMP.Containers`? (Say if I had a vector of variable names that is a subset of `JuMP.all_variables`).

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 10, 2022, 11:04pm UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539/2 "2022-10-10T23:04:30Z")

</div>

Do you just want to get the variable values that are registered in `object_dictionary(model)`?

So something like the following:

```julia
d = Dict(
    k => value.(v) for 
    (k, v) in object_dictionary(model) if v isa AbstractArray{VariableRef}
)

```

You approach is very slow because it doesn’t short-circuit if the key is already found. Do something like:

```julia
function get_results_dict(m::JuMP.Model)
    av = JuMP.all_variables(m);
    d = Dict()
    for v in av
        # special handling of time series, sub-indices, etc.
        # e.g. a variable could have three sub-indices in a JuMP.Containers
        v = string(v) # e.g. "x[a,2,3]"
        k = Symbol(v[1:prevind(v, findfirst('[', v))[1]]) # :x
        if !haskey(d, k)
            vm = value.(m[k])
            d[k] = vm.data
        end
        # e.g. typeof(vm) is JuMP.Containers.SparseAxisArray{Float64, 3, Tuple{SubString{String}, Int64, Int64}}
    end
    return d
end

```

---

<div class="post-metadata">

**Author:** ![nlaws](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nlaws/32/22205_2.png) [@nlaws](https://discourse.julialang.org/u/nlaws)\
**Post date:** [October 11, 2022, 12:04am UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539/3 "2022-10-11T00:04:48Z")

</div>

I do have the short-circuit handling that you mentioned, specifically:

```julia
if k ∉ keys(d)

```

in place of your

```julia
if !haskey(d,k)

```

Maybe `!haskey` is much better than `∉`?

Nonetheless, the `object_dictionary` method looks promising. But I’d like to handle special sub-indexing. For example:

```julia
julia> d = Dict(
           k => value.(v) for 
           (k, v) in object_dictionary(model) if v isa AbstractArray{VariableRef}
       )
Dict{Symbol, JuMP.Containers.SparseAxisArray{Float64, 3, K} where K<:Tuple{Any, Any, Any}} with 5 entries:
  :Qⱼ => [632, 2, 4216] = 0.0…
  :Pᵢⱼ => [632-645, 3, 367] = 0.0…
  :vsqrd => [632, 2, 4216] = 0.997919…
  :Pⱼ => [632, 2, 4216] = 0.0…
  :Qᵢⱼ => [632-645, 3, 367] = 0.0…

julia> d[:Qⱼ]
JuMP.Containers.SparseAxisArray{Float64, 3, Tuple{SubString{String}, Int64, Int64}} with 280320 entries:
  [611, 3, 2821] = -0.0013604
  [632, 1, 2220] = 0.0
  [632, 1, 7419] = 0.0
  [632, 2, 3126] = 0.0
...

```

I’d like to access the sub-indices of `:Qⱼ` like ` d[:Qⱼ]["611"][3][2821]` (the indices are bus-name, phase, and time step in a power flow model). Is there something that I can do efficiently based on the typeof a value in `object_dictionary`, e.g. `SparseAxisArray{Float64, 3, Tuple{SubString{String}, Int64, Int64}}`?

(Edit: I’d like this indexing because then I can use the attributes of any given dimension for post-processing; sort of going from flat-file style single key-value pairs to sub-dictionary style indexing).

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 11, 2022, 12:14am UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539/4 "2022-10-11T00:14:23Z")

</div>

> I do have the short-circuit handling that you mentioned

I don’t see it in your example?

> Nonetheless, the `object_dictionary` method looks promising

I think what you really need is a proper data structure to handle the variables. Don’t dump everything into a JuMP model and then try to rediscover the structure after the fact.

> But I’d like to handle special sub-indexing … like `d[:Qⱼ]["611"][3][2821]`

Then `SparseAxisArray` is not the type for you. If you want nested dictionaries, you’ll have to write that conversion yourself.

I will say that recent JuMP version support slicing like `d[:Qⱼ]["611", :, :]` but that creates a new dictionary each time which might be expensive.

---

<div class="post-metadata">

**Author:** ![nlaws](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nlaws/32/22205_2.png) [@nlaws](https://discourse.julialang.org/u/nlaws)\
**Post date:** [October 11, 2022, 12:31am UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539/5 "2022-10-11T00:31:30Z")

</div>

> If you want nested dictionaries, you’ll have to write that conversion yourself.

That is what I have done, and it’s really slow. So I’m asking how I do that efficiently.

So is the lesson to not use

```julia
@variable(model, x[dims1, dims2, dims3] >= 0)

```

where the `dims1/2/3` are vectors of indices?

Maybe I should create dictionaries of variables and use the keys of those dictionaries to get the `value`s? (But doesn’t that take away the efficiency of the `object_dictionary`?)

---

<div class="post-metadata">

**Author:** ![nlaws](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nlaws/32/22205_2.png) [@nlaws](https://discourse.julialang.org/u/nlaws)\
**Post date:** [October 11, 2022, 12:42am UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539/6 "2022-10-11T00:42:45Z")

</div>

> [@odow](#):
>
> I will say that recent JuMP version support slicing like `d[:Qⱼ]["611", :, :]` but that creates a new dictionary each time which might be expensive.

That’s great! This capability is one of the use-cases that I am trying to accomplish with my variable values sub-dictionaries, For example, what I want is

```julia
d[:Qⱼ]["611"][1] .+ d[:Qⱼ]["611"][2]

```

but now I can use

```julia
d[:Qⱼ]["611", 1, :] .+ d[:Qⱼ]["611", 2, :]

```

Is this capability released or do I need to `dev` JuMP to get it?

---

<div class="post-metadata">

**Author:** ![nlaws](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nlaws/32/22205_2.png) [@nlaws](https://discourse.julialang.org/u/nlaws)\
**Post date:** [October 11, 2022, 12:55am UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539/7 "2022-10-11T00:55:14Z")

</div>

> [@odow](#):
>
> ```julia
> d = Dict(
> k => value.(v) for 
> (k, v) in object_dictionary(model) if v isa AbstractArray{VariableRef}
> )
> 
> ```

Is the solution. There are some nuances to how you define your variables though. If you do not want to use `JuMP.Containers` in the `Dict` values then you need to define variables in your own containers.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [October 11, 2022, 1:01am UTC](https://discourse.julialang.org/t/efficient-way-to-get-the-jump-value-of-all-variables-in-a-jump-model/88539/8 "2022-10-11T01:01:01Z")

</div>

> So is the lesson to not use
> 
> ```julia
> @variable(model, x[dims1, dims2, dims3] >= 0)
> 
> ```
> 
> where the `dims1/2/3` are vectors of indices?

No, I don’t think that’s the lesson. I think the lesson is to choose an appropriate data structure for what you want to achieve. JuMP provides a few defaults, but they aren’t always the best choice:

> **[Variables · JuMP](https://jump.dev/JuMP.jl/stable/manual/variables/#User-defined-containers)**
>
> Documentation for JuMP.

In your case, you wanted `x[i][j][k]`, which none of the JuMP containers support. So don’t try to force a JuMP container to meet your requirements; choose a different data structure up front.

> do I need to `dev` JuMP to get it?

It was released in JuMP v1.3.0: [Release notes · JuMP](https://jump.dev/JuMP.jl/stable/release_notes/#Version-1.3.0-(September-5,-2022))

> If you do not want to use `JuMP.Containers` in the `Dict` values then you need to define variables in your own containers.

Sure. You could write a different function to `value.(v)` that handled things appropriately.