Workarounds for splatting in index sets?

Specific Domains Optimization (Mathematical)
1

Hi! I ran into a small issue when writing a MILP model, namely the fact that the splatting operator cannot be used in the definition of an index set. What I’m trying to achieve is to define JuMP variables with dimensions specified by a vector. I understand this is probably not something that is going to be changed in the future versions of JuMP for technical reasons (I’m one of those people who knows just enough about macros to stay away from them), but I wonder if people here have come up with creative workarounds. As an example of what doesn’t work:

using JuMP
# I would like a JuMP variable array with dimensions 
# specified by the vector a, 2x3x4 in this case.
a=[1:2, 1:3, 1:4] 

model = Model()
@variable(model, x[a...] >= 0) # Throws an error

And my workaround, sufficient for my use case where the indices are simply of the form 1:k, does not work if you want more general index sets like [1:2, ["red","blue"]]:

using JuMP
a=[1:2, 1:3, 1:4]
model = Model()
x = Array{VariableRef}(undef, Tuple(length.(a)))
for index in CartesianIndices(x)
    x[index] = @variable(model, base_name="x[$(join(Tuple(index),','))]", lower_bound=0)
end
1 Like
2

Does this modification do what you’re looking for?

for (index,bn) in zip(CartesianIndices(x),Base.product(a...))
    x[index] = @variable(model, base_name="x[$(join(bn,','))]", lower_bound=0)
end
3

Hi @solliolli, welcome to the forum!

Your work-around is on the right track. A key thing to remember is that you can write your own data structures to use with JuMP; you don’t need to be limited to the syntax that JuMP provides.

In your case, I would do something like:

julia> using JuMP

julia> a = [1:2, ["red", "blue", "green"], 1:4]
3-element Vector{AbstractVector}:
 1:2
 ["red", "blue", "green"]
 1:4

julia> model = Model();

julia> @variable(model, x[a...] >= 0)
ERROR: LoadError: At REPL[32]:1: `@variable(model, x[a...] >= 0)`: cannot use splatting operator `...` in the definition of an index set.
Stacktrace:
 [1] error(::String, ::String)
   @ Base ./error.jl:44
 [2] (::JuMP.Containers.var"#error_fn#98"{String})(str::String)
   @ JuMP.Containers ~/.julia/dev/JuMP/src/Containers/macro.jl:331
 [3] build_ref_sets(error_fn::JuMP.Containers.var"#error_fn#98"{String}, expr::Expr)
   @ JuMP.Containers ~/.julia/dev/JuMP/src/Containers/macro.jl:345
 [4] parse_ref_sets(error_fn::JuMP.Containers.var"#error_fn#98"{String}, expr::Expr; invalid_index_variables::Vector{Symbol})
   @ JuMP.Containers ~/.julia/dev/JuMP/src/Containers/macro.jl:299
 [5] var"@variable"(__source__::LineNumberNode, __module__::Module, input_args::Vararg{Any})
   @ JuMP ~/.julia/dev/JuMP/src/macros/@variable.jl:194
in expression starting at REPL[32]:1

julia> function my_add_variable(model::JuMP.Model, a::Vector)
           x = @variable(model, [1:prod(length.(a))], lower_bound = 0)
           for (xi, ai) in zip(x, Iterators.product(a...))
               set_name(xi, replace("x[$ai]", "(" => "", ")" => ""))
           end
           return Containers.DenseAxisArray(reshape(x, length.(a)...), a...)
       end
my_add_variable (generic function with 2 methods)

julia> x = my_add_variable(model, a)
3-dimensional DenseAxisArray{VariableRef,3,...} with index sets:
    Dimension 1, 1:2
    Dimension 2, ["red", "blue", "green"]
    Dimension 3, 1:4
And data, a 2×3×4 Array{VariableRef, 3}:
[:, :, 1] =
 x[1, "red", 1]  x[1, "blue", 1]  x[1, "green", 1]
 x[2, "red", 1]  x[2, "blue", 1]  x[2, "green", 1]

[:, :, 2] =
 x[1, "red", 2]  x[1, "blue", 2]  x[1, "green", 2]
 x[2, "red", 2]  x[2, "blue", 2]  x[2, "green", 2]

[:, :, 3] =
 x[1, "red", 3]  x[1, "blue", 3]  x[1, "green", 3]
 x[2, "red", 3]  x[2, "blue", 3]  x[2, "green", 3]

[:, :, 4] =
 x[1, "red", 4]  x[1, "blue", 4]  x[1, "green", 4]
 x[2, "red", 4]  x[2, "blue", 4]  x[2, "green", 4]
4

If I define my function your_add_variable(), I get the following result, where the second index is not of type string.

julia> function your_add_variable(model::JuMP.Model, a::Vector)    
           x = Array{VariableRef}(undef, Tuple(length.(a)))        
           for (index,bn) in zip(CartesianIndices(x),Base.product(a...))
               x[index] = @variable(model, base_name="x[$(join(bn,','))]", lower_bound=0)
           end
       Containers.DenseAxisArray(x,a...)
       end
your_add_variable (generic function with 1 method)

julia> your_add_variable(model,a)
3-dimensional DenseAxisArray{VariableRef,3,...} with index sets:
    Dimension 1, 1:2
    Dimension 2, ["red", "blue", "green"]
    Dimension 3, 1:4
And data, a 2×3×4 Array{VariableRef, 3}:
[:, :, 1] =
 x[1,red,1]  x[1,blue,1]  x[1,green,1]
 x[2,red,1]  x[2,blue,1]  x[2,green,1]

[:, :, 2] =
 x[1,red,2]  x[1,blue,2]  x[1,green,2]
 x[2,red,2]  x[2,blue,2]  x[2,green,2]

[:, :, 3] =
 x[1,red,3]  x[1,blue,3]  x[1,green,3]
 x[2,red,3]  x[2,blue,3]  x[2,green,3]

[:, :, 4] =
 x[1,red,4]  x[1,blue,4]  x[1,green,4]
 x[2,red,4]  x[2,blue,4]  x[2,green,4]

What is the difference between my your_add_variable and your my_add_variable?

Are both fine?
If so, is one preferable to the other, for some reason?

1 Like
5

What is the difference between my your_add_variable and your my_add_variable?

Perhaps to state the obvious: my_add_variable creates a vector of variables and then reshapes into a matrix. your_add_variable creates an empty matrix and then fill one-by-one.

Are both fine?

Yes.

If so, is one preferable to the other, for some reason?

Nope. Choose whichever one you find the simplest to understand.

I like your join trick for the name. I didn’t think of that:

julia> function my_add_variable(model::JuMP.Model, a::Vector)
           x = @variable(model, [1:prod(length.(a))], lower_bound = 0)
           for (xi, ai) in zip(x, Iterators.product(a...))
               set_name(xi, "x[$(join(ai, ", "))]")
           end
           return Containers.DenseAxisArray(reshape(x, length.(a)...), a...)
       end
6

This is @solliolli’s idea.
I, on my own, added the use of the Base.product function to be used together with the Cartesian indices, to obtain the names

1 Like
7

Indeed :smile: I didn’t scroll up far enough

8

If I try to use the ones provided by your_add_variable() it doesn’t work

julia> yourx=your_add_variable(model,a)
3-dimensional DenseAxisArray{VariableRef,3,...} with index sets:        
    Dimension 1, 1:2
    Dimension 2, ["red", "blue", "green"]
    Dimension 3, 1:4
And data, a 2×3×4 Array{VariableRef, 3}:
[:, :, 1] =
 x[1,red,1]  x[1,blue,1]  x[1,green,1]
 x[2,red,1]  x[2,blue,1]  x[2,green,1]

[:, :, 2] =
 x[1,red,2]  x[1,blue,2]  x[1,green,2]
 x[2,red,2]  x[2,blue,2]  x[2,green,2]

[:, :, 3] =
 x[1,red,3]  x[1,blue,3]  x[1,green,3]
 x[2,red,3]  x[2,blue,3]  x[2,green,3]

[:, :, 4] =
 x[1,red,4]  x[1,blue,4]  x[1,green,4]
 x[2,red,4]  x[2,blue,4]  x[2,green,4]

julia> fix(yourx[1,red,3],3;force=true)
ERROR: UndefVarError: `red` not defined
Stacktrace:
9

It is "red" the string. red the variable doesn’t exist.

10

Thank you both, this is the nice and fruitful forum experience I expected!

This is indeed prettier and more functional, the reshape at the end is a nice trick that did not cross my mind.

1 Like
11

I’m not sure I fully understand what the situation is behind the scenes.
I understand that what the name_base function sets is only to facilitate “human reading”.

julia>  function my_add_variable(model::JuMP.Model, a::Vector)
                   x = @variable(model, [1:prod(length.(a))], lower_bound = 0)
                   for (xi, ai) in zip(x, Iterators.product(a...))      
                       set_name(xi,randstring('a':'z', 6))
                   end
                   return Containers.DenseAxisArray(reshape(x, length.(a)...), a...)
               end
my_add_variable (generic function with 1 method)

julia> x = my_add_variable(model, a)
3-dimensional DenseAxisArray{VariableRef,3,...} with index sets:
    Dimension 1, 1:2
    Dimension 2, ["red", "blue", "green"]
    Dimension 3, 1:4
And data, a 2×3×4 Array{VariableRef, 3}:
[:, :, 1] =
 frwiol  cduuyy  iinatx
 ppzwzl  qyyqbk  rgucfd

[:, :, 2] =
 qepntc  eumldd  jfpmeg
 tkmxzi  fxyavd  wptzrn

[:, :, 3] =
 ewuiqn  sfrkom  xqzsrq
 jexcux  wfmzjg  kffpse

[:, :, 4] =
 rpvjdr  hpdcgf  igsgib
 qxiqld  kcgyfc  kztysx



Perhaps the road could be "shortened" in this way
model = Model();
y=@variables(model, begin
           x[Base.product(a...)].>=0
    end) ;

ourx=Containers.DenseAxisArray(reshape(y[1].data, length.(a)...), a...)

or is it inappropriate to use internal aspects like this?

Can I think of the Containers.DenseAxisArray structure as a sort of multidimensional NamedTuple?

So would it make sense to do something like this?

julia> our_add_variable(a)=Containers.DenseAxisArray(collect(Base.product(a...)), a...)
our_add_variable (generic function with 1 method)

julia> our_add_variable(a)
3-dimensional DenseAxisArray{Tuple{Int64, String, Int64},3,...} with index sets:
    Dimension 1, 1:2
    Dimension 2, ["red", "blue", "green"]
    Dimension 3, 1:4
And data, a 2×3×4 Array{Tuple{Int64, String, Int64}, 3}:
[:, :, 1] =
 (1, "red", 1)  (1, "blue", 1)  (1, "green", 1)
 (2, "red", 1)  (2, "blue", 1)  (2, "green", 1)

[:, :, 2] =
 (1, "red", 2)  (1, "blue", 2)  (1, "green", 2)
 (2, "red", 2)  (2, "blue", 2)  (2, "green", 2)

[:, :, 3] =
 (1, "red", 3)  (1, "blue", 3)  (1, "green", 3)
 (2, "red", 3)  (2, "blue", 3)  (2, "green", 3)

[:, :, 4] =
 (1, "red", 4)  (1, "blue", 4)  (1, "green", 4)
 (2, "red", 4)  (2, "blue", 4)  (2, "green", 4)