# Help in making multi-dimensional equations a bit sexier

**URL:** <https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217>\
**Category:** General Usage\
**Tags:** question, metaprogramming\
**Created:** [October 3, 2017, 12:24pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217 "2017-10-03T12:24:04Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [October 3, 2017, 12:24pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/1 "2017-10-03T12:24:04Z")

</div>

I need to write some equations that loop over several dimensions, and I created my own function `getData(parName,dim1,dim2)` and `setData(value,dim1,dim2)` that, using anonymous functions, [I dynamically wrapped](https://stackoverflow.com/questions/46531884/dynamic-function-name-definition-in-julia-possible) to `parName(dim1,dim2)` and `parName!(value,dim1,dim2)`.

So I can now write my model as:

```julia
[par1!( par2(d1,d2)+par3(d1,d2) ,d1,d2,dfix3) for d1 in DIM1, d2 in DIM2]

```

where the first argument of `par1!()` function is the value that I want to store.

Now, it’s still not very inspiring, so I am trying to make this syntax a bit more easy to write (and to read/maintain).

Specifically, I am trying to write a macro that let’s write the equation as:

```julia
@meq par1!(d1 in DIM1, d2 in DIM2, dfix3) = par2(d1,d2)+par3(d1,d2)

```

That’s sexy!

While I have used macros in julia, it’s the first time I am trying to write one.

So far, I managed to write:

```julia
macro meq(eq)
    dump(eq)
    lhs_par = eq.args[1].args[1]
    rhs = eq.args[2]
    lhs_dims = eq.args[1].args[2:end]
    loop_counters = [d.args[2] for d in lhs_dims if typeof(d) == Expr]
    loop_sets = [d.args[3] for d in lhs_dims if typeof(d) == Expr]
    loop_wholeElements = [d for d in lhs_dims if typeof(d) == Expr]
    lhs_dims_placeholders = []
    for d in lhs_dims
        if typeof(d) == Expr
            push!(lhs_dims_placeholders,d.args[2])
        else
            push!(lhs_dims_placeholders,d)
        end
    end
    outExp = quote
      [$(lhs_par)($(rhs),$(lhs_dims_placeholders ...)) for $(loop_wholeElements ...)]
    end
    return outExp
end

```

While I think “I am almost there”, unfortunately the above code return a syntax error (“invalid iteration specification”) due to the `for $(loop_wholeElements)` part… indeed I don’t know how to expand the expressions in lhs\_dims\_placeholders and loop\_wholeElements in order to “assemble” the expanded expression…

---

<div class="post-metadata">

**Author:** ![andrewsteck](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andrewsteck/32/1876_2.png) [@andrewsteck](https://discourse.julialang.org/u/andrewsteck)\
**Post date:** [October 3, 2017, 3:06pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/2 "2017-10-03T15:06:18Z")

</div>

I can’t help with writing macros, but you may want to check out [cartesian indexing](https://julialang.org/blog/2016/02/iteration) as a way to save you some time.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [October 3, 2017, 3:26pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/3 "2017-10-03T15:26:14Z")

</div>

My instinct is that there may be a better way to do this. Are your your parameter structures actually multi-dimensional arrays? If so, have you considered making them subtypes of `AbstractArray`? You would just need to implement `size()`, `getindex()`, `setindex()` and `IndexStyle()` [https://docs.julialang.org/en/stable/manual/interfaces/#man-interface-array-1](https://docs.julialang.org/en/stable/manual/interfaces/#man-interface-array-1) and you’d get multi-dimensional indexing, iteration, and lots of other features for free. In particular, you’d be able to use Base.Cartesian to generate multi-dimensional loops over your data.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [October 3, 2017, 3:27pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/4 "2017-10-03T15:27:54Z")

</div>

Not sure about your particular application here, but have you considered that using a macro might not be appropriate? Have you tried using a regular function that takes an `Expr` as input and then returns an `Expr` as output, and then using `eval` on it? Perhaps that’s worth a try first, before going the macro route.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [October 3, 2017, 3:32pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/5 "2017-10-03T15:32:43Z")

</div>

I think your problem might be the `...` in

```julia
$(loop_wholeElements)

```

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [October 3, 2017, 3:35pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/6 "2017-10-03T15:35:30Z")

</div>

Thank you. My actual data is in the form of a DataFrame parName|region|d1|d2|y|value.

The reason I am using my own get/setData() functions is that I implement my own application-specific query logic on the data.  
For example some variables are known at regional level, but other only an national one, so if data is not available at regional level the query pick it up it at national one, or if a variable is not available for that year I pick it up for the last available year…

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [October 3, 2017, 3:44pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/7 "2017-10-03T15:44:03Z")

</div>

Still it doesn’t compile it without it

`show(loop_wholeElements)` returns `Expr[:(d1 in DIM1), :(d2 in DIM2)]` and, when I leave the outExp as

```julia
outExp = quote
      [$(lhs_par)($(rhs),$(lhs_dims_placeholders ...)) for d1 in DIM1, d2 in DIM2]
    end

```

(to let the macro compile) `show(outExp)` returns:

```julia
quote # /home/lobianco/git/ffsm/test.jl, line 47:
    [par1!(begin # /home/lobianco/git/ffsm/test.jl, line 54:
            par2(d1, d2) + par3(d1, d2)
        end, d1, d2, dfix3) for d1 = DIM1, d2 = DIM2]
end

```

I don’t understand why the output expression has (a) the begin/end parts wrapping the replaced lhs expression; (b) the code `for d1 in DIM1, d2 in DIM2` replaced with `for d1 = DIM1, d2 = DIM2`

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [October 3, 2017, 3:56pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/8 "2017-10-03T15:56:40Z")

</div>

Here, try this, I think this should get you closer to what you want, it doesn’t give you the error about iterators and makes the expression you are looking for, i believe

```julia
return :([$(Expr(:generator,:($(Expr(:call,lhs_par,rhs,lhs_dims_placeholders...))),loop_wholeElements...))])

```

Although, you might want to replace `rhs` with something like `rhs.args[2]`, if you want to avoid having the extra quote block in your final expression.

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [October 3, 2017, 10:04pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/9 "2017-10-03T22:04:47Z")

</div>

x-ref: [Julia macro for transforming `f(dim1,dim2,..) = value` into `f(value,dim1,dim2,..)` - Stack Overflow](https://stackoverflow.com/questions/46546440/julia-macro-for-transforming-fdim1-dim2-value-into-fvalue-dim1-dim2)

---

<div class="post-metadata">

**Author:** ![Gnimuc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gnimuc/32/2194_2.png) [@Gnimuc](https://discourse.julialang.org/u/Gnimuc)\
**Post date:** [October 4, 2017, 7:53am UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/10 "2017-10-04T07:53:35Z")

</div>

I’m wondering why the following two expressions behave differently on the scope of `x`:

```julia
julia> macro bar(ex)
           Expr(:comprehension, 1, ex)
       end
@bar (macro with 1 method)

julia> @macroexpand @bar(x in X)
:($(Expr(:comprehension, 1, :(Main.x in Main.X))))

julia> @macroexpand @bar(x = X)
:($(Expr(:comprehension, 1, :(#50#x = Main.X))))

```

The result shows that the `x` of `x in X` is in the global scope, but the `x` of `x = X` is in the local scope, and both of the `X` are in the global scope.

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [October 4, 2017, 8:25am UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/11 "2017-10-04T08:25:31Z")

</div>

Thank you… my knowledge about metaprogrammming is too limited to fully understand the issue… just looks odd that, for example, if I use this expression with your macro:

```julia
@meq pl!(tp in secProducts, r in fr2,"",y2) = sum(pl_(r,pp,"",y2)*a_(r,pp,tp,y2) for pp in priProducts) + m_(r,tp,"",y2)

```

I obtain:

```julia
 :($(Expr(:comprehension, :(pl!(begin # /home/lobianco/git/ffsm/ffsm.jl, line 127:
             sum((pl_(r, pp, "", y2) * a_(r, pp, tp, y2) for pp = priProducts)) + m_(r, tp, "", y2)
         end, tp, r, "", y2)), :(tp = secProducts), :(r = fr2))))

```

Or, if I use `rhs.args[2]` instead of `args`, I obtain:

```julia
:($(Expr(:comprehension, :(pl!(sum((pl_(r, pp, "", y2) * a_(r, pp, tp, y2) for pp = priProducts)) + m_(r, tp, "", y2), tp, r, "", y2)), :(tp = secProducts), :(r = fr2))))

```

(notice in both cases the transformation of `for pp in priProducts` to `for pp = priProducts`)

In any case your macro seems to still working, I accepted your answer on S.O.

As an OT question, as SO and [discourse.julialang.org](http://discourse.julialang.org) are two separate but not disjoint communities, is it considered legit to post on SO and here ?

Many thanks to everyone that helped me with this question.

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [October 4, 2017, 9:10am UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/12 "2017-10-04T09:10:37Z")

</div>

Double posting is discouraged, but if you choose to do it you should post the cross-reference both places.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [October 4, 2017, 12:06pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/13 "2017-10-04T12:06:01Z")

</div>

> [@sylvaticus](#):
>
> The reason I am using my own get/setData() functions is that I implement my own application-specific query logic on the data.

Just wrap your data in a new type (possibly a subtype of `AbstractArray`), and define `getindex` and `setindex!` etcetera functions on that type. Probably don’t use macros for this. Then you can just do:

```julia
for d1 in DIM1, d2 in DIM2
    par1[d1,d2,dfix3] = par2[d1,d2] + par3[d1,d2]
end

```

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [October 4, 2017, 12:35pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/14 "2017-10-04T12:35:39Z")

</div>

Yes, I see now my original solution has an extra parenthesis in it.

Here is the final corrected version, this should give you exactly what you need:

```julia
return Expr(:comprehension,Expr(:generator,Expr(:call,lhs_par,rhs.args[2],lhs_dims_placeholders...),loop_wholeElements...))

```

Although, you need to be careful about using `rhs.args[2]` if you are not entering your command from the REPL, in that case you need to replace `[2]` with `[1]`.

It returns

```julia
:([par1!(par2(d1, d2) + par3(d1, d2), d1, d2, dfix3) for d1 in DIM1, d2 in DIM2])

```

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [October 4, 2017, 1:24pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/15 "2017-10-04T13:24:43Z")

</div>

Alright, so I went ahead and tested this. I was not able to make the `macro` work yet, but as a `function` it works (also, in order for it to work you need `=` not `in` and change the array element for `d.args[2]`:

```julia
par1!(a,b,c,d) = 0; par2(a,b) = 0; par3(a,b) = 0; dfix3 = 0;
function meq(eq)
    lhs_par = eq.args[1].args[1]
    rhs = eq.args[2]
    lhs_dims = eq.args[1].args[2:end]
    loop_counters = [d.args[2] for d in lhs_dims if typeof(d) == Expr]
    loop_sets = [d.args[2] for d in lhs_dims if typeof(d) == Expr]
    loop_wholeElements = [d for d in lhs_dims if typeof(d) == Expr]
    lhs_dims_placeholders = []
    for d in lhs_dims
        if typeof(d) == Expr
            push!(lhs_dims_placeholders,d.args[2])
        else
            push!(lhs_dims_placeholders,d)
        end
    end
    return Expr(:comprehension,Expr(:generator,Expr(:call,lhs_par,rhs.args[2],lhs_dims_placeholders...),loop_wholeElements...))
end

```

Then computing the result like this

```julia
julia> meq(:(par1!(d1 = 1:2, d2 = 1:2, 3) = par2(d1,d2)+par3(d1,d2))) |> eval

2×2 Array{Int64,2}:
 0 0
 0 0

```

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [October 4, 2017, 1:52pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/16 "2017-10-04T13:52:24Z")

</div>

Thank you… both versions now work 🙂

I am using them to write a bio-economic model with a syntax similar to that of JuMP, so I can use JuMP for the optimisation part, and this syntax for everything else (but I am also testing the suggestion of stevengj):

```julia
# set local prices for transformed products
@meq pl!(tp in secProducts, r in fr2,"",y2) = sum(pl_(r,pp,"",y2)*a_(r,pp,tp,y2) for pp in priProducts) + m_(r,tp,"",y2)

# set local demand for primary products
@meq dl!(r in fr2, pp in priProducts, "",y2) = sum(sl_(r, tp,"",y2) * a_(r,pp,tp, y2) for tp in secProducts)

# set total supply (local + abroad[exports])
@meq st!(r in fr2, p in products, "", y2) = sl_(r, p, "", y2) + sa_(r, p, "", y2)

# set total demand (local + abroad[imports])
@meq dt!(r in fr2, p in products, "", y2) = da_(r, p, "", y2) + dl_(r, p, "", y2)

# set initial values for the ces function
@meq q1!(r in fr2, tp in secProducts,"",y1) = 1.0 / (1.0+((dl_(r,tp)/da_(r,tp))^(1/psi_(r,tp)) ) * (pl_(r,tp)/pl_(wld,tp)) )
@meq p1!(r in fr2, tp in secProducts,"",y1) = 1 - q1_(r, tp,"",y1)

# composite demand/price
@meq dc!(r in fr2, tp in secProducts) = (q1_(r,tp,"",y1)*da_(r,tp)^( (psi_(r,tp)-1)/psi_(r,tp) )+p1_(r,tp)*dl_(r,tp)^( (psi_(r,tp)-1)/psi_(r,tp) ) )^(psi_(r,tp)/(psi_(r,tp)-1))
@meq pc!(r in fr2, tp in secProducts) = (da_(r,tp)/dc_(r,tp)) * pl_(wld,tp) + (dl_(r,tp)/dc_(r,tp)) * pl_(r,tp)

# weighted price
@meq pw!(r in fr2, tp in secProducts) = (dl_(r,tp)*pl_(r,tp) + da_(r,tp)*pl_(wld,tp))/dl_(r,tp)+da_(r,tp)

```

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [October 4, 2017, 9:24pm UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/17 "2017-10-04T21:24:48Z")

</div>

Just another general remark, for metaprogramming I have found that using the `Expr(::Symbol,args...)` constructor is overall much more robust and reliable than using the `$` interpolation. The best way to get a first look at a how to construct an unusual expression correctly is to `dump` it and then inspect the `head` and `args` to see how the `Expr` constructor can be applied. With interpolation, sometimes weird things happen.

---

<div class="post-metadata">

**Author:** ![shan](https://avatars.discourse-cdn.com/v4/letter/s/7ab992/32.png) [@shan](https://discourse.julialang.org/u/shan)\
**Post date:** [December 1, 2018, 4:42am UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/18 "2018-12-01T04:42:26Z")

</div>

for j in 1:nhubs, for k in 1:nplant  
if supply\_plant[j,k]==1  
println(“supply\_plant[$j,$k]=”,supply\_plant[j,k])  
else  
end  
end

i am getting this error…ERROR: syntax: invalid iteration specification

Please tell me how to resolve it

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [December 1, 2018, 5:46am UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/19 "2018-12-01T05:46:26Z")

</div>

If you have a new question, please create a new thread rather than posting on an old thread.

---

<div class="post-metadata">

**Author:** ![y4lu](https://avatars.discourse-cdn.com/v4/letter/y/47e85d/32.png) [@y4lu](https://discourse.julialang.org/u/y4lu)\
**Post date:** [December 1, 2018, 6:45am UTC](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217/20 "2018-12-01T06:45:35Z")

</div>

But since it’s here

`for j in 1:nhubs, k in 1:nplant`  
should be fine

[Next page](https://discourse.julialang.org/t/help-in-making-multi-dimensional-equations-a-bit-sexier/6217.md?page=2)