Unpack named tuples (does anyone know how to do it?)

General Usage
1

Hi all,

I just wanted to know if it is possible to unpack named tuples in the following context. I know that Parameters.jl addresses some related issues, but I have not been able to figure out if it can handle the following case. Define a named tuple using @with_kw:

ParamsNamedTuple = @with_kw (
  T = 1.0,
  K = 0.5,
  M = T + K,
  v = zeros(2),
  d = 0.1,
  t = [i * d for i in 0:10]
)

We can unpack particular components of a particular instance using:

julia> params = ParamsNamedTuple();
julia> @unpack T, K = params;
julia> T
1.0
julia> K
0.5
julia> M
ERROR: UndefVarError: M not defined

However, I would like to unpack them all:

@unpack params

Is this possible?

On the other hand, I know that we can unpack all the objects if instead of defining a named tuple we define a struct, such as:

@with_kw struct ParamsStruct{T1, T2, T3, T4, T5, T6}
  T::T1 = 1.0
  K::T2 = 0.5
  M::T3 = T + K
  v::T4 = zeros(2)
  d::T5 = 0.1
  t::T6 = [i * d for i in 0:10]
end

Then:

params = ParamsStruct()
@unpack_ParamsStruct params

Any unpacking strategy would be ok. There is no need to use a macro in Parameters.jl or any other package.

Thank you!

2 Likes
2

you need to know ahead of time the number of values if you gonna assign them. but extracting the values of a NamedTuple can be done via the function values

tup = (a=3,b=5) #example named tuple
tup_values = values(tup) # (3,5)
a,b = values(tup) #a = 3, b = 5, two variables defined

this approach requires assigning the variable names manually, if you want that automatically (i do not recommend this, as it can collide with any defined variable), maybe this macro can help:

function _unpack(x::NamedTuple)
    kk = keys(x)
    vv = values(x)
    i = 1
    for k in kk
        @eval $k = $vv[$i]
        i +=1
    end
end


macro unpack_namedtuple(arg)
 quote
    _unpack($arg)
end |> esc
end

tup = (a=1,b=2)
@unpack_namedtuple tup #a and b are now defined in the scope

this can fail in this case, for example:

tup = (tup=2,b=3)
@unpack_namedtuple tup
julia>tup #the tuple is lost
2
1 Like
3

The @eval solution also only works at global scope:

julia> function foo()
         tup = (a = 1, b=2)
         @unpack_namedtuple tup
         a = a + 1
       end
foo (generic function with 1 method)

julia> foo()
ERROR: UndefVarError: a not defined
Stacktrace:
 [1] foo() at ./REPL[6]:4
 [2] top-level scope at REPL[7]:1

(this fails because the a created by the macro is a global variable).

To the original question, I think the answer is no: thereā€™s no efficient way to do this. You have to, somewhere, somehow, specify the set of local variables you want to create. A macro canā€™t do anything magical: it just makes it easier to write out code that you could otherwise write out by hand. If thereā€™s no way to write the code by hand, then thereā€™s no way to implement it via a macro.

Unpacking a subset of fields with @unpack a, b = params works because by listing a, b = params you are providing enough information for the macro to create code like:

a = params.a
b = params.b

The @unpack_ParamsStruct call works because that macro was defined automatically when you ran @with_kw, which looked at your struct definition and picked out all of the field names.

But what would the generated code be for the hypothetical @unpack params? A macro just takes some code and produces other code. The input to the macro is just the symbol :params. How could the macro know that it should unpack params into fields a, b, and c?

2 Likes
7

Hi @rdeits, thanks for your detailed answer!

That is exactly the question. However, this was just syntax, it was not the proposed solution. I know that the input to the macro is just the symbol :params. I though that the macro may return something like @longemen3000 suggested. Actually, I thought maybe the macro was already coded in some package and I wanted to know in which one :smile:.

Finally, as I understand, a solution may be the same one in Parameters.jl for structs, i.e., create a macro @unpack_ParamsNamedTuple on the fly.

I have already raised an issue in Parameters.jl but I am still waiting for @mauro3 response :grinning:.

Thanks and regards!

8

If you have a specific NamedTuple type that you use an alias for, you might as well just define struct, they are much nicer for dispatch etc.

1 Like
9

Thatā€™s totally reasonable to hope for, but it wonā€™t work precisely because the suggested code, even if written out by hand, still doesnā€™t work (it creates global variables, not local ones, because that is inherently how @eval always works).

The only way for @unpack params to work would be for the type of params to be known when the @unpack macro is expanded, but thatā€™s impossible: if youā€™re inside a function, the macro will be expanded before the function is run and therefore before params has a value.

2 Likes
10

This is correct!

But building a specific macro on the fly will solve this issue, right?

11

Hi @Tamas_Papp, thanks for your reply.

You may find useful this discussion in order to see why I would like to have named tuples instead of structs.

12

I am still not sure if I understand why you canā€™t use structs, even after reading that issue, but as suggested there you can just have the user pass functions.

The fundamental difficulty with a macro is that macros operate on syntax, while (in general) the fields depend on the type of a value. There are ways of jumping around these two domains with eval, but usually that ends up being the least idiomatic solution for a problem.

2 Likes
13

I started thinking about this recently after seeing this thread. I mostly agree with @Tamas_Papp that a struct is likely the best solution, however I think these sorts of code transformations are interesting to think about.

Basically, what would be needed to do this is a macro that knows about type information. Base Julia does not have such a construct, but IRTools.jl sorta does with itā€™s @dynamo.

Here is a dynamo I call with that in some senses will behave like @unpack, even on a named Tuple:

using IRTools: IRTools, @dynamo, @code_ir, postwalk,  prewalk, IR, argument!, 
using IRTools: xcall, var, insertafter!, isexpr, branches, blocks
using Setfield: @set

@dynamo function with(f, obj)
    ir = IR(f)
    objarg = argument!(ir)
    d = Dict()
    for fn āˆˆ reverse(fieldnames(obj))
        d[fn] = insertafter!(ir, var(1), xcall(:getfield, objarg, QuoteNode(fn)))
    end

    function replacer(x)
        if x isa GlobalRef && x.name āˆˆ fieldnames(obj)
            d[x.name]
        else
            x
        end
    end

    for (n, st) āˆˆ ir
        ir[n] = prewalk(replacer, st)
    end

    for blck āˆˆ blocks(ir)
        bs = branches(blck)
        for i āˆˆ eachindex(bs)
            b = bs[i]
            bs[i] = @set b.condition = postwalk(replacer, b.condition)
            for j āˆˆ eachindex(bs[i].args) 
                bs[i].args[j] = postwalk(replacer, bs[i].args[j])
            end
        end
        ir
    end
    ir
end

Apologies for the spaghetti code, Iā€™m not yet very skilled with writing IR. Hereā€™s with in action:

julia> nt = (;a=1, b=4.0, c="hi", d=true)
(a = 1, b = 4.0, c = "hi", d = true)

julia> with(nt) do
           if !d
               (a - b)/(a + b)
           else
               c
           end
       end
"hi"

This is a rather brittle solution that has many weaknesses.

For instance, if thereā€™s a local variable available named d, then d wonā€™t get replaced with nt.d:

julia> let
           nt = (;a=1, b=4.0, c="hi", d=true)
           d = false
           with(nt) do
               if !d
                   (a - b)/(a + b)
               else
                   c
               end
           end
       end
-0.6

Iā€™m sure there are other problems and corner cases, but I learned a lot writing this so I thought Iā€™d share.

2 Likes
14

btw

1 Like