Map over namedtuple keys and values: confusing error

[aplavin]

Hi! I want to map a function, which gets a Pair, over both keys and values of a namedtuple, and the most obvious way seems to be using, well, pairs:

f(p) = (p.first, p.second)
nt = (A=5, B=6)

pairs(nt) |> collect
2-element Array{Pair{Symbol,Int64},1}:
 :A => 5
 :B => 6
# looks exactly what I want

pairs(nt) |> first |> f
(:A, 5)
# applying to the first element works, so map should also work - right?

map(f, pairs(nt))
map is not defined on dictionaries
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] map(::Function, ::Base.Iterators.Pairs{Symbol,Int64,Tuple{Symbol,Symbol},NamedTuple{(:A, :B),Tuple{Int64,Int64}}}) at ./abstractarray.jl:2101
 [3] top-level scope at In[42]:1

Not sure why it fails, error message makes no sense to me.

[Tamas_Papp]

The error message tells you that this is deliberately undefined. The reason for this is that mapping the values, and keeping the keys (ie returning a NamedTuple{(:A,:B)} is also “reasonable” behavior, and at this point it is undecided whether this is preferable.

Erroring now allows this to be defined in a later Julia version without making it a breaking change.

[zgornel]

map(f, collect(pairs(nt))) works.

[aplavin]

The thing is that error message is about dictionaries, and I don’t pass neither Dict nor a NamedTuple to map - it’s just an plain iterable of pairs.

[aplavin]

Bumping this thread - so there is indeed no way to map or broadcast over pairs(...) (iterable of pairs) without collecting this iterable into an array?

[Tamas_Papp]

There a multiple ways of mapping a NamedTuple, depending on whether you want the result to be

1. inferred,
2. also NamedTuple,
3. with the same keys, or different ones (ie whether those are mapped, too).

Please provide an MWE, with the desired input and output, for your specific use case.

Please read: make it easier to help you

Welcome to the Julia Discourse! We are enthusiastic about helping Julia programmers, both beginner and experienced. This public service announcement (PSA) outlines best practices when asking for help. Following these points makes it easier for us to help you and more likely you’ll get a prompt, useful answer. Keywords are highlighted to make it easier to refer to specific points. Choose a descriptive title that captures the key part of your question, eg “plots with multiple axes” instead of …

[aplavin]

Below is what I would expect from mapping/broadcasting on iterable of pairs - note that there is no mapping over a namedtuple or dictionary here, even though the error says that.

    f = x::Pair -> (x.first, x.second)
    nt = (a=1, b=2, c=3)
    map(f, nt |> pairs |> collect)  # gives expected result: list of `f` applications to each pair
    map(f, nt |> pairs)  # error: map is not defined on dictionaries
    dct = Dict(:a => 1, :b => 2, :c => 3)
    map(f, dct |> pairs |> collect)  # gives expected result: list of `f` applications to each pair
    map(f, dct |> pairs)  # error: map is not defined on dictionaries

Same with broadcasting instead of map. The error message is a bit less confusing in that case: ArgumentError: broadcasting over dictionaries and NamedTuples is reserved, which is still not completely correct. Looks like mapping and broadcasting over dicts and namedtuples is disabled on purpose (sounds good, because there is no unambiguous way to iterate them), but somehow this also gets applied to pairs iterator which isn’t a dict nor a namedtuple.

[Tamas_Papp]

Yes, the error message is confusing. I thought there was an issue or even a PR about this, but I cannot dig it up at the moment.

[aplavin]

But is the behavior intentional? As of now, it seems like there is no way to iterate over pairs without collect. Why disallow map(f, pairs(nt)) in the first place?

[Tamas_Papp]

It is a consequence of Pairs <: AbstractDict.

Personally I consider this unfortunate, but I don’t know if there is any deeper reason for it. Like you said, leaving map(f, ::NamedTuple) and similar undefined is fine, but pairs should provide an iterable that just works in all contexts where iterables work.

That said, a current workaround is an “identity iterator” like

julia> n = (a = 5, b = 6)
(a = 5, b = 6)

julia> map(x -> (x..., ), Iterators.filter(_ -> true, pairs(n)))
2-element Array{Tuple{Symbol,Int64},1}:
 (:a, 5)
 (:b, 6)

(there are other options, didn’t benchmark).

[zgornel]

I do not believe there are any explicit conditions regarding map’s inputs however, looks to me that some implicit order (or index) of the elements needs to exist. map is not applicable on AbstractSets either which I believe is unfortunate as well. I would see no issue for example in map(x->x+1, (i for i in 1:10)) to return (x+1 for x in 1:10) i.e. another iterator.