Destructuring syntax in julia 1.7

Internals & Design
63

The problem is that getproperty may execute arbitrary code (as opposed to just looking up struct fields), so when we implemented tab completion we wanted to be a bit conservative/cautious about whether this is good idea to do during completion. Might be worth revisiting.

3 Likes
64

You’re right, it doesn’t infer Any. Running this constructor, the resulting NamedTuple adopts the types of the Dict’s values:

julia>NamedTuple(d::Dict{String,T} where T) = NamedTuple{Tuple(Symbol(k) for k ∈ keys(d))}(v for v ∈ values(d))
NamedTuple

julia> typeof(NamedTuple(Dict("a"=>1, "b"=>2)))
NamedTuple{(:b, :a), Tuple{Int64, Int64}}

I suppose one could force the types, but I’m yak shaving now:

NamedTuple(d::Dict{String,T}) where T = 
    NamedTuple{Tuple(Symbol(k) for k ∈ keys(d)), NTuple{length(d), T}}(v for v ∈ values(d))

Nonetheless, running @code_warntype on these constructors flags some Vararg types.

65

Any julia object always has a concrete type - this is not related to what inference thinks the object type might be:

julia> MyTuple(d::Dict{String,T})  where T = NamedTuple{Tuple(Symbol(k) for k ∈ keys(d))}(v for v ∈ values(d))
MyTuple (generic function with 1 method)

julia> @code_warntype MyTuple(Dict("foo" => :bar))
MethodInstance for MyTuple(::Dict{String, Symbol})
  from MyTuple(d::Dict{String}) @ Main REPL[1]:1
Arguments
  #self#::Core.Const(MyTuple)
  d::Dict{String, Symbol}
Locals
  #3::var"#3#4"
Body::NamedTuple
1 ─      (#3 = %new(Main.:(var"#3#4")))
│   %2 = #3::Core.Const(var"#3#4"())
│   %3 = Main.keys(d)::Base.KeySet{String, Dict{String, Symbol}}
│   %4 = Base.Generator(%2, %3)::Base.Generator{Base.KeySet{String, Dict{String, Symbol}}, var"#3#4"}
│   %5 = Main.Tuple(%4)::Tuple{Vararg{Symbol}}
│   %6 = Core.apply_type(Main.NamedTuple, %5)::Type{NamedTuple{_A}} where _A
│   %7 = Main.values(d)::Base.ValueIterator{Dict{String, Symbol}}
│   %8 = Base.Generator(Base.identity, %7)::Base.Generator{Base.ValueIterator{Dict{String, Symbol}}, typeof(identity)}
│   %9 = (%6)(%8)::NamedTuple
└──      return %9

The inferred NamedTuple return type visible in @code_warntype is ultimately what matters to inference, NOT the concrete type of the actual NamedTuple object. Inference doesn’t know about the object and its concrete type - it can’t, since the object doesn’t exist when inference runs.

Moreover, since the names in the Dict are not part of its type, it cannot be type stable, no matter how much you twist it - this will always be type unstable.

You see those Vararg due to the number of the elements of the Dict being unknown to the compiler.

1 Like
66

I’m revisiting this, now that I’ve learned (a lot) more about the type system, and you’re right about the type instability. The lessons learned here have been quite useful in some recent work. :pray:

That said, it’s not unique to my custom NamedTuple constructor, because the built-in constructor is already type-unstable when you feed it a Dict{Symbol} or Tuple{Pair{Symbol}}.

@code_warntype NamedTuple(Dict(:a=>1, :b=>2))
@code_warntype NamedTuple((:a=>1, :b=>2))
67

This is what we expect though right?

Those symbols are runtime values you are lifting back to the type domain as NamedTuple keys (the classic case for type instability). I guess you are hoping constant propagation and escape analysis could work through the Dict/Pair constructors? In the Dict constructor :a and :b are copied to a keys vector, then they are read back out of it in the NamedTuple constructor.

2 Likes
68

Yes, it’s expected behavior. I had originally thought @Sukera’s critique was somehow unique to my custom constructor, so I didn’t correctly understand what was meant by it.

69

It’s always helpful to have a concrete example at hand, but yeah, the problem of type stability is not unique to your particular code. Lots of people encounter it.

2 Likes