Type stability problem (Vcat, Tuple and function)

General Usage
, , ,
1

Hey Julianners,
I am working on type stability and I just can’t understand what is going on here. Finally I could minimalise my example to 3 line.

@code_warntype (vcat([randn(1,1,3) for i in 1:3]...))
baz() = (vcat([randn(1,1,3) for i in 1:3]...),)
@code_warntype baz()

I tried like 40-60 different idea to make this basic structure type stable but something is wrong with the Tuple typestability… with Generator I could see the types in many case without converting it to Tuple which is ultimately create a Instable “Tuple” or “Tuple{Any,Any,Any}” and many more different types.
So how can I achieve the baz to be type safe here. (I tried with notation too without any success.)

Any idea what can be the problem here?

2

Annotating the output of vcat seems to do the trick (these cat functions tend to cause type-instabilities, because the type of the output is dependent on the input. There is some discussion on this in this recent thread):

julia> baz() = (vcat([randn(1,1,3) for i in 1:3]...)::Array{Float64,3},)
baz (generic function with 1 method)

julia> @code_warntype baz()
Variables
  #self#::Core.Compiler.Const(baz, false)
  #7::var"#7#8"

Body::Tuple{Array{Float64,3}}
1 ─      (#7 = %new(Main.:(var"#7#8")))
│   %2 = #7::Core.Compiler.Const(var"#7#8"(), false)
│   %3 = (1:3)::Core.Compiler.Const(1:3, false)
│   %4 = Base.Generator(%2, %3)::Core.Compiler.Const(Base.Generator{UnitRange{Int64},var"#7#8"}(var"#7#8"(), 1:3), false)
│   %5 = Base.collect(%4)::Array{Array{Float64,3},1}
│   %6 = Core._apply_iterate(Base.iterate, Main.vcat, %5)::Any
│   %7 = Core.apply_type(Main.Array, Main.Float64, 3)::Core.Compiler.Const(Array{Float64,3}, false)
│   %8 = Core.typeassert(%6, %7)::Array{Float64,3}
│   %9 = Core.tuple(%8)::Tuple{Array{Float64,3}}
└──      return %9

Still, my preferred way to deal with such thing, if possible, would be to keep those transformations on the data outside the functions that are critical for performance, and pass the “final” data structure to those. In that case, even if there are type-instabilities in the “preparation” of the data, it will not propagate into the critical functions.

Edit: As pointed by @mcabbott , the problem here is the splatting, and is not of the same sort of that of the other thread. It would be the same if you used cat, though, even without the splatting:

julia> baz() = (cat([randn(1,1,3) for i in 1:3],dims=2),)
baz (generic function with 1 method)

julia> @code_warntype baz()
Variables
  #self#::Core.Compiler.Const(baz, false)
  #23::var"#23#24"

Body::Tuple{Array} #<--- RED
2 Likes
3

I think that a good rule of thumb is never to splat a vector, if you are concerned about speed, there’s usually another way. It is safe to splat tuples (although not to make tuples from vectors, that’s effectively a splat). These should be type-stable:

baz_nt() = vcat(ntuple(i -> randn(1,1,3), 3)...); # splat a tuple

baz_red() = reduce(vcat, [randn(1,1,3) for i in 1:3]);  # no splat
10 Likes
4

So great answers guys!! I can’t believe someone always know the answer! I was… totally lost already!
THANK you for both of the answers!

I see I had 2 problem and one solved like that, the other one is still a little foggy for me, is there a better way to do this?

qux(x::Tuple{A,B}) where {A,B} = Tuple((reduce(vcat, x[1]),reduce(vcat, x[2])))
qux(x::Tuple{A,B,C}) where {A,B,C} = Tuple((reduce(vcat, x[1]),reduce(vcat, x[2]),reduce(vcat, x[3])))
@code_warntype qux(([randn(1,1,3) for i in 1:3], [randn(1,1,3) for i in 1:2]))
@code_warntype qux(([randn(1,1,3) for i in 1:3], [randn(1,1,3) for i in 1:2], [randn(1,1,3,1) for i in 1:2]))

I don’t know how can I create Tuple that is type stable. If I do:
qux(x) = Tuple(reduce(vcat,y) for y in x)
I got Tuple type in return with RED.

5

I think we are approaching a point where does not make sense to try to achieve type-stability.

qux(x) = Tuple(reduce(vcat,y) for y in x)

If different values of x (not types) change the type of the return (i.e., the amount of elements in x will determine the type of the tuple returned), then type-instability is inherent and impossible to avoid. What you can do is minimize damage by either:

  1. Putting a type annotation by the side of each call to qux (this only works if you are always working with vectors of the same size and type and therefore you know which will be the type returned by the function), e.g., qux(my_known_length_and_type_vector) :: Tuple{Int, Int}.
  2. Let the function return a type-unstable value, and then, immediately pass this value as an argument to another function (that does everything you could want to do with this value). This way, for each possible type returned by the type-unstable function, this second function will compile a specialized version and the type-instability will be restricted to dynamic dispatch of this single secondary function (instead of a dynamic dispatch for every subsequent call that takes the variable of unstable type).
3 Likes
6

Hey,

Yeah, the first version seems like a good solution.
I don’t think the second works. The compiler only knows “Any” about the return, so it just can’t precompile the right version. I think that will be compiled dinamically during runtime, I guess.

Thank you for all the answers!

7

That is the default in Julia, most function will be “compiled dinamically during runtime”. The difference when dealing with a type-unstable value is that the function call will be dinamically dispatched (also, if your values vary a lot on type, you will inccur in the compilation cost multiple times), but the fact the compilation will happen during runtime is completely normal.

8

Yes, I understand that it is “compiled dinamically during runtime”, but I need precompileable version, so avoid the RED types in compilation time.
So my goal here is the type stability and precompile the whole program.

For me compilation takes 120-150 seconds while the run is like 2 seconds in certain fast usecase. precompile would allow it to run/develop times faster.

9

That looks quite a lot. Maybe you want to take a look at GitHub - timholy/SnoopCompile.jl: Making packages work faster with more extensive precompilation.

2 Likes