How @code_warntype , precompilation, execution and type interference works?

Marcell_Havlik · January 20, 2021, 3:49pm

Hey Julianners,

I found a really strange behavior! But shit is pretty serious thing all in all.

using InteractiveUtils
struct me{I,O}
	a::Vector{Array{Float32,I}}
	b::Vector{Array{Float32,O}}
end

double(x::Vector{Array{Float32, 3}}) = 2 .* x

function foo(n) 
	double(n.a)
end

function bar()
	m=me{3,2}([randn(Float32,2,3,4)],[randn(Float32,2,4)])
	foo(m)
end
@code_warntype bar()

So bar is an uninterferable thing… which is strange… But ok.
Then running this code snippet:

@time  bar();
@code_warntype bar()
double(x::Vector{Array{Float32, 3}}) = 2 .* x
@code_warntype bar()

After redefining double(…), bar() becomes an interferable class!!

What is going on here?

Marcell_Havlik · January 20, 2021, 5:19pm

I don’t know if I was using it right… but I think I found the error with SnoopCompile.jl

Replacing double to 2 to 2.f0:
double(x::Vector{Array{Float32, 3}}) = 2.f0 .* x
So the problem maybe was with: *(::Int64, ::Array{Float32, 3})

Interesting anyway.
If someone else know the problem came from something different then let me know.

lmiq · January 20, 2021, 5:31pm

I think that the problem is more related with the parameterization of the me struct. If you define it with:

julia> struct me{I,O}
               a::I
               b::O
       end

Everything becomes type-stable. I do not have the full expertise to explain this, but I think that to use the sizes of the arrays as the parameters of the type as you intended, one would expect to use static arrays.

For instance, this is type-stable as well:

julia> using StaticArrays

julia> struct me{I,O}
               a::Vector{SVector{Float32,I}}
               b::Vector{SVector{Float32,O}}
       end

Marcell_Havlik · January 20, 2021, 5:39pm

Interesting point. Thank you for the answer.
I tested it with the struct you described and it gives:

Body::AbstractVector{var"#s827"} where var"#s827"
1 ─ %1  = Core.apply_type(Main.Array, Main.Float32, 3)::Core.Const(Array{Float32, 3})
│   %2  = Core.apply_type(Main.Vector, %1)::Core.Const(Vector{Array{Float32, 3}})
│   %3  = Core.apply_type(Main.Array, Main.Float32, 2)::Core.Const(Matrix{Float32})
│   %4  = Core.apply_type(Main.Vector, %3)::Core.Const(Vector{Matrix{Float32}})
│   %5  = Core.apply_type(Main.me, %2, %4)::Core.Const(me{Vector{Array{Float32, 3}}, Vector{Matrix{Float32}}})
│   %6  = Main.randn(Main.Float32, 2, 1, 1)::Array{Float32, 3}
│   %7  = Base.vect(%6)::Vector{Array{Float32, 3}}
│   %8  = Main.randn(Main.Float32, 2, 1)::Matrix{Float32}
│   %9  = Base.vect(%8)::Vector{Matrix{Float32}}
│         (m = (%5)(%7, %9))
│   %11 = Main.foo(m)::AbstractVector{var"#s827"} where var"#s827"
└──       return %11

Instead of this:

Body::Vector{Array{Float32, 3}}
1 ─ %1  = Core.apply_type(Main.Array, Main.Float32, 3)::Core.Const(Array{Float32, 3})
│   %2  = Core.apply_type(Main.Vector, %1)::Core.Const(Vector{Array{Float32, 3}})
│   %3  = Core.apply_type(Main.Array, Main.Float32, 2)::Core.Const(Matrix{Float32})
│   %4  = Core.apply_type(Main.Vector, %3)::Core.Const(Vector{Matrix{Float32}})
│   %5  = Core.apply_type(Main.me, %2, %4)::Core.Const(me{Vector{Array{Float32, 3}}, Vector{Matrix{Float32}}})
│   %6  = Main.randn(Main.Float32, 2, 1, 1)::Array{Float32, 3}
│   %7  = Base.vect(%6)::Vector{Array{Float32, 3}}
│   %8  = Main.randn(Main.Float32, 2, 1)::Matrix{Float32}
│   %9  = Base.vect(%8)::Vector{Matrix{Float32}}
│         (m = (%5)(%7, %9))
│   %11 = Main.foo(m)::Vector{Array{Float32, 3}}
└──       return %11

So I don’t think the struct has anything to do with the problem.

Btw, I am still looking for the answer why does this replacement of this
double(x::Vector{Array{Float32, 3}}) = 2 .* x with this double(x::Vector{Array{Float32, 3}}) = 2.f0 .* x solves the problem.

By the way thank you for the StaticArrays idea, I never understood what is that and I will look after that too.

lmiq · January 20, 2021, 5:42pm

Is this the same code as that of your MWE? I do not get that (and changing the 2 to 2.f0 does not change anything here). With the static arrays I get, from @code_warntype:

Body::Union{}
1 ─ %1 = Core.apply_type(Main.me, 3, 2)::Core.Compiler.Const(me{3,2}, false)
│   %2 = Main.randn(Main.Float32, 2, 3, 4)::Array{Float32,3}
│   %3 = Base.vect(%2)::Array{Array{Float32,3},1}
│   %4 = Main.randn(Main.Float32, 2, 4)::Array{Float32,2}
│   %5 = Base.vect(%4)::Array{Array{Float32,2},1}
│        (m = (%1)(%3, %5))
│        Main.foo(m)
└──      Core.Compiler.Const(:(return %7), false)

Marcell_Havlik · January 20, 2021, 5:47pm

Yeah, the problem is that if you replace the struct with

       struct me{I,O}
               a::I
               b::O
       end

then you have to replace the bar function:
m=me{3,2}([randn(Float32,2,3,4)],[randn(Float32,2,4)])
wtih this
m=me{Vector{Array{Float32,3}},Vector{Array{Float32,2}}}([randn(Float32,2,3,4)],[randn(Float32,2,4)])

Because in the other case something strange happenes that is not equal with the test. ( Union{} in the return)

lmiq · January 20, 2021, 6:16pm

Well, here is a more minimal example:

using InteractiveUtils

struct me{T}
  a::T
end

double(x) = 2 .* x

function bar()
  m=me([[0]])
  double(m.a)
end

@time bar();
@code_warntype bar()

double(x) = 2 .* x

@code_warntype bar()

I think that is definitely a bug.

Result:

  0.055437 seconds (163.06 k allocations: 8.348 MiB)
Variables
  #self#::Core.Compiler.Const(bar, false)
  m::me{Array{Array{Int64,1},1}}

Body::Union{BitArray{1}, Array}
1 ─ %1 = Base.vect(0)::Array{Int64,1}
│   %2 = Base.vect(%1)::Array{Array{Int64,1},1}
│        (m = Main.me(%2))
│   %4 = Base.getproperty(m, :a)::Array{Array{Int64,1},1}
│   %5 = Main.double(%4)::Union{BitArray{1}, Array}
└──      return %5
Variables
  #self#::Core.Compiler.Const(bar, false)
  m::me{Array{Array{Int64,1},1}}

Body::Array{Array{Int64,1},1}
1 ─ %1 = Base.vect(0)::Array{Int64,1}
│   %2 = Base.vect(%1)::Array{Array{Int64,1},1}
│        (m = Main.me(%2))
│   %4 = Base.getproperty(m, :a)::Array{Array{Int64,1},1}
│   %5 = Main.double(%4)::Array{Array{Int64,1},1}
└──      return %5

mbauman · January 20, 2021, 6:34pm

Hmmm… even minimaler on Julia 1.5:

julia> function bar()
         m=Ref([[0]])
         double(m[])
       end
bar (generic function with 1 method)

julia> double(x) = 2 .* x
double (generic function with 1 method)

julia> @code_warntype bar()
Variables
  #self#::Core.Compiler.Const(bar, false)
  m::Base.RefValue{Array{Array{Int64,1},1}}

Body::Union{BitArray{1}, Array}
1 ─ %1 = Base.vect(0)::Array{Int64,1}
│   %2 = Base.vect(%1)::Array{Array{Int64,1},1}
│        (m = Main.Ref(%2))
│   %4 = Base.getindex(m)::Array{Array{Int64,1},1}
│   %5 = Main.double(%4)::Union{BitArray{1}, Array}
└──      return %5

julia> bar()
1-element Array{Array{Int64,1},1}:
 [0]

julia> double(x) = 2 .* x
double (generic function with 1 method)

julia> @code_warntype bar()
Variables
  #self#::Core.Compiler.Const(bar, false)
  m::Base.RefValue{Array{Array{Int64,1},1}}

Body::Array{Array{Int64,1},1}
1 ─ %1 = Base.vect(0)::Array{Int64,1}
│   %2 = Base.vect(%1)::Array{Array{Int64,1},1}
│        (m = Main.Ref(%2))
│   %4 = Base.getindex(m)::Array{Array{Int64,1},1}
│   %5 = Main.double(%4)::Array{Array{Int64,1},1}
└──      return %5

different but similar on Julia master

julia> function bar()
         m=Ref([[0]])
         double(m[])
       end
bar (generic function with 1 method)

julia> double(x) = 2 .* x
double (generic function with 1 method)

julia> @code_warntype bar()
Variables
  #self#::Core.Const(bar)
  m::Base.RefValue{Vector{Vector{Int64}}}

Body::AbstractVector{var"#s829"} where var"#s829"
1 ─ %1 = Base.vect(0)::Vector{Int64}
│   %2 = Base.vect(%1)::Vector{Vector{Int64}}
│        (m = Main.Ref(%2))
│   %4 = Base.getindex(m)::Vector{Vector{Int64}}
│   %5 = Main.double(%4)::AbstractVector{var"#s829"} where var"#s829"
└──      return %5

julia> bar()
1-element Vector{Vector{Int64}}:
 [0]

julia> double(x) = 2 .* x
double (generic function with 1 method)

julia> @code_warntype bar()
Variables
  #self#::Core.Const(bar)
  m::Base.RefValue{Vector{Vector{Int64}}}

Body::Vector{Vector{Int64}}
1 ─ %1 = Base.vect(0)::Vector{Int64}
│   %2 = Base.vect(%1)::Vector{Vector{Int64}}
│        (m = Main.Ref(%2))
│   %4 = Base.getindex(m)::Vector{Vector{Int64}}
│   %5 = Main.double(%4)::Vector{Vector{Int64}}
└──      return %5

This feels familiar but I can’t find the issue on GitHub. Pretty sure there is one.

raminammour · January 20, 2021, 8:40pm

Prob this one and its relatives

Marcell_Havlik · January 20, 2021, 9:17pm

Wow! I was like, I have the solution but that is some bug that shows it is not just in my one scenario. (Also I am on Julia 1.6.) I thought the problem was with the Int to Float32 conversion in my case, but this is even crazier.

Are there any idea what can be the problem there? I can think of:

The type interference algotihm just doesn’t have the knowledge how to expand a (Int * Array{Float64})
Using .* instead of *
Using [[1. 1.]] intead of [[1.]] also works

Any exact idea what can be the problem there? Julia type interference algorithm has a hard time at Type1{Type2{Type3}}, anything more then fitting 2 type.

FPGro · January 20, 2021, 9:42pm

I don’t think it’s any of the bullet pointed things, those are trivial under normal circumstances. Clicking through the related GitHub issues left me with the impression, that this may be an unfortunate combination of the compiler giving up on inference at some point (either because of some some time-to-first-plot optimization that makes inference stop after enough nesting steps to reduce time, or because of some overlooked buggy recursion there) and caching (parts of) the inference results so far. I guess that may lead to correct inference on the second try, if cached stuff somehow solves the problem that occured in the first place. But this is really just a guess, one would have to know exactly why and where it fails to infer in the first place.

raminammour · January 21, 2021, 2:54pm

Yes, given Jeff’s response, I don’t think we will be solving it on discourse anytime soon

Topic		Replies	Views
`code_warntype` understanding New to Julia performance , type , numerics	30	5553	October 20, 2020
Ominous type instability Performance bug , code_warntype , type-stability	10	1308	August 20, 2022
Why Julia fails to infere types in this example? What is the cost of this? What are requirements for type inference? General Usage question , inference , type	12	498	May 8, 2024
Suggestion: introduce something like a @inferable or @typestable macro to the language Internals & Design	30	1585	August 21, 2022
@code_warntype result depending on compiled functions General Usage question , code_warntype , type-stability	0	186	October 27, 2023

How @code_warntype , precompilation, execution and type interference works?

Related topics