I was thinking about the Appendix with JET.jl of the Rust vs Julia in scientific computing blog, and I started wondering if we could create a code_errortype or @code_errortype that errors if @code_warntype would display red. This would allow us to test for the presence of Any when we did not intend to use dynamic typing.
In the appendix of the blog, a distinction is made between Rust’s type inference and Julia’s type inference. This difference is due to the semantics of Rust being distinct from Julia in terms of when type inference can be done. In the example below, the type of v is not determined until v.push(1.0).
Rust is allowed to retroactively determine the type of v due to type inference for v, but Julia is not. Vec::new() does not imply any element type here.
In Julia, v = [] is equivalent to v = Any[]. The Any element type is implied, and the type of v is determined on that line as Vector{Any}. It’s not that Julia cannot do type inference here, it’s just that we never allowed or asked it to do so while implicitly asking Julia to use dynamic typing.
We can ask Julia to do type inference at the construction of v1 though if this is done after at least one element types is known.
julia> function test()
v2 = [1.0] # Vector{Float64}, type inference
v1 = [v2] # Vector{Vector{Float64}}, type inference
push!(v1, [1]) # The type of v1 does not change!
# [1] is converted from Vector{Int} to Vector{Float64}
last = pop!(v1) # Vector{Float64}
println("OK") # OK
println(last.pop()) # JET.jl static analysis error
println("No problem!") # Unreachable
nothing
end
test (generic function with 1 method)
julia> @report_call test()
═════ 1 possible error found ═════
┌ test() @ Main ./REPL[68]:8
│┌ getproperty(x::Vector{Float64}, f::Symbol) @ Base ./Base.jl:37
││ type Array has no field pop: Base.getfield(x::Vector{Float64}, f::Symbol)
│└────────────────────
Not only does @code_warntype know that v1 will be Vector{Vector{Float64}}. It even figured out that the function is not going to be returning nothing.
julia> @code_warntype test()
MethodInstance for test()
from test() @ Main REPL[68]:1
Arguments
#self#::Core.Const(test)
Locals
last::Vector{Float64}
v1::Vector{Vector{Float64}}
v2::Vector{Float64}
Body::Union{}
1 ─ (v2 = Base.vect(1.0))
│ (v1 = Base.vect(v2))
│ %3 = v1::Vector{Vector{Float64}}
│ %4 = Base.vect(1)::Vector{Int64}
│ Main.push!(%3, %4)
│ (last = Main.pop!(v1))
│ Main.println("OK")
│ Base.getproperty(last, :pop)
│ Core.Const(:((%8)()))
│ Core.Const(:(Main.println(%9)))
│ Core.Const(:(Main.println("No problem!")))
└── Core.Const(:(return Main.nothing))
Compare that output of @code_warntype to the following which knows the return type.
julia> function test_nopop()
v2 = [1.0] # Vector{Float64}, type inference
v1 = [v2] # Vector{Vector{Float64}}, type inference
push!(v1, [1]) # The type of v1 does not change!
last = pop!(v1) # Vector{Float64}
println("OK") # OK
#println(last.pop()) # Runtime error 💥
println("No problem!") # Not reachable
nothing
end
test_nopop (generic function with 1 method)
julia> @code_warntype test_nopop()
MethodInstance for test_nopop()
from test_nopop() @ Main REPL[91]:1
Arguments
#self#::Core.Const(test_nopop)
Locals
last::Vector{Float64}
v1::Vector{Vector{Float64}}
v2::Vector{Float64}
Body::Nothing
1 ─ (v2 = Base.vect(1.0))
│ (v1 = Base.vect(v2))
│ %3 = v1::Vector{Vector{Float64}}
│ %4 = Base.vect(1)::Vector{Int64}
│ Main.push!(%3, %4)
│ (last = Main.pop!(v1))
│ Main.println("OK")
│ Main.println("No problem!")
└── return Main.nothing
The appendix amounts to “JET.jl static analysis does not work when we ask Julia to be dynamic”. That’s fair.
However, we also have tools to check for dynamic types and it may be possible to refactor to avoid them. In this specific case, after noting the differences between how Rust and Julia do type inference, we could actually refactor the Julia code to infer the type of v1 to be a Vector with a narrower element type.
One capability that is missing is the ability to detect “red” in @code_warntype as part of testing.
julia> function code_errortype(f, types; optimize::Bool=false, debuginfo::Symbol=:default, kwargs...)
for (src, rettype) in code_typed(f, types; optimize, kwargs...)
p = src.parent
nargs::Int = 0
if p isa Core.MethodInstance
p.def isa Method && (nargs = p.def.nargs)
end
if src.slotnames !== nothing
slotnames = Base.sourceinfo_slotnames(src)[nargs+1:end]
non_dispatchable = (!Base.isdispatchelem).(src.slottypes[nargs+1:end])
if any(non_dispatchable)
msg_io = IOContext(IOBuffer(), :color => true)
println(msg_io, "Non-dispatchable slot type detected. Consider improving type inference:")
for (nd, name, type) in zip(non_dispatchable, slotnames, src.slottypes[nargs+1:end])
if nd
print(msg_io, " ", name)
Base.emphasize(msg_io, "::$type")
println(msg_io)
end
end
error(String(take!(msg_io.io)))
end
end
end
end
code_errortype (generic function with 1 method)
julia> function test()
v1 = [] # Vector{Any}
v2 = [1.0]
push!(v1, v2) # v1 is still Vector{Any}
last = pop!(v1) # Any
println("OK") # OK
println(last.pop()) # Runtime error 💥
println("No problem!") # Not reachable
nothing
end
test (generic function with 1 method)
julia> code_errortype(test, Tuple{})
ERROR: Non-dispatchable slot type detected. Consider improving type inference:
last::Any
Stacktrace:
[1] error(s::String)
@ Base .\error.jl:35
[2] code_errortype(f::Function, types::Type; optimize::Bool, debuginfo::Symbol, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Main .\REPL[256]:21
[3] code_errortype(f::Function, types::Type)
@ Main .\REPL[256]:1
[4] top-level scope
@ REPL[258]:1
Some questions for the reader:
- Does a function such as
code_errortypealready exist? - Do you have any suggestions how to improve this detection function?
- Should the presence of
Vector{Any}also be reported as an issue? - Does a return type of
Union{}indicate that the method always throws?