Type instability in closure after reassigning a variable

I’ve stumbled recently on this behavior (julia 1.3.1):

function test()
    s = "hello"
    s = lowercase(s)
    x() = s
    x()
end

and @code_warntype test() yields

Variables
  #self#::Core.Compiler.Const(test, false)
  s@_2::Core.Box
  x::var"#x#65"
  s@_4::Union{}

Body::Any
1 ─       (s@_2 = Core.Box())
│         Core.NewvarNode(:(x))
│         Core.setfield!(s@_2, :contents, "hello")
│   %4  = Core.isdefined(s@_2, :contents)::Bool
└──       goto #3 if not %4
2 ─       goto #4
3 ─       Core.NewvarNode(:(s@_4))
└──       s@_4
4 ┄ %9  = Core.getfield(s@_2, :contents)::Any
│   %10 = Main.lowercase(%9)::Union{AbstractChar, String}
│         Core.setfield!(s@_2, :contents, %10)
│         (x = %new(Main.:(var"#x#65"), s@_2))
│   %13 = (x)()::Any
└──       return %13

It seems that offending line is s = lowercase(s), because this versions are type stable

function test2()
    s = "hello"
    s2 = lowercase(s)
    x() = s2
    x()
end

function test3()
    s = "hello"
    s = lowercase(s)
    x(z) = z
    x(s)
end

function test4()
    s = "hello"
    x() = s
    x()
end

Can anyone explain why is it happening? And is there any rule of thumb how to proceed in situations like this? May be something like “do not reassign variables”, “if variable is reassigned, specify it as an argument to closure”.

It’s not a problem to patch my real world function with anything like test2, test3 approach, but it’s more interesting to understand what is going on.

1 Like

Sounds like you’re hitting https://github.com/JuliaLang/julia/issues/15276.

1 Like

The manual has a discussion about this: https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured-1.

4 Likes

Thank you, definitely I should have studied manual better. Somehow I didn’t connect this problem with the one described in manual, but now it is obvious.

Maybe you can improve the manual?

1 Like

Maybe changing “Performance of captured variable” to something like “Performance of captured variables in closures” makes sense (to get the closure keyword in there)?

2 Likes

To be honest I am kind of scared to change anything in the manual :slight_smile: It was written by smart people who know what they are doing. I guess it was my fault, should have paid more attention.

Well, you’re possibly the smartest right now about how this section could be improved. The PR will be reviewed, so what could possibly go wrong…

3 Likes