Either the def of `local` is wrong, Or the current julia behavior is unexpected

[mlechu]

@WalterMadelim

I don’t know if local x means:

• introduce a new name x, which is required to be a local name in the current scope

or

• We might have owned a name x in the current scope already, and now we declare it to be local.

The first one is closer to accurate. I’m not sure, but I think you’re working with a wrong definition of “declare”; usually “declaring a variable” means introducing one. You can’t “make” a non-local variable local, and I’m not sure what “owned” means.

My most precise description would be “Introduce[1], if not present, a new local variable x in this local scope unless… [2]”

[1] Benny is right that “introduce/declare” could be mistaken for something that happens at runtime, which is false. Every name is resolved to a variable by the end of lowering, and the order of execution of the code doesn’t matter.

[2] There are a couple of checks that the introduced variable doesn’t conflict with an argument, a declared global, or a static parameter (function f() where {x}; end) in this exact local scope. This is because it isn’t clear what the user wants to refer to with that name:

function f(x=1)
    local x = 2
    x # what is this? user probably wanted two variables; throw an error.
end

@WalterMadelim

Otherwise it’s extremely easy for users to write local x = t.a, y = 4, which mixed up with the let syntax

Agree this is unfortunate. This is because both local a = b = c and t.a, y = 4 are valid. I think there are open issues suggesting e.g. requiring parens (t.a, y) to get the tuple-assignment parsing, but we would need to figure out how to not break existing code.

[Vasily_Pisarev]

After testing a few things, I think local is either under-documented or under-designed.
For example: let; local x::Int; local x; x=5; end is valid and let; local x::Int; local x::Any; x=5; end is not. I can see the rationale for the latter, but then local x and local x::Any have different semantics.

It feels like no one has thoroughly thought out a consistent semantics of what should happen if local x appears multiple times in the same scope or is applied to a name that is already inferred as local for other reasons. In other words, I would just consider double declaration as local an UB until the semantics is documented and not bother trying to reverse-engineer what the present semantics is.

[WalterMadelim]

@mlechu

usually “declaring a variable” means introducing one

Okay it’s usually not always.

@mlechu

and I’m not sure what “owned” means.

let x # already owned
    local x # now declare it to be local
end

@mlechu

My most precise description would be “Introduce[1], if not present, a new local variable x in this local scope unless… [2]”

Okay, then probably the def of the current ?help local needs to be improved.

@mlechu

[2] There are a couple of checks that the introduced variable doesn’t conflict with an argument, a declared global, or a static parameter (function f() where {x}; end) in this exact local scope. This is because it isn’t clear what the user wants to refer to with that name:

I deem it not resonable. Others may not either. Because of inconsistency

@affans

For consistency, perhaps the first example should not throw an error since you can do

@mlechu

function f(x=1)
    local x = 2
    x # what is this? user probably wanted two variables; throw an error.
end

I think this should be deemed normal, nor ERROR. There is only one x (or two) here, which are both local. This is inconsistent with

let x = 1
x = 2
local x
end

[WalterMadelim]

Sorry friends. I have some other things to do.

I’ll be back when I have free time.

[Benny]

@Vasily_Pisarev

then local x and local x::Any have different semantics.

Well, yes, local x doesn’t declare the variable to have a specified type to check and convert right-hand sides of assignments, so it doesn’t actually conflict with another local x::Int that does. Maybe you’re thinking of composite types’ fields and methods’ arguments being specified to default to ::Any, but those doesn’t imply local/global declarations do.

@WalterMadelim

@mlechu

function f(x=1)
    local x = 2
    x # what is this? user probably wanted two variables; throw an error.
end

I think this should be deemed normal, nor ERROR.

These are both fair opinions with tradeoffs, the erroring one just made it into the implementation. I’d actually prefer that the language didn’t allow multiple local declarations per scope level, but fixing that would be a breaking change. For what it’s worth, removing this particular error would not be a breaking change because every previous written program works normally.

[Vasily_Pisarev]

@Benny

Well, yes, local x doesn’t declare the variable to have a specified type to check and convert right-hand sides of assignments, so it doesn’t actually conflict with another local x::Int that does.

Yes, but the compiler could technically deduce that if T<:S, then local x::S does not conflict with local x::T even though both do conversions. In that case, there could be implicit ::Any.
My point is not that the current choice is unreasonable, it’s that the behavior of double local declaration is hardly explainable by any reason but “at some point some of the devs decided this case should be handled that way”. Like, why (a) does effectively create a new variable shadowing the argument and (b) just errors out?

# a
julia> function foo(x::AbstractString)
           x::Float64 = parse(Float64, x) + 1
           x
       end;

julia> foo("2")
3.0

# b
julia> function foo(x::Int)
           local x
           x + 1
       end
ERROR: syntax: local variable name "x" conflicts with an argument

[sgaure]

@Vasily_Pisarev

Like, why (a) does effectively create a new variable shadowing the argument and (b) just errors out?

I agree that the logic is somewhat convoluted, but in (a) no new variable is created. It’s just stated that in the scope of the function body, assignments to x should be converted to Float64. In (b) it can be argued that there’s a chance there’s a user error.

I.e. in (a), every

x = sth

will effectively be altered to

x = convert(Float64, sth)::Float64

See e.g.:

struct My
    a::Int
end
Base.convert(::Type{Float64}, x::My) = x.a

function foo(x::My)
    y::Float64 = x
end

julia> foo(My(1))
ERROR: TypeError: in typeassert, expected Float64, got a value of type Int64

[Benny]

@Vasily_Pisarev

why (a) does effectively create a new variable shadowing the argument

Since you made this example after talking about conflicting variable type declarations, I’ll draw your attention to this:

@Sukera

Contrary to popular believe, ::Int in method signatures are NOT type assertions, they are (almost exclusively) relevant for selecting methods during dispatch.

So your (a) example only imposed 1 type declaration on x, and its apparent change from a subtype of the method argument annotation is allowed in a dynamically typed language. Conflicting type declarations or a local declaration with the same name as a method argument are entirely different issues.

@Vasily_Pisarev

Yes, but the compiler could technically deduce that if T<:S, then local x::S does not conflict with local x::T even though both do conversions. In that case, there could be implicit ::Any.

The lowerer could reconcile multiple type declarations, but it doesn’t. It’s such a code smell, it just errors.

julia> let;
       local x::Int
       local x::Any
       x = 1
       end
ERROR: syntax: multiple type declarations for "x"

Even if multiple subtype declarations were allowed, that doesn’t imply implicit ::Any declarations need to happen either. They certainly don’t now:

julia> Meta.@lower let;
       local x
       x = 1
       end
:($(Expr(:thunk, CodeInfo(
    @ REPL[3]:3 within `top-level scope`
1 ─     x = 1
└──     return 1
))))

julia> Meta.@lower let;
       local x::Any
       x = 1
       end
:($(Expr(:thunk, CodeInfo(
    @ none within `top-level scope`
1 ─       Core.NewvarNode(:(x))
│   @ REPL[4]:3 within `top-level scope`
│         @_2 = 1
│   %3  = @_2
│   %4  = Any
│   %5  = %3 isa %4
└──       goto #3 if not %5
2 ─       goto #4
3 ─ %8  = Any
│   %9  = @_2
│   %10 = Base.convert(%8, %9)
│   %11 = Any
└──       @_2 = Core.typeassert(%10, %11)
4 ┄ %13 = @_2
│         x = %13
└──       return 1
))))

[Vasily_Pisarev]

@sgaure

I agree that the logic is somewhat convoluted, but in (a) no new variable is created. It’s just stated that in the scope of the function body, assignments to x should be converted to Float64. In (b) it can be argued that there’s a chance there’s a user error.

Well, @code_warntype shows something interesting (and something I didn’t notice earlier).

1. Arguments and Locals are distinct blocks
2. A new variable (“a local”) is created.
3. All assignments to it do go through convert…
4. … except the first implicit one x (variable) = x (argument).

Tbh, looks like a bug.

Also, notably, (2) happens implicitly any time we re-assign function argument to something else.
And there is also a clue to why (b) is an error. local x creates a variable x which is distinct from argument x. But now, argument x becomes inaccessible. Now, why not create an implicit variable initialized with the argument value? I don’t know. Maybe nobody complained about that behavior, so nobody cared to revise it.

[Benny]

@Vasily_Pisarev

Well, @code_warntype shows something interesting (and something I didn’t notice earlier).
…
Tbh, looks like a bug.

@code_warntype is showing lowered code, which is an implementation detail, not the language. It’s allowed to do anything as long as it accomplishes the language-level program. So this:

@Vasily_Pisarev

A new variable (“a local”) is created.

isn’t true on the language level. It’s often close enough you can figure out some ways to improve the source code, but mentally reversing the lowering is still a chore.

You can access implementation details at runtime with varying difficulty e.g. Core.Box fields, reflection, pointer magic. That doesn’t mean they describe the language, and they’re free to change across any Julia versions to improve performance.

[WalterMadelim]

@sgaure

but in (a) no new variable is created.

At least from the code_warntype viewpoint, something new is created, which is highly unintuitive.
This is why I suggest in another thread that we typically should not reassign to a name which is already an argument. This would make things extremely awful…

---

Anyway, these problems don’t exist if we at the user level avoid reassignment to an argument.

[WalterMadelim]

One situation to be wary of

function f(x)
    for i = 1:2
        if i == 1
            x = Ref(i)
        else
            x.x += 1
        end
    end
end
function f()
    for i = 1:2
        if i == 1
            x = Ref(i)
        else
            x.x += 1
        end
    end
end

[sgaure]

@WalterMadelim

@sgaure

but in (a) no new variable is created.

At least from the code_warntype viewpoint, something new is created, which is highly unintuitive.

It does not make sense to examine lowered code to “prove” that “something new is created”. Variables in julia are just labels with defined semantics. How they appear in lowered code, be it as seen in Meta.@lower, @code_lowered, @code_typed, @code_warntype, @code_llvm or @code_native, has no bearing on this. Even entire loops can disappear lower down:

function sumN(N)
    local s = 0
    for i in 1:N
        s += i
    end
    s
end

The function sumN(N::Int) compiles to basically three instructions, addition by one, multiplication, and shift right by one bit (N*(N+1) ÷ 2), though with some provisions for overflow and negative N. Based on this, we could claim that neither s, nor i, nor 1:N is “created”. On the other hand, sumN(N::Float64) compiles to an entire vectorized mouthful.

julia> @btime sumN(1_000_000_000)
  2.204 ns (0 allocations: 0 bytes)
500000000500000000

Likewise, in

function fun(N)
    local s = N
    for i in 1:50
        s += i^2
    end
    s
end

fun(N::Int) compiles to N + 42925. No s, no i, no squaring, no loop.

julia> @code_llvm fun(1)
; Function Signature: fun(Int64)
;  @ REPL[12]:1 within `fun`
define i64 @julia_fun_6586(i64 signext %"N::Int64") #0 {
top:
  %0 = add i64 %"N::Int64", 42925
;  @ REPL[12]:5 within `fun`
  ret i64 %0
}

Whereas, for fun(::Float64), the loop is unrolled:

julia> @code_llvm fun(1.0)
; Function Signature: fun(Float64)
;  @ REPL[17]:1 within `fun`
define double @julia_fun_6603(double %"N::Float64") #0 {
top:
;  @ REPL[17]:4 within `fun`
; ┌ @ promotion.jl:433 within `+` @ float.jl:495
   %0 = fadd double %"N::Float64", 1.000000e+00
   %1 = fadd double %0, 4.000000e+00
   %2 = fadd double %1, 9.000000e+00
   %3 = fadd double %2, 1.600000e+01
   ...
   %49 = fadd double %48, 2.500000e+03
   ret double %49
}

One could claim that the %1, %2, etc. is s1, s2 etc, i.e. that a new s variable is “created” for each iteration of the loop. This is a consequence of the “single assignment” format. However, when stuff is further translated to native code, it’s just a single register which is updated.

Unless you do @fastmath to loosen up the IEEE float semantics:

function fun(N)
    local s = N
    @fastmath for i in 1:50
        s += i^2
    end
    s
end
julia> @code_llvm fun(1.0)
; Function Signature: fun(Float64)
;  @ REPL[21]:1 within `fun`
define double @julia_fun_6628(double %"N::Float64") #0 {
top:
  %0 = insertelement <4 x double> <double poison, double 0.000000e+00, double 0.000000e+00, double 0.000000e+00>, double %"N::Float64", i64 0
;  @ REPL[21]:4 within `fun`
; ┌ @ fastmath.jl:274 within `add_fast` @ fastmath.jl:167
   %1 = fadd fast <4 x double> %0, <double 8.636000e+03, double 9.200000e+03, double 9.788000e+03, double 1.040000e+04>
   %2 = call fast double @llvm.vector.reduce.fadd.v4f64(double 4.901000e+03, <4 x double> %1)
; └
;  @ REPL[21]:5 within `fun`
  ret double %2
}

(Note that if you add the double constants in there, 8636+9200+9788+10400+4901, you get the old 42925 from the Int version).
It doesn’t make sense to consider various optimizations and transformations to judge what some piece of source code actually “means” in the language.