Is it okay that, in Julia, code that never executes can change the result of programs?

[sadish-d]

Consider the following code. The x expression right after the inner let block throws an error because x is not defined in that scope. This is expected given the scoping rules inside let end block.

let
    let
        x = 0
    end
    x # throws "ERROR: UndefVarError: `x` not defined"
end

Now I modify the above and add some code in the outer let block-- code that never actually executes.

let
    let
        x = 0
    end
    x # does this throw "ERROR: UndefVarError: `x` not defined" ?
    if false
        @assert false # just to demonstrate that this block never executes
        x = 1 # try and comment this out
    end
end

let
    let
        x = 0
    end
    x # does this throw "ERROR: UndefVarError: `x` not defined" ?
    return
    @assert false # just to demonstrate that this code never executes
    x = 1 # try and comment this out
end

In the first of the two modified versions, the code in the if block never executes, and in the second modified version, the code after the return never executes. Yet the previous UndefVarError is now gone. If you comment out the x = 1 expression, you get the error back.

Apparently, it doesn’t matter that x = 1 never executes because “scoping rules are static and not path sensitive”. see: unreachable code interacts with scoping rules and changes program behavior · Issue #56874 · JuliaLang/julia · GitHub

There are other examples, though the underlying reasons behind the behavior here might be different (see: implement local const · Issue #5148 · JuliaLang/julia · GitHub ):

function f()
    if true
        _f() = nothing
    else
        @assert false # just to demonstrate that this block never executes
        _f() = 0
    end
end
g = f()
@assert g() == 0

My questions:

• What does “static” mean?
• Is Julia two languages-- a static one and a non-static one? And does it mean that when one reads Julia code, one should read the static meaning of the code first and go over the same code to with a non-static perspective?
• Is it just Julia or is this an issue in other languages as well? I don’t know much Rust, but I tried to get into confusing scoping rules with it, and the language makes it hard because it makes variable shadowing very clear with explicit let statements. Scopes are clearly delimited with {} and variables can’t go into a scope and get out of it alive. Basically, if you read the code sequentially, you can figure out what it does.

[greatpet]

Here are three variations of your first example. Version 1:

let
    let
        x = 0
        # x is not a variable in the parent scope and only
        # lives in this `let` block
    end
    x # throws "ERROR: UndefVarError: `x` not defined"
end

Version 2 curing the problem with a local declaration:

let
    local x
    let
        x = 0 # refers to variable x from parent scope outside this `let` block
    end
    x # returns 0
end

Version 3, i.e. with “code that never executes”, is equivalent to an implicit local declaration:

let
    let
        x = 0
    end
    if false
        x = 1
        # assignment statement (inside or outside `if`) has the same effect
        # as `local x` in the previous version
    end
    x # returns 0
end

So in summary, the “code that never executes” has the same effect as a declaration statement. The latter also doesn’t explicit “execute” but changes the meaning of the code.

[Mason]

@sadish-d

• What does “static” mean?

Static here means that the parsing / scoping rules don’t depend on the actual dataflow, or values. How a piece of code is parsed depends only on the literal string of code being parsed.

@sadish-d

• Is Julia two languages-- a static one and a non-static one? And does it mean that when one reads Julia code, one should read the static meaning of the code first and go over the same code to with a non-static perspective?

No, julia is one language. Its parsing is static (like almost every single non-lisp language I know of), and its types are dynamic. These are two different, mostly unrelated aspects of language design.

[greatpet]

@sadish-d

function f()
    if true
        _f() = nothing
    else
        @assert false # just to demonstrate that this block never executes
        _f() = 0
    end
end
g = f()
@assert g() == 0

According to Julia’s manual, such code may become illegal in the future:

However, you should not define local methods conditionally or subject to control flow, as in

function f2(inc)
    if inc
        g(x) = x + 1
    else
        g(x) = x - 1
    end
end

function f3()
    function g end
    return g
    g() = 0
end

as it is not clear what function will end up getting defined. In the future, it might be an error to define local methods in this manner.

[Benny]

@sadish-d

Basically, if you read the code sequentially, you can figure out what it does.

That has its own “problems”. In C:

#include <stdio.h>
int main()
{
    int x = 0;
    {
        x = 1; // outer x
        printf("x = %d\n", x);
        float x;
        x = 2; // inner x, converts
        printf("x = %f\n", x);
    }
    printf("x = %d\n", x);
    return 0;
}

prints

x = 1
x = 2.000000
x = 1

meaning the inner local scope used the same symbol x for 2 separate variables. Of course, that’s not a problem if you’re just used to the rule and the debugging headaches. You mentioned Rust’s let statement (which for those who are unaware is not a locally scoped block like Julia’s let), those also can make a symbol switch statically typed variables within the same scope. Thankfully, conditional variable declarations can’t make a symbol’s variable ambiguous in a scope because if statements introduce local scope, all {} blocks do.

No need for a scope’s symbol switching variables in a dynamically typed language, in fact it’s actively avoided. Python keeps up the appearance of sequential execution by forcing you to write the global/nonlocal statements before any assignments within the scope, but it’s not necessary for the parser to figure out where variables are. Far more of Julia’s blocks introduce local scope, so explicit declarations like that would be a higher burden; in this example, you’d be forced to put local x after the first let. On the other hand, our if statements get to not introduce local scope, which is pretty convenient and lets us do a lot more in the global scope.

[giordano]

That’s currently hitting a bug:

disallow methods of local functions in different blocks

(reason why that syntax may be disallowed in the future)

[sadish-d]

Even having never written a word of C, this particular example looks straightforward to me. There’s an integer x declared in the outer scope which is accessible from the inner scope. The x is shadowed by another x declared a float. And neither of them outlive their scope, which is demarcated by {}. I don’t have to worry that code I add later will affect the behavior of code written thus far; let alone code that never executes.

I agree that if blocks not introducing scope is more ergonomic. But I imagine that defining variables inside if blocks is a common pattern. And it looks very error-prone to me.

If conditionally defining functions should be illegal, why not make conditionally declaring variables illegal as well? The reasoning might be different but the rule seems cohesive to me.

[Benny]

@sadish-d

If conditionally defining functions should be illegal, why not make conditionally declaring variables illegal as well?

Misunderstandings to clear up:

1. Conditionally declaring variables isn’t typically illegal in any language. I said a variable declaration would be ineffective by itself in an if-statement that introduces local scope, that is all its assignments must conditionally occur in the scope as well.
2. Conditional functions are not at all illegal in Julia or many other dynamic languages, the Github issue shows you how.
3. Multimethods is already unusual for a dynamic language, and the reason for the undefined behavior (if it was outright illegal, it’d error) of conditionally defining methods of the same closure is that implementing the straightforward behavior also kills a lot of optimization. If some genius figures it out one day, it could be supported.

@sadish-d

I don’t have to worry that code I add later will affect the behavior of code written thus far; let alone code that never executes.

Easy, paste code with a variable declaration accidentally reusing a symbol in the middle of a scope. If you like static typing and closures, you’ll hate juggling multiple variables for the same symbol within a scope. If all you want is an isolated bit of code to run, then put it in its own local scope (declare everything in {} or let end, more easily a helper function), but if you want to affect the preexisting variables, then you need to look at what the whole scope does. Pasting without care is dangerous and discouraged anywhere, and no language’s scoping rules can save you from legal bugs.

[sadish-d]

Conditionally declaring variables isn’t typically illegal in any language. I said a variable declaration would be ineffective by itself in an if-statement that introduces local scope, that is all its assignments must conditionally occur in the scope as well.

Declaring a variable conditionally (or declaring and assigning) would not be a problem in languages where if blocks introduce their own scope. So it makes sense that they don’t need to make it illegal. The issue arises precisely because if blocks don’t create new scope in Julia, which interacts with other scoping rules to give us the issue that I bring up.

Easy, paste code with a variable declaration accidentally reusing a symbol in the middle of a scope.

Sure, if I put something in a middle of a scope, I should look at the entire scope to check how I’m using the variable. But Julia’s scope are not clearly demarcated with blocks as other languages do with {}.

but if you want to affect the preexisting variables, then you need to look at what the whole scope does.

I don’t want to “affect preexisting variables”, and that’s precisely my point. “Preexisting” in Julia does not mean appearing ahead in the code.

Say I have:

let
	let
		x = 0
		# ...
		# a whole lot of inner code
		# ...
		# ...
	end
	@assert x == 0 # errors

	# ...
	# a whole lot of outer code
	# ...
	# ...

end

Now, I add an if block at the end of the outer block, completely outside the inner block.

let
	let
		x = 0
		# ...
		# a whole lot of inner code
		# ...
		# ...
	end
	@assert x == 0 # does not error

	# ...
	# a whole lot of outer code
	# ...
	# ...

	if false
		local x
	end
end

Here, the x in the if block is the “preexisting” variable because it is declared in the outer scope.

In Julia, I have to check not just whether the variable already exists in the outer code, but also in the inner code. I don’t like it. Maybe someday I’ll learn to live with it. But that’s still not the issue I’m raising here (though I’ve complained about it before). And yes, it’s good advice to keep your blocks small, put things in functions. I admit all of those.

My issue is that code inside an if false ... end affects what the program does. And any way I look at it, this is just wrong. And I am asking specifically whether or not I am correct in thinking so.

[Benny]

@sadish-d

because if blocks don’t create new scope in Julia
…
My issue is that code inside an if false ... end affects what the program does. And any way I look at it, this is just wrong.

There are plenty of languages where if statements don’t introduce new scope, and in your opinion all those languages are “wrong”. You’re entitled to your opinion, most people just don’t write intentionally dead code and expect nothing to go awry.

@sadish-d

But Julia’s scope are not clearly demarcated with blocks as other languages do with {}.

What do you mean? We have block expressions with ends, and only a couple don’t introduce new scope. That’s as explicit as it gets.

@sadish-d

I don’t want to “affect preexisting variables”

Then don’t edit that scope, leave it alone. That’s true for any language, you have to take care to change or not change the preexisting code. The only syntactic mechanism that lets you insert code with less (not zero!) risk is referential transparency, but you can’t reassign variables.

[sadish-d]

I created this post because I am interested in having informed opinions.

There are plenty of languages where if statements don’t introduce new scope, and in your opinion all those languages are “wrong”.

No, that is not my opinion. My opinion is that it is incorrect behavior for programming languages to let dead code change what a program does.

most people just don’t write intentionally dead code and expect nothing to go awry.

The intentionally dead code is a simplification of actual bugs that I (and probably others) could easily write if they’re not aware of the issue. I was presenting a minimum working example, as is encouraged on this forum.

I’m not much of a programmer, and I do not have much exprience with other languages. Which is why in my initial post, I asked whether similar issues arise in other programming languages, and if so, how they deal with them. @Benny, you gave an example with C, which if anything shows that the issue I raise is not of concern in C.

Then don’t edit that scope, leave it alone.

That’s the thing. I would say I’m not editing the scope: I put the variable declaration behind a condition that can never hold. And @Mason’s told me that that won’t work because the parser still sees the variable declaration. That’s an explanation of why the code behaves the way it does. It does not answer my question: is the behavior correct?

[Benny]

@sadish-d

No, that is not my opinion. My opinion is that it is incorrect behavior for programming languages to let dead code change what a program does.

That’s exactly what I was referring to. That happens in other languages, albeit in other ways. They tend to error and force you to declare earlier, which I already mentioned could have been done in Julia but isn’t necessary for the parser to figure out variable scoping, in fact the parser would need to do that in order to detect said error.

@sadish-d

you gave an example with C, which if anything shows that the issue I raise is not of concern in C.

Right, I wasn’t reproducing your example at all, I was talking about the costs of something else you brought up, multiple variable declarations working in order in the same scope.

@sadish-d

I would say I’m not editing the scope

Then you’d be wrong because you’re writing code into the scope, it’s exactly how it sounds.

@sadish-d

It does not answer my question: is the behavior correct?

Correct as in follows the documented scoping rules to the letter, then yes. Correct as in whether the rules should be like that, that’s up to your opinion. I don’t entirely approve of the scoping rules either in other ways. Julia v1 can’t safely change the rules, and Julia v2 isn’t actively planned, so whatever our opinion, we can only make peace with the rules to use the language.

[lmiq]

@sadish-d

My issue is that code inside an if false ... end affects what the program does. And any way I look at it, this is just wrong. And I am asking specifically whether or not I am correct in thinking so.

I agree that that looks bad. In practice, though, I don’t see myself having issues with this.

[jwortmann]

@sadish-d

My issue is that code inside an if false ... end affects what the program does.

As your main point seems to be about the behavior of “dead code” (if false), consider the example modified like this

let
    let
        x = 0
    end
    x
    if rand() < 0.5
        x = 1
    end
end

Now you don’t know whether x = 1 actually runs or not. But when the execution reaches the line with the single x, Julia still needs to know whether it should throw an UndefVarError or not. How could it do that at the time when it doesn’t yet know whether the condition in the next line evaluates to true or false? You could argue that a literal false in the condition could be treated specially, but then consider a function call in place of the if-condition, where this function directly returns a literal false. Should it still be considered as “dead code” in this case? Or imagine the function is complicated or runs for a long time before it returns a value - then it would become very intransparent when reading the code and examining its behavior.

(That said, I agree that compared to other languages, variable scoping rules in Julia are quite complicated with their three orthogonal concepts global vs. local, soft vs. hard scope and interactive vs. non-interactive context.)

[mihalybaci]

let
    let
        x = 0
    end
    x
    if rand(Bool)
        x = 1
    end
end

This works to return either trueor false directly.

[sadish-d]

@jwortmann, in your example, I would expect the code to error half the time. Just as this code does:

if rand() > 0.5
    error()
end

I used dead code to simplify the example for the sake of demonstration. That sort of code is rare in practice. And I’m not suggesting checking for false literal or anything of that sort. Before saying how language should change, I wanted to establish whether it should. It should only change if it is currently incorrect.

I’ve heard Julia developers distinguish between the notion of Julia the language, and Julia the implementation of the language. Maybe that is a distinction to keep in mind here, and perhaps what I was trying to get at when I asked in my initial post whether Julia is two languages.

You say

Julia still needs to know whether it should throw an UndefVarError or not.

I wonder if the Julia you refer to is Julia the implementation of the language, and not Julia the language. In other words, could say this instead?

Given the current implementation of Julia, it needs to know at parse time whether it should throw UndefVarError.

There have been some comments about how Julia is implemented: it is the parser that is responsible for scoping behavior (i think) and it does not understand the logic of data flow. I appreciate these comments because they help me understand how Julia actually works as implemented currently.

But maybe the heart of my question is whether, in the case of conditionally declared local variables go, Julia the implementation corresponds with Julia the language. It should be the language driving the implementaiton of the language, not the other way around.