If b = a, when do changes in a affect b?

a = 2 * a is not modifying the array originally referenced by the variable a, it’s re- assigning a to a new array.

Certain operations mutate, others don’t. You might want a .= 2 .* a, which will do element-wise operations

Edit: beat me to it!

4 Likes

If George is my brother and my brother gets a haircut, when does George get his haircut?

6 Likes

Because what you observe is a direct consequence of the following simple rules, each of which should be pretty unquestionable on its own:

  • Every variable is a reference to a chunk of memory somewhere on your machine.
  • a = b makes a point to the same chunk of memory as b.
  • a[1] = 123 writes the number 123 into the first 64 bits of the chunk of memory pointed to by a (because Julia specifies that Int == Int64 is to be represented using 64 bits).
  • 2*a allocates a new chunk of memory of the same size as that pointed to by a, and writes what you would expect into this new chunk of memory.
6 Likes

This is not specific to Julia. For example, Python too behaves this way:

In [95]: a = np.array([1,2])

In [96]: b = a              

In [97]: a[0] = 4           

In [98]: a                  
Out[98]: array([4, 2])

In [99]: b                  
Out[99]: array([4, 2])

In [100]: a = 2*a           

In [101]: a                 
Out[101]: array([8, 4])

In [102]: b                 
Out[102]: array([4, 2])

From the photo it might be only in September. :face_with_hand_over_mouth:

3 Likes

Here a resume of previous answers that appeared here:

https://m3g.github.io/JuliaNotes.jl/stable/assignment/

3 Likes

How do functions fit into this picture? If I say

f(x) = x
g = f

then any method added to f also affects g,

f(x, y) = x+y
g(1, 2)           # gives 3

but adding methods to g is not allowed:

julia> g(x, y, z) = x+y+z
ERROR: cannot define function g; it already has a value

So it seems that in this case g is not an alias for f.

1 Like

Yeah, functions are a bit weird in this case. The thing to know is that defining a function like f(x, y) = ... or function f(x, y) ... end implicitly makes f const when at global scope. To make g an alias for f which you can add methods to at global scope, then it needs to be const too:

julia> f(x) = x
f (generic function with 1 method)

julia> const g = f
f (generic function with 1 method)

julia> g(x, y, z) = x + y + z
f (generic function with 2 methods)

I don’t actually know if there’s a way to do this not at top-level scope.

2 Likes

So the situation is quite subtle: We have

f(x) = x
g = f
g(x, y) = x+y  # ERROR: cannot define function g; it already has a value

and

f(x) = x
const g = f
g(x, y) = x+y  # adds a new method to both f and g

as well as

function a()
    f(x) = x
    g = f
    g(x, y) = x+y
    return g  # returns a function with a single method (for two args)
end

Maybe this carries this thread too far away, but what is the rationale behind these different behaviours?

I think that what is happening here is the mixture of two things: scopes and function definitions.

If I am not mistaken (maybe I am), functions receive permanent global labels in the global scope, independently if they are defined inside or outside a function. Thus, your first f(x)=x will create a global constant label (something like "#f#2"). When you do g=f you are binding the local variable g to f. Finally, when defining g(x,y) you are defining a new global constant variable g, with a new totally unrelated label, like "#g#4".

This is different from what happens in global scope (where that errors) because if you define f(x) or g(x) in the global scope, the constant labels they receive are actually f and g in that scope, thus you cannot redefine those labels anymore. Inside the function the labels can be anything (nobody can interact with them anyway), thus they do not overlap with the local variable named f.

Thanks for the explanation. Is there any reason that it should be that way? If I say const g=f in global scope, then g(x,y)=x+y modifies the method table for f (as far as I can tell). Would it be a bad design choice if Julia did the same when I omit the const or when I’m in local scope?

It would disallow binding a label to a function and later to something else, like:

julia> f() = 1
f (generic function with 1 method)

julia> g = f
f (generic function with 1 method)

julia> g()
1

julia> g = "hello"
"hello"

It disallowing that is a good idea or not, I don’t know (but almost certainly that won’t change anytime soon).

From my understanding, there’s no explicit rationale, but this behaviour is a consequence of how “it all works”. Let’s go through it.

1. What does const mean?

At its core, const is a promise to the compiler that the value of a variable won’t change and it’s a binding guarantee that the type of that same variable can’t change. You can see this for yourself:

julia> const a = 1                                                                                   
1                                                                                                    
                                                                                                     
julia> typeof(a)                                                                                     
Int64                                                                                                
                                                                                                     
julia> a = 2                                                                                         
WARNING: redefinition of constant a. This may fail, cause incorrect answers, or produce other errors.
2                                                                                                    
                                                                                                     
julia> a                                                                                             
2                                                                                                    
                                                                                                     
julia> a = 3.0                                                                                       
ERROR: invalid redefinition of constant a                                                            
Stacktrace:                                                                                          
 [1] top-level scope                                                                                 
   @ REPL[5]:1                                                                                       
                                                                                                     
julia> typeof(a)                                                                                     
Int64                                                                                                

Julia lets you change the value of a (but warns that this may affect existing/compiled code), but not its type.

2. Functions and constness

In julia, every function has its own type:

julia> sin(1)                    
0.8414709848078965               
                                 
julia> cos(1)                    
0.5403023058681398               
                                 
julia> typeof(sin), typeof(cos)  
(typeof(sin), typeof(cos))       
                                 
julia> typeof(sin) == typeof(cos)
false                            

julia> isconst(Base, :sin), isconst(Base, :cos)
(true, true)                                   

As a consequence of functions having their own type and functions being constant, you quite literally can’t assign a new function (with a different method table, one being !== to the old one) to a binding that already is const and pointing to a function, since the type of the binding would have to change.

What if we could do that, i.e. that binding wasn’t const? Well, the compiler wouldn’t have the guarantee that the variable/function didn’t change implementation under its nose, while compiling/running other code. It would have to insert more general lookups for every call, even those that don’t change, instead of being able to cache and inline existing compiled code. As you can imagine, this kills performance and a whole bunch of optimizations that rely on code being inlined, unrolled and subsequently eliminated/replaced by faster equivalents.

3. What about other bindings?

If we now introduce a new non-const binding g like you’ve done, we’ve basically got the same situation as described above. At any point, g might point to a new function, thus change its type or it might point to no function at all (though it would still be callable, since everything in julia is callable). So when code uses g, the compiler has to insert checks for each access, simply because it might change type! It’s the same reason any global you use should be const in the first place. Because of this restriction though, you can’t add methods to the function that g is pointing to (let’s call that f), since g is not f at all, it just gives a very unstable direction that, at the moment, points to f.

If you now make g a constant in the first place, the trouble goes away - it can’t readily be distinguished from f, since its type can’t change and functions (or rather, the mathod table associated with the type of the function) only have one instance (otherwise we’d be back at non-constness and lookups everywhere). Thus, it’s possible to create new methods for “f” (which is also “g”).

4. How does local scope play into this?

In your local scope example, f(x) = x and g(x, y) = x+y are two inner functions. In order to be able to use those inside of a, julia does a little trick: it moves those two functions outside of a and compiles them as part of the dependency chain of compiling a. In order to avoid name clashes with functions outside of a, they’re given a generated name, which is what’s being used in a internally.

julia> module Example                                                       
       f(x,y) = x*y                                                          
       
       function a()                                                          
           f(x) = x                                                          
           g = f                                                             
           g(x, y) = x+y                                                     
           return g
       end                                                                   
       end                                                                   
Main.Example                                                                
                                                                             
julia> names(Example, all=true)                                             
11-element Vector{Symbol}:                                                   
 Symbol("#a")                                                                
 Symbol("#eval")                                                             
 Symbol("#f")                                                                
 Symbol("#f#1")            # I'm generated!
 Symbol("#g#2")            # and so am I!
 Symbol("#include")                                                          
 :Example                                                                   
 :a                                                                          
 :eval                                                                       
 :f                                                                          
 :include                                                                    

But here’s something you might not have expected:

julia> x = Example.a()                                     
(::Main.Example.var"#g#2") (generic function with 1 method)

Why is there only one method? Let’s investigate with @code_lowered:

julia> @code_lowered Example.a()           
CodeInfo(                                   
1 ─     f = %new(Main.Example.:(var"#f#1"))
│       g = f                               
│       g = %new(Main.Example.:(var"#g#2"))
└──     return g                            
)                                           

So our intuition was correct - both function definitions are hoisted out of a as regular anonymous functions (which are implemented as structs with no fields in this case, since there’s no captured state) and in place of the function we return an instance of that anonymous function (which has the method table attached). Since g-the-function doesn’t have anything to do with f-the-function and the variable g is not a constant (which don’t exist in local scope anyway), it’s just a regular rebinding of a variable and no method is “added” to f-the-function at all.


All in all, it’s important to remember that = is not mathematical equivalence, it’s an assignment.

7 Likes

Thanks a lot for your detailed post! It clarifies a lot. Let me make a comment about the last part first.

Local scope

Your explanation is very interesting and exactly the kind of rationale I was looking for. It seems that Julia extracts and combines all local method definitions this way, no matter where they occur inside the function. For example, if I say

function b(n)
    f() = 1
    if n == 1
        f(x) = x
    end
    return f
end

then b always returns a function with two methods, no matter what n is:

julia> b(0)
(::var"#f#1") (generic function with 2 methods)

I guess that this is not what most people would expect. Is this behavior mentioned anywhere in the documentation?

Global scope without const

Here I have a follow-up question. As you explain, if I say g = f without const, then there is a performance penalty whenever I use g. But why should this prevent a subsequent method definition like g(x,y)=x+y from changing f? That f is const doesn’t stop me from modifying its method table, and then Julia has to decide whether to recompile code or not. What difference would it make if I could change the method table of f not only directly, but indirectly via g?

Function assignment and ===

The following additional aspect occurred to me: If f is a function and I say g = f, then f === g holds. According to the documentation, x === y means that

x and y are identical, in the sense that no program could distinguish them.

Is this still true in this case, given the differences between using f or g?

2 Likes

I had some other stuff going on, so I couldn’t answer right away, sorry!

Yes - lowering happens before runtime, at which point n is not known and since both cases refer to the same function with no ambiguity, it’s somewhat harmless to define it. There is an issue on github somewhere similar to this, but I can’t find it right now, sorry :confused:

This gets a little technical, but the short story is that you’d circumvent the need for eval (i.e. returning to global scope) since you could use g passed into a function h to define a method at runtime and immediately use that function through f. To my understanding, this is not allowed - each method for a function has a “world age” in which it is defined and it can only be used in subsequent world ages.

For a longer/more indepth explanation, I have to refer you to this wonderful paper reviewing the world age mechanism :slight_smile:

That is a very interesting observation! I’m unsure whether this is a bug though, because I can’t think of a way to distinguish the two without trying to define a new method :thinking: E.g. even nameof “lies” to you, since you’re querying f all along:

julia> f(x) = x                   
f (generic function with 1 method)
                                  
julia> g = f                      
f (generic function with 1 method)
                                  
julia> const h = f                
f (generic function with 1 method)
                                  
julia> nameof(g), nameof(h)       
(:f, :f)                          

I think the crucial difference is that for almost all intents and purposes, g and f truly are indistinguishable, except when trying to create a new binding with the same name. It’s just that the syntax z(x) = x (by design) creates a new constant variable z bound to a function of the same name or adds a method to an existing one of the given name in the current top level scope. Notably, this does not try to read any value from z - methods and functions are defined based on types, not values (yes, callable structs are a thing, but that isn’t any different - the method table is chosen on the type of the struct, not its value, even though the value is available inside of the method).

That said, if you manage to come up with a function j(x) such that it returns a boolean value indicating whether or not x is the real f or the non-const g (perferably without, maybe with modifying the global method table?), it’d certainly be an interesting bug report :slight_smile:

1 Like

Again many thanks for your explanations!

I guess that there’s no function that can distinguish between the functions f and g. I don’t know, however, if that’s actually necessary. The doc string for === talks about a “program”, and that seems to be used for any sort of Julia code in the documentation. In this Stack Overflow post, @StefanKarpinski writes that

x === y is true when two objects are programmatically indistinguishable – i.e. you cannot write code that demonstrates any difference between x and y

To me this suggests that once you have f === g, then all subsequent Julia code should give the same results, no matter if previously I first defined the function f and then said g = f or the other way around. But trying to define another method for, say, f shows a difference.

What Stefan says is right, he just is reasoning about objects/values not about bindings.

When you declare a new method you are not only dealing with values, but with bindings also. If you do it in global scope you get:

julia> f(z) = z + 1
f (generic function with 1 method)

julia> g = f
f (generic function with 1 method)

julia> g(z, a) = z + a
ERROR: cannot define function g; it already has a value
Stacktrace:
 [1] top-level scope at none:0
 [2] top-level scope at REPL[3]:1

What is related to bindings, the fact the binding has a values is just incidental to the fact it already exists.

2 Likes

Yikes, it seems there aren’t just side-effects in case of functions in global scope, but for types as well, due to constness Just ran into an unexpected case where using assignment to create aliases causes allocation:

$ cat t_alloc_vec3.jl
using BenchmarkTools

struct vec3
    x::Float64
    y::Float64
    z::Float64
end

# If not marked const: will allocate!
alias = vec3

# If marked const: no allocation
const const_alias = vec3

@btime vec3(0,0,0)
@btime alias(0,0,0)
@btime const_alias(0,0,0)

$ j t_alloc_vec3.jl
  0.014 ns (0 allocations: 0 bytes)
  146.272 ns (1 allocation: 32 bytes)
  0.014 ns (0 allocations: 0 bytes)

This is a case of using a non constant global variable, which should be avoided in performance critical code.

2 Likes

Right, makes sense. I guess I was expecting having a variable referencing a type would not get impacted, although I didn’t really think it through.

1 Like