Why the global variable is not updated even already initialized and included in all modules?

CRquantum · August 20, 2021, 4:17am

Sorry for another stupid question again. First of all, I am not proud of using global variables.
My question is very simple.

Three modules, A, B, and C, they are in A.jl, B.jl, C.jl.
I have defined an array called musigma in C.jl.
musigma is global, now, even if I export it, and include C.jl in all the files, the musigma is still not recognized by all the modules other than C.jl. Why?

C.jl is below:

module C
export mean_covar_init,musigma
mutable struct Mean_covar
    w::Float64
end
const musigma = Array{Mean_covar,1}() # const is global
function mean_covar_init()
    resize!(musigma, 2)
    for k in 1:2
        musigma[k] = Mean_covar(0.0)
        musigma[k].w = 2.0
    end
    println(musigma)
    return nothing
end
end

The function mean_covar_init() just initilize musigma and print it.

B.jl is below

module B
include("C.jl")
using .C
export init
function init()
  println(musigma)
end
end

the function init() just print the musigma from module C.

A.jl is below, it is the main program and it just display the musigma from module B and C,

include("B.jl")
using .B
include("C.jl")
using .C
mean_covar_init()
init()

In Julia, I just run

include("A.jl")

because I know this is a way to run xxx.jl program, and what is show in REPL is

julia> include("A.jl")
Main.C.Mean_covar[Main.C.Mean_covar(2.0), Main.C.Mean_covar(2.0)]
Main.B.C.Mean_covar[]

However, what I expected should be

julia> include("A.jl")
Main.C.Mean_covar[Main.C.Mean_covar(2.0), Main.C.Mean_covar(2.0)]
Main.C.Mean_covar[Main.C.Mean_covar(2.0), Main.C.Mean_covar(2.0)]

I mean, in A.jl,I do

mean_covar_init()

first, so musigma has been initialzied. Since its const and global, its value should have been stored.

Next, in A.jl I did

init()

This function display the musigma from module B.
I expect that this musigma should be the same as the musigma in C. Because they should be the same.

Now, why in Module B, musigma is not recognized as has been initialized?

Could anyone please give some hints, what am I missing?

Thank you very much indeed in advance!

stillyslalom · August 20, 2021, 4:45am

include doesn’t keep track of logical dependencies between files, it pretty much just copy-pastes a file into the module from which it’s called. When you call include("C.jl") from module B, you’re creating a new submodule Main.B.C that’s completely distinct from Main.C. If this is your first package, I’d recommend starting with everything in one unified module (likely split across multiple files to include from the top level). You can always split things into submodules later on if it makes more sense that way, but starting out with module-heavy code is a recipe for logistical pain points, all the more so when you’re relying on global const musigma (which is, as has been said before, a headache waiting to happen).

That said, if you’re deeply attached to the “one file = one module” style of code organization, you might want to try FromFile.jl.

CRquantum · August 20, 2021, 5:00am

Thank you very much indeed!

I will try to look at FromFile.jl, it looks like good but I just tried and it seems does not work, it is method to preventing using include by using @ from instead, but it seems does not keep track of the most up to date values.

Uhm, I still wonder, so in Julia, if I really define some global variables within one module, there is no native way like include or import so that I can access those most up to date values?

Uhm, so what is the use of include and using? Are they just for using functions in some modules?

I mean even if I do not use global variable, I still need put them into in object in the module, say Module C.
But I still need to initialize the object and perhaps update them a little bit in module C.
I need this object to be used in other modules too, like module B, A.
If I cannot get the most up to date object easily, then it will be very inconvenient.
Like, in some module B, I need to call a function in Module C to get that object first, then use the object in Module B. It seems not very convenient.

Sorry again if the question is stupid.

stillyslalom · August 20, 2021, 5:58am

When developing a package, it’s often useful to split the code across several files, each containing tools for one particular task (or set of tasks), then using include to join the files back together in a top-level module. Remember, include is just a fancy way of doing a copy-paste from one file to another, and you can think of include("file.jl") as pasting the contents of file.jl into that spot.

using is a convenient way to load a module and all its exported symbols. It’s woven into Julia’s package management system - when you use Pkg.add("Plots"), for example, that allows Julia to figure out what code needs to be loaded with using Plots.

I’ll return to my earlier recommendation - try to break the Fortran habit of making every file into its own module. Instead, structure your code like this:

A.jl

module A

export MuSigma, MeanCovar
export calculate_covariance

include("types.jl") # contains definition of MuSigma

# best to define global consts in the top-level file
const MUSIGMA = MuSigma() 

include("operations.jl") # contains `calculate_covariance`

end

types.jl

struct MeanCovar
    w::Float64
end

struct MuSigma
    mc::Vector{MeanCovar}
end

function MuSigma(fill = 2.0)
    return MuSigma([MeanCovar(fill) for _ = 1:2])
end

operations.jl

function calculate_covariance(ms::MuSigma)
    ms.mc[1].w + ms.mc[2].w
end

function calculate_covariance()
    global MUSIGMA
    return calculate_covariance(MUSIGMA)
end

Essentially, you’re first defining structs, then describing how they can be constructed, then writing functions that can act upon a given instance of a struct. If you need a global struct, it’s best to initialize it at the top level (A.jl, in your example) after you’ve defined the necessary constructor. You can then use your routines like this:

julia> using A

julia> ms = MuSigma(3)
MuSigma(MeanCovar[MeanCovar(3.0), MeanCovar(3.0)])

julia> calculate_covariance(ms)
6.0

Sukera · August 20, 2021, 6:26am

When you do include("C.jl") in B.jl, you basically end up with this:

module B

# include("C.jl")
module C
export mean_covar_init,musigma
mutable struct Mean_covar
    w::Float64
end
const musigma = Array{Mean_covar,1}() # const is global _to module C only_
function mean_covar_init()
    resize!(musigma, 2)
    for k in 1:2
        musigma[k] = Mean_covar(0.0)
        musigma[k].w = 2.0
    end
    println(musigma)
    return nothing
end
end

using .C
export init
function init()
  println(musigma)
end
end

That is, the code in C.jl is literally pasted into the position of include("C.jl"). As a consequence, if you have both include("B.jl") and include("C.jl") in A.jl, you will have two distinct modules named C - one directly below Main as Main.C, one directly below B as B.C (or equivalently Main.B.C). This is part of what’s called “namespacing”, the practice of seperating code into named units so that two distinct pieces of code can use the same name (function, variable…) to refer to two different things. Since they’re two distinct modules, they don’t share their “global” variables, because global variables are only global in the module they’re declared in (i.e. they don’t leak into the code containing the module). See Scope of Variables · The Julia Language for more information about global scopes (i.e. the scope of variables in the top level of a module).

CRquantum · August 20, 2021, 6:50pm

@stillyslalom @Sukera Thank you all!

May I ask, (I am still a little confused), say, if I have a module C in file C.jl.
Now if I want to use this module C, do I just do

include("C.jl")

is this include enough?

Or do I need to do

include("C.jl")
using .C

But the above looks redundant, isn’t it? However I know the above two lines seems are needed. The single include(“C.jl”) may not be enough.

Sukera · August 20, 2021, 7:03pm

They are not redundant, they are two seperate concepts. The first (include) pastes the source code at the location of the include. The second (using) makes the pasted code available for use, by interacting with the namespace that you’ve included, provided C.jl has a module named C inside of it. If there is no module at the top level of C.jl, using .C is not needed because the code in question is just pasted into the place of the include. It is not wrapped in a module automatically.

For example, if C.jl has this code inside of it:

module D

export f

f(x) = x
end

you’d do

include("C.jl")
using .D

to be able to call f from the module D. If f were not exported from D, you’d have to call D.f instead.

If, however, the code in C.jl is this:

f(x) = 2x

you only need include("C.jl") to call f and not using .D, because there is no D - the file only has a function definition inside, not a module definition. Unlike in python, there is no implicit module definition or namespacing happening.

pdeffebach · August 20, 2021, 7:18pm

This doesn’t quite seem right. An MWE would help, FromFile.jl should work in this instance.

CRquantum · August 20, 2021, 7:19pm

Right right right! I got you!

‘Include’ does not automatically do ‘using’, it can be used for not only module, but also any file with any contents.

From your all suggestions, I got that if there are in case some global variables, it is better to put all of them in one place. Initialize/update them in one place.

Thank you very much!

CRquantum · August 20, 2021, 7:32pm

Thank you very much!

MWE is below, three files, A.jl, B.jl, C.jl

A.jl is

include("B.jl")
using .B
include("C.jl")
using .C
mean_covar_init()
init()

B.jl is

module B
include("C.jl")
using .C
export init
function init()
    println(musigma[])
end
end

C.jl is

module C
export mean_covar_init,musigma
const musigma = Ref{Float64}(1.0)
function mean_covar_init()
    musigma[] = 100.0
    println(musigma[])
    return nothing
end
end

musigma is defined in C, and initialized in C as 100.0.
I want to from B read the initialized musigma which should be 100.0

A.jl is main program, which call the musigma in C and B.
Therefore, in the REPL, by doing include(“A.jl”), I should expect, 100.0 and 100.0 both. Like,

julia> include(“A.jl”)
100.0
100.0

However, what I got is

julia> include(“A.jl”)
100.0
1.0

Obviously the musigma in module B does not have the initialized value. Means module B does not keep track of the changes of musigma.

Is it possible to use FromFile.jl to keep track of musigma in B?

I tried FromFile.jl but it looks like it is just a wrapper for include, and it does not track the change of musigma. Am I missing something?

But again, I apologize the above three files looks like Fortran way, I know in Julia people do not do things like that.

pdeffebach · August 20, 2021, 7:37pm

As others have mentioned in this thread, you should not do include("C.jl") multiple times. Please read the responses earlier on in this thread to understand why that’s problematic.

Reasoning about this dependency tree is very hard, which is why FromFile.jl is helpful (but really, just put everything in one module). What’s happening is that when you call init, the global variable being acted upon is not the same one that was acted upon in mean_covar_init(). This is because you have two different musigmas floating around. One is Main.B.C.musigma and the other is Main.c.musigma

CRquantum · August 20, 2021, 7:41pm

Thank you. Really, I kind of got it.
In Fortran things are compiled, and the compiler does the linking module and dependence things.
In Julia, it is not compiled in that automatically way, therefore I need to be more careful and not too sloppy.

pdeffebach · August 20, 2021, 7:44pm

Julia does compile many modules and resolve them. This is not an issue of static-vs-delayed compilation. However Julia’s resolver doesn’t automatically work with submodules, but rather top-level modules. If you spit out your sub-modules as dependencies (which I don’t recommend you do), then Julia would take care of the include order automatically, just like Fortran.

To do this within a single module, with lots of submodules, use FromFile.jl. But again, the best option is to just not do this: put everything in one namespace.