Proper way to use modules

New to Julia
1

With julia <=0.6, I used to bring in all my modules in one place and then I would import the loaded modules within other modules as needed with “import …Mod1” as below.

#my_main.jl

push!(LOAD_PATH, Base.source_dir())

using Mod1
using Mod2

Mod2.bar(5)                                                                                                                             
Mod1.foo(5)

#Mod1.jl

module Mod1

function foo(x)
    println("x+1: ", x+1)
end
                                                                                                                                     
end

#Mod2.jl

module Mod2

import ..Mod1
#or
#import Main.Mod1

function bar(x)
    println("x:   ", x)
    Mod1.foo(x)
end

end #module

Then I would run my_main.jl. In julia 1.0, this gives “WARNING: could not import Mod2.Mod1 into Mod2” and a subsequent “UndefVarError: Mod1 not defined”.

If in Mod2.jl I put “using Mod1” instead of the import “…Mod1”, things work.

Maybe someone can help give me the right mental image of “using”, but this feels weird to me because it feels like I now have a Main.Mod1 and a Mod2.Mod1, which I would expect to effectively be different modules (each getting compiled separately).

So say I made my Mod1.jl as follows:

module Mod1                                                                                                                             
                                                                                                                                        
counter = 0

function foo(x)
    global counter
    println("counter: ", counter)
    counter += 1                                                                                                                                     
end

end #module

It’s not intuitive to me whether there should two instances of counter or one. (There is one.) The output of “julia my_main.jl” is

x:   5
counter: 0
counter: 1

2 questions:

  1. What is a smart, stylistic way to use one module (Mod1) in many other modules? Put “using Mod1” in all the modules? Do something else?
  2. Does anyone have an intuitive explanation for what using does with respect to instances of the code it brings in?

Thanks.

2

  1. Don’t use LOAD_PATH. Instead, pkg> add or pkg> dev the relevant packages to the global environment or a custom project. See https://julialang.github.io/Pkg.jl

  2. If Mod1 and the module that is using it are submodules of a third module, then using ..Mod1 is fine. Otherwise, using Mod1.

  3. Code loading is documented: Code Loading · The Julia Language but I would recommend becoming familiar with the Pkg docs I linked above first.

3 Likes
3

Thanks @Tamas_Papp.

4

So @Tamas_Papp, are you suggesting I would make every module its own package?

The way I’ve created the modules, there are many and some are fairly small, so I think this approach would add quite a bit of overhead (e.g., a package directory for each module). But is that the julia style? Each module is a package, so you probably don’t want a lot of small modules unless there is a real reason for it?

5

That mostly depends on whether the modules are useful as self-contained collections of code for a particular purpose. Otherwise, use submodules, or if they are really tiny then just use a single module.

I generally put things in a single module and use submodules for medium-sized projects only, and/or split into other packages when it makes sense. YMMV. You are basically free to do what you like, Julia supports a wide range of choices.

6

@Tamas_Papp I’m hung up on your suggestion 1. In my case, my entire body of code could be a project, but I don’t think it makes sense to make any of the modules a project themselves (per your “That mostly depends on whether the modules are useful as self-contained collections of code”). The module files are not all in the same directory (and I want them to continue to be organized in separate directories), so I need to specify those directories. Do you do that in the Pkg configuration somewhere? (I’m not seeing it.) Thanks.

7

Subdirectories of src are useful for this kind of organization.

8

In your src/MyModule.jl (relative to Project.toml) you could have

module MyModule

include("../that/file/wherever_it_is.jl")
include("../the_other/file/somewhere_else.jl")

end

which is non-standard but should work.

9

Thanks. I get the include option. …was just wondering if there was some better way of doing that in trying to understand what you were suggesting.

An advantage of

push!(LOAD_PATH, "../that/file")
push!(LOAD_PATH, "../the_other/file")

using wherever_it_is
using somewhere_else

compared to what you wrote (assuming they are modules), is compilations of wherever_it_is and somewhere_else will potentially be cached, right?

10

This is the only thing that actually seems to work. But, there are folks that say “make a package…” But making a package is a very sticky wicket indeed.

11

I disagree. I’ve created like 5 packages in the past few days. Each time it goes like this:

using PkgSkeleton
PkgSkeleton.generate("/paththatdoesnotalreadyexist/mynewpackage")

Done. Package created. You can do the same thing without using PkgSkeleton.jl, but this way tt has the recommended directory structure, documentation mostly ready to go, and proper Travis (or whatever) files you want. PkgTemplates.jl works similarly.

You add your code in mynewpackage/src.

Whatever packages you use in that code need to be added to the Project.toml. So you ] to get to the package manager

dev /paththatdoesnotalreadyexist/mynewpackage
activate mynewpackage
add Statistics LinearAlgebra # and so on

It wasn’t always so obvious to me either, but it really is easy.

2 Likes
12

I have been finally packaging my code lately, and once you understand the process and get to know the tools, it isn’t hard. And it is really much better for code reuse and such. My packages are private, not planning on register anything, but I still document everything, write tests, and I’m generally happy with the process. There are a few hitches here and there, for sure, but nothing impossible.

I just finished one package today, separating some code that I had in another package that I finished yesterday.

13

Not really, packages are very lightweight in Julia (like branches in git — the idea is that you don’t even think about making one if it is needed), and various tools exist to make creating them convenient, ranging from barebones pkg> generate to various template generators.

2 Likes