How to use local, relative modules (best practice, simple)?

Hi,

Julia states to feel like a scripting language. Hence I would like to find out the most simple way to modularize (smaller) projects in julia.

Here is a minimal example that worked for me after some research and trial & error -
Main script main.jl using helper module helper.jl.

Directory structure:

testproj
  - helper.jl
  - main.jl

helper.jl:

module Helper

function helloworld()
    println("Hello world")
end

end

main.jl:

include("./helper.jl")
using .Helper: helloworld

helloworld()

include here is suboptimal, if I have read correctly:

The expression include("source.jl") causes the contents of the file source.jl to be evaluated in the global scope of the module where the include call occurs. If include("source.jl") is called multiple times, source.jl is evaluated multiple times.

Also we need two lines of code for module import instead of just one - include and using.

  1. Is there any way to reduce module imports to one line - using .Hello: helloworld?
  2. In this context: is modifying LOAD_PATH to include the current project a viable thing? How and where would I want to modify this path locally inside my project?
  3. Is there any way to “auto-wrap” a file as module, so we don’t need module Helper... end inside helper.jl, as used in other languages like Python, JavaScript?

Thanks in advance!
jakob

I think what you’re doing there is the best way to do it. Sure, the include evaluates the module in top-level scope, but that is not an issue - you actually want the Helper module to be available at the top level.

It’s true that it does require both include and using. This is because the two statements do different things: include is about where to find the source code file, and using is about which names to bring into the scope of Main.

As far as I know, there is no way to force Julia to conflate files with modules. That is, you can’t have it automatically add modules to your different source files. I would argue it doesn’t add any benefit, either.

2 Likes

You could skip the module in helper, then you would just bring all the things from the helper file into scope with the include, no need for using. Though if you want it to be a separate namespace this will not allow for that.

4 Likes

You have probably searched around already, but here is a post and links therein that is a recommended read Best practise: organising code in Julia - #2 by stevengj

To Julia, files and modules are two very different things. A file can define multiple or no modules at all. A module can span multiple files.

That said, I have export JULIA_LOAD_PATH=".:$JULIA_LOAD_PATH" in my shell profile. With that, import X will look for a file X.jl in the directory from which Julia was launched and load the module therein defined. That mimics Python’s behavior I think.
Discovery is case-sensitive, so you would need to rename helper.jl -> Helper.jl.

6 Likes

And for the sake of completeness: if you do want a separate namespace, then you can skip the using line, since by include-ing the file that defines the module Helper, it’s already available to use. So, modifying your main.jl:

include("./helper.jl")
# using .Helper: helloworld # <- This is not needed

Helper.helloworld() # `helloworld` is not imported
3 Likes

Just include("helper.jl") is equivalent and more idiomatic.

1 Like

How I do it i.e.

The “main.jl”.

(@isdefined MyDates) == false ? include("mymod.jl") : nothing

MyDates.mydate()

The module (mymod.jl) is only loaded on the first start of the “main.jl”
This makes “main,jl” faster on multiple starts(in the same session)…

The “mymod.jl”

module MyDates

export mydate     # What functions of the module are available?

(@isdefined Dates) == false ? using Dates : nothing
mydate() = print(string(day(now()),".",month(now()),".",year(now())))

end

.

Thank you for this tipp!

Why doesn’t this work though, when there is a Project.toml file inside testproj project folder (I had added one before)? With same steps and existent file, it results in:

ERROR: LoadError: ArgumentError: Package Helper not found in current path: Run import Pkg; Pkg.add("Helper") to install the Helper package.

One other thing: Shoudn’t it be sufficient to just specify --project argument to make modules available via JULIA_LOAD_PATH? Like

cd /path/to/testproj
julia --project=. main.jl

According to LOAD_PATH docs:

it defaults to ["@", "@v#.#", "@stdlib"]

@ refers to the “current active environment”, the initial value of which is initially determined by the JULIA_PROJECT environment variable or the --project command-line option.

Unfortunately, this does not work and JULIA_LOAD_PATH needs to be set manually.


Some more references:
\https://discourse.julialang.org/t/how-to-add-a-folder-path-in-your-jl-script/61901
\https://discourse.julialang.org/t/newbie-question-on-search-path/12384

I see the point of @isdefined in main.jl, but not in the code of the module (how could Dates be defined there?)

(@isdefined module/package/variable) brings back a true/false and this is
the expression for the following ternary operator. It is a short form for:

if something is not defined/installed then install it otherwise do nothing

In this way complete blocks, can be defined that only one time,at the first
start should be executed(in one session).
i.e.

(@isdefined StatsPlots) == false ?
begin
      using GraphRecipes, StatsPlots
      StatsPlots.theme(:sand)
      StatsPlots.default(size=(700,700))
end : nothing

btw.
“Dates” is a Core/Base-feature. So it can be queried everywhere…
Hope my english is good enough to explain that.

I just had the same issue and it is so annoying not being able to do it easily.

I found the easiest way is in your helper.jl, to include “export helloworld”, then you can call it directly.

Kind regards

The general purpose of @isdefined is clear. What I meant is that I don’t see the point of this pattern:

module MyDates
(@isdefined Dates) == false ? using Dates : nothing
# more code ...
end

With module MyDates you are creating a brand new module with its own namespace; almost nothing is defined there, except Base, eval, include and friends. So Dates will never be defined at that point, and the condition @isdefined Dates will always return false. It doesn’t matter if the script "mymod.jl" that contains that code had been executed before in that session.

If that were the case, executing it again will create another module also called MyDates (there will be a warning about that), where Dates is not defined at the outset either.

“current active environment” refers to the dependency graph as defined in the Project.toml and Manifest.toml.

Why doesn’t this work though, when there is a Project.toml file inside testproj project folder

With a Project.toml you commit to reproducibility. I believe the relevant passage from the manual is

Which dependencies a package in a package directory can import depends on whether the package contains a project file:

  • If it has a project file, it can only import those packages which are identified in the [deps] section of the project file.
    […]

One part of ensuring reproducibility is to avoid name clashes. Any dependency (i.e. anything loaded with import or using) needs an UUID for that reason. But that is quite simple. If the code in Helper is capsuled from the rest, then make it it’s own package. From the REPL you can generate a bare-bones structure with

]generate Helper

That will make directories Helper and Helper/src, generate a uuid, write it to Helper/Project.toml, and an entry point Helper/Helper.jl defining the module.

You can now add this package to the main project: ]dev ./Helper

Then again, including Helper.jl multiple times might not be that bad. Depends on what’s in it really.

If that’s too much hassle, but you still want another than the default environment, which is always a good idea, you can make one in a folder different from TestProj and use that. That’ll keep the Project.toml out of TestProj.

1 Like

This question comes up periodically.
Consider FromFile.jl, which aims to simplify this kind of thing.

2 Likes

Or, for a super-lightweight solution go here.

1 Like

I’m going to guess that you’re probably coming from a Python background? (As am I.)

It may be helpful if I point out something that I just realized this morning which is that

  • in Python, a single statement import does in a single line what Julia does with two lines of code: include(...) followed by import or using

To explain in further detail. Whereas in Python you would have either this

from mymodule import something

or this

import mymodule

in Julia, you require two statements. The first statement grabs the code from another file

include('myfile.jl')

and the second statement brings this new module into the Main module scope

using mymodule

or

import mymodule

I believe I’m correct in thinking that include('myfile.jl') will attach the top level module from myfile.jl to the Main module.

In other words, if myfile.jl contains this module definition

module MyModule
end

then inlcude('myfile.jl') will create the name Main.MyModule.


The reason for the difference to Python is that in Python, there is a rigid structure imposed by the fact that the names of files and directories define modules. In Julia, modules are explicitly defined with the module syntax.

This means that import X in Python is unambiguous by default. It loads the code from X.py and it already knows the module name is X.