Apparent inconsistency importing modules


#1

Hi there,
I am new to Julia but I have written a few lines of Julia and I am still juggling with this, and it feels like I still don’t get it or I am not doing something right. So start off, when I want to import a module using using it seems I have two choices, I either include the file source file that has that module so I can then call using, or I append that directory to LOAD_PATH? I don’t really understand the latter. Like will Julia look inside source files in the loaded directories? And I am not sure I would want to modify global variables to have my software run?

I have something like

test.jl
Mod/Mod.jl

and tried push!(LOAD_PATH, ".\\Mod") from within test.jl, that did not work. I would like to understand how this works, but still, I would like for my software to just run without touching global variables.

But the main thing I am concerned about is this below

#mod1.jl
module Mod1
  x = 1
end

#mod2.jl
include("mod1.jl")
module Mod2
  using Mod1
  x = Mod1.x
end

#mod3.jl
module Mod3
  include("mod2.jl")
end

Here, julia mod2.jl works and julia mod3.jl doesn’t (it cannot load Mod1).

I have had a frustrating experience so far dealing with modules in Julia. Like, for instance, I tend to define module_file_jl=nothing inside module_file.jl so I can include as follows

!isdefined(:module_file_jl) && include("module_file.jl")

I have the above all over my code base as to avoid type reloads that lead to run-time errors where equivalent types do not match (this was painful for a while) when I include module_file.jl in several other files. But that trick solved the problem, but it certainly feels like a hack.

I have gone through the documentation for Modules, but I really need help here as this is bad (the way I am working on it). Thank you.

Pedro


#2

Ok, a few things:

  1. Definitely don’t do the !isdefined(:foo_jl) && include("foo.jl") trick. That’s a C-ism that should never be needed in Julia.
  2. When you do import Foo or using Foo, Julia looks in your package directory and then also looks in your LOAD_PATH for Foo.jl or Foo/src/Foo.jl (so yes, as you suspect, it does look inside your src folder).
  3. include() is typically used when you want to split up a big module definition across multiple files. Using it to include a dependent module as you’ve done is a bit weird and probably not a good idea. Instead, if you rename your module files to match their module names (as in, module Foo in Foo.jl or Foo/src/Foo.jl) and ensure the folder containing your modules is in LOAD_PATH, then you can simply do:
# Mod1.jl
module Mod1
  x = 1
end

# Mod2.jl
module Mod2
  using Mod1
end

and so on.


#3
  1. edit: (thanks to rdeits) “I think you need a fully qualified path when you add it to LOAD_PATH, Julia doesn’t convert that relative path (or ones that start with ~ for your home directory) to an absolute path.” is wrong.

You do need to expand ~, Julia does not handle it in paths.

  1. You shouldn’t need to include("mod1.jl") in mod2.jl, if you have your LOAD_PATH set up correctly, it will be able to find mod1.jl.

  2. Same for Mod3, you shouldn’t need to include it, simply have using Mod2.

  3. If you want these to be found when calling julia from the command line, make sure you have JULIA_LOAD_PATH set in your environment. See JULIA_LOAD_PATH in the documentation.

I hope this will be helpful!


#4

I think you need a fully qualified path when you add it to LOAD_PATH, Julia doesn’t convert that relative path (or ones that start with ~ for your home directory) to an absolute path.

That does not seem to be correct. For example, with Foo.jl in my current directory:

julia> push!(LOAD_PATH, ".")
3-element Array{Any,1}:
 "/home/rdeits/apps/julia-0.6.2/local/share/julia/site/v0.6"
 "/home/rdeits/apps/julia-0.6.2/share/julia/site/v0.6"      
 "."                                                        

julia> using Foo


#5

OK. Good to know. I always run into the problem that Julia doesn’t handle the ~ for the home directory in paths.


#6
expanduser("~")

#7

Yeah, that’s a good point. ~ is only meaningful to your shell.


#8

Thank you all for the input. I understand modules better now. Now, what if I have a deeper tree of source code,

dir0/
dir0/dir1/
dir0/dir1/dir2/

Do I need to add each one of these directories to LOAD_PATH?


#9

Also, I am able to use module Foo in Foo.jl but not in Foo/src/Foo.jl

So push!(LOAD_PATH,"/Path/To/Foo") works if I have /Path/To/Foo/Foo.jl but not /Path/To/Foo/src/Foo.jl

Am I missing something about the Foo/src/Foo.jl?


#10

In your case, you would need /Path/To/Foo/Foo/src/Foo.jl (you have to replace Foo.jl with Foo/src/Foo.jl). Or you could just add /Path/To to your LOAD_PATH.


#11

That worked! Thank you. Else regarding the comment above, we need to add each nested directory separately right?


#12

It’s true that LOAD_PATH is not recursive, so yes, you would need each folder containing a module on your path.

That said, I would suggest reconsidering your design. Having nested folders on your LOAD_PATH is a pretty unusual design in Julia, and it will make it harder to, for example, publish any of those modules as a package for others to use. Much more typical is having a single folder containing all of your top-level modules, and then sub-folders only as necessary for organizing sub-modules.

For example, JuMP consists of a single module JuMP broken up into many files: https://github.com/JuliaOpt/JuMP.jl/tree/master/src . It also has a sub-module called Derivatives, and that module’s definition is include()ed by the main JuMP.jl: https://github.com/JuliaOpt/JuMP.jl/blob/c44401beb3ada8f2f36a9d12f8b4061f472d85b6/src/JuMP.jl#L24 The result of all this is that you only need a single folder (wherever the entire JuMP repo lives) on your path, but you can still load JuMP with using JuMP and load its submodule with using JuMP.Derivatives.

Even better, if you’ve installed JuMP with Pkg.add("JuMP"), then it will be in your Pkg.dir() folder which is automatically available for using and import without any LOAD_PATH modification.


#13

Thanks for sharing that, and that makes more sense to me now.