Packages, modules, and files

Internals & Design
1

The exact connection between packages, modules, and source files is unclear to me, even after reading the Pkg3 Julep and the relevant chapters of the documentation. Please note that this is not a criticism of either the current state or what will replace it with Pkg3, I am just trying to understand how things work.

My understanding of the status quo is the following:

  1. A package is a collection of files and metadata. The have names like Foo.jl.

  2. A module is a piece of source code, eg module Foo ... end.

  3. using Foo and import Foo try to locate a Foo.jl in LOAD_PATH, specifically a Foo/src/Foo.jl inside a package, or just a Foo.jl for a module that is just a source file.

While in theory modules, source files, and packages could more or less be orthogonal, in practice using (and import) will not find Foo unless it is in a Foo.jl somewhere, using the above mechanism. The only other option is to include its source file. Looking at the source for loading, within a few function calls syntax that started with modules (eg using) quickly involves package structures.

It is not possible to have a module Bar that has a source file src/Foo.jl that has a module Baz ... end (ie it is of course possible, but Julia will just ignore it). Even if 2 of the 3 names match; all 3 of them have to match for the module to be loadable with import/using.

Similarly, a differently named (extra) file with module in a package structure is just ignored, unless included explicitly. Foo/src/Foo.jl may define module Bar ... end. In this case using Bar fails, and using Foo gives a warning about not being able to load the package. If there happens to be a module Foo, Bar is just loaded with it silently.

Pkg3 seems to hook into using, and offer installation of the missing package.

So, in practice, loading a module with using Foo and import Foo work only if

  1. it is in a package that is in LOAD_PATH with the name Foo/src/Foo.jl,

  2. it is in a standalone source file in LOAD_PATH with the name Foo.jl,

  3. it was in a piece of code that is already evaluated (eg with include), in which case using Foo and import Foo are no-ops.

Is this correct?

5 Likes
2

Why is package structure such a bad thing?

There are minimal ways to inform load hierarchies

4

Interesting, but I don’t see what it has to do with my question.

I am not saying it is. Again, my question is about understanding the implicit assumptions in the current setup, not about solving an actual issue, or a problem that I identified with it.