# Implicitly loaded modules in the future?

**URL:** <https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646>\
**Category:** Internals & Design\
**Tags:** question, module, code-organization\
**Created:** [June 9, 2021, 8:58pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646 "2021-06-09T20:58:00Z")\
**Posts on this page:** 20\
**Page:** 10

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [June 21, 2021, 12:03pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/183 "2021-06-21T12:03:31Z")

</div>

> [@StefanKarpinski](#):
>
> Julia has this already (with a number of additional features and improvements): packages.

I followed this thread with interest, but one thing that seems to stay a bit out of focus is the differences in philosophy/design between Julia and (say) Python when it comes to packaging and reuse. This influences the way modules work in Julia and perhaps (partly) causes the perceived mismatches with other languages.

- In Julia the package is the main unit of reuse and distribution, as it contains all necessary details on dependencies for a set of source files. Plus packages have explicit support for easy deployment via e.g. git repositories and management using `Pkg`.
- Even though Julia packages are usually structured in modules, using a Julia module by itself is not really a supported goal (you need to go through a package instead) and it shows in the way they are implemented. For example, a module might be split over multiple source files and there is no general way to handle such a module by itself except by using the package the module is contained in.
- In contrast in Python any `.py` file comprises a module. And as such it (implicitly) list its dependencies through the relevant `import` statements contained in it. So a Python module can usually be reused without too much trouble and is much more of a standalone unit than a Julia module. For example, this helps in writing test code for individual modules as you simply import the module to test in a driver script and don’t need to worry about the package the module is part of. The same goes for a Python package, which is not much more than a directory of files, leading to a set of nicely namespaced modules.
- But support for module/package deployment is not as tightly integrated in the Python ecosystem, even though there are things like PyPI, setuptools and pip to handle packaging and deployment. All in all, handling dependencies is quite a bit more involved and requires more manual labour in Python than in Julia.

At least, this is my 10,000 feet view, based on limited experience with Julia (but much more with Python).

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 21, 2021, 12:19pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/184 "2021-06-21T12:19:27Z")

</div>

> [@paulmelis](#):
>
> Even though Julia packages are usually structured in modules, using a Julia module by itself is not really a supported goal (you need to go through a package instead) and it shows in the way they are implemented. For example, a module might be split over multiple source files and there is no general way to handle such a module by itself except by using the package the module is contained in.

This is not true. The general non-package way to use a module defined in one or more files is to do `include("file-with-mymodule.jl"); import .MyModule`, where `"file-with-mymodule.jl"` is the file that contains `module MyModule ...` (and which possibly includes other files).

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [June 21, 2021, 12:26pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/185 "2021-06-21T12:26:08Z")

</div>

> [@stevengj](#):
>
> This is not true. The general non-package way to use a module defined in one or more files is to do `include("file-with-mymodule.jl"); import .MyModule` , where `"file-with-mymodule.jl"` is the file that contains the `module MyModule ...` (and possibly includes other files).

But it seems you have no guarantee _in general_ that the included module file will pull in all necessary dependencies? As dependencies are quite often listed at the top-level package `.jl` file and not in the module file, with individual parts making up the package `include()`d below all the dependencies. I.e. structured like

```julia
module M

import ...
using ...

include("file1.jl")
include("file2.jl")
...

end

```

Including `file1.jl` as you suggest will most likely not work.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 21, 2021, 12:29pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/186 "2021-06-21T12:29:14Z")

</div>

> [@paulmelis](#):
>
> Including `file1.jl` as you suggest will most likely not work.

Yes it will, as long as `file1.jl` defines a module (i.e. it is of the form `module ... end`). Any `import` and `using` statements _outside_ of the `module` statement (in the enclosing scope) don’t affect it.

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [June 21, 2021, 12:46pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/187 "2021-06-21T12:46:26Z")

</div>

> [@stevengj](#):
>
> Yes it will, as long as `file1.jl` defines a module (i.e. it is of the form `module ... end` ). Any `import` and `using` statements _outside_ of the `module` statement (in the enclosing scope) don’t affect it.

Okay, but this nicely highlights the semantic differences of a “module” in Julia and one in Python. Even if `file1.jl` defines a module and lists all dependencies it can `include()` dozens of other source files which individually will not list their dependencies, and cannot easily be used by themselves. So indeed a Julia “module” will be usable outside of its enclosing package the level of use in general is coarser than in (say) Python where every source `.py` _file_ is guaranteed to be importable by itself (and also each make up a module). I guess the main point I was trying to make was that in general in Julia a single `.jl` file does not equal a Julia module and is not expected to be easily usable by itself (yes, you can `include()` it, but that might require dependencies to be manually specified as well).

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 21, 2021, 2:17pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/188 "2021-06-21T14:17:03Z")

</div>

> [@paulmelis](#):
>
> is coarser than in (say) Python where every source `.py` _file_ is guaranteed to be importable by itself

Yes, Julia is not Python—believe me, we know 😉 . But saying “files ≠ modules” in Julia is different from saying that “using a Julia module by itself” (not as a package) is not supported.

I think people here are generally open to ways that make it easier to import modules from files without having an explicit `include`, e.g. by `import "Foo.jl"` or some other syntax. But I think it’s unlikely that Julia will ever _require_ each file to define a new namespace, ala Python.

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [June 21, 2021, 2:30pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/189 "2021-06-21T14:30:44Z")

</div>

> [@stevengj](#):
>
> I think people here are generally open to ways that make it easier to import modules from files without having an explicit `include` , e.g. by `import "Foo.jl"` or some other syntax. But I think it’s unlikely that Julia will ever _require_ each file to define a new namespace, ala Python.

This really captures where I find these threads lose me – the threads oscillate between two requests, one of which seems great and one of which seems not great:

1. **Great** : It should be easier to treat a file like a module that has a sealed namespace so that importing it makes all introduced names explicit.
2. **Not Great** : It should be impossible to break up a single module into multiple files.

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [June 21, 2021, 3:10pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/190 "2021-06-21T15:10:24Z")

</div>

> [@stevengj](#):
>
> Yes, Julia is not Python—believe me, we know 😉 . But saying “files ≠ modules” in Julia is different from saying that “using a Julia module by itself” (not as a package) is not supported.

I actually wasn’t aware you could still use a module separately like you showed. I mostly saw them as the building blocks for packages. In fact, most package repos I’ve looked at so far seem to be a single top-level module including many separate files.

> [@stevengj](#):
>
> I think people here are generally open to ways that make it easier to import modules from files without having an explicit `include` , e.g. by `import "Foo.jl"` or some other syntax. But I think it’s unlikely that Julia will ever _require_ each file to define a new namespace, ala Python.

I actually wasn’t advocating Julia moving towards the Python module scheme. Just trying to make an observation on the differences.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [June 21, 2021, 3:37pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/191 "2021-06-21T15:37:22Z")

</div>

> [@johnmyleswhite](#):
>
> **Not Great** : It should be impossible to break up a single module into multiple files.

Could not be separated as “submodules” like it is described here: [Understanding C++ Modules: Part 1: Hello Modules, and Module Units](https://vector-of-bool.github.io/2019/03/10/modules-1.html) ?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [June 21, 2021, 3:42pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/192 "2021-06-21T15:42:15Z")

</div>

> [@stevengj](#):
>
> make it easier to import modules from files without having an explicit `include` , e.g. by `import "Foo.jl"` or some other syntax

I am not sure if it is by design or accidental, but I find it really great that `using` and `import` currently resolve names via the stacked project environments, not the file system.

I understand that it is occasionally tedious for some use cases, but the benefits are also significant for writing modular code.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [June 21, 2021, 4:27pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/193 "2021-06-21T16:27:26Z")

</div>

It seems that people are confused that there is an extra level of granularity _below_ module, that is throwing them off.

If you put each module in its own file (simply don’t split modules over different files), what problems remain?

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [June 21, 2021, 4:31pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/194 "2021-06-21T16:31:16Z")

</div>

> [@DNF](#):
>
> what problems remain?

That `include("module.jl");using Module` is slightly longer than `using("module.jl")` I imagine. Not much to worry about IMO.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [June 21, 2021, 4:59pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/195 "2021-06-21T16:59:51Z")

</div>

No, that is not the issue that has been raised.

The issue is that if you have a moderately complicated dependency structure on internal modules, you can’t simply do `include("module.jl")` over and over again, because that introduces a _new_ `Module` every time.

OP is (essentially) asking for a dependency resolver among different file-modules that are used internally in a package. This is currently clunky to do, requiring messing with `LOAD_PATH` or creating a very complicated directory structure.

This is a perfectly reasonable request and deserves developer consideration.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [June 21, 2021, 5:21pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/196 "2021-06-21T17:21:37Z")

</div>

> [@pdeffebach](#):
>
> you can’t simply do `include("module.jl")` over and over again

Sorry for being dense, but why would one do that?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [June 21, 2021, 5:28pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/197 "2021-06-21T17:28:03Z")

</div>

Please see earlier in this discussion (it’s long enough already). Basically, it’s a common design pattern to have sub-modules with a networked structure. People like submodules because it restricts the name-space they have to think about when writing code.

---

<div class="post-metadata">

**Author:** ![Roger-luo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roger-luo/32/3399_2.png) [@Roger-luo](https://discourse.julialang.org/u/Roger-luo)\
**Post date:** [June 21, 2021, 5:33pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/198 "2021-06-21T17:33:33Z")

</div>

> [@tbeason](#):
>
> Packages can be very very small and very specific. No reason not to make a package!

Registering and making a package takes a lot more time than just defining a module especially when the project is not mature enough. If there’s 10 components of a mono package that can be seperate to small modules it will take 30 days to register does this sounds reasonable to you? Not to say there are private commercial users who wants to write a mono package within one private repo

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [June 21, 2021, 5:43pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/199 "2021-06-21T17:43:32Z")

</div>

Liking submodules does not explain why would someone do `include("a.jl")` repeatedly. It is simply a wrong design decision/implementation, nothing else. No messing with LOAD\_PATH is required either.

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [June 21, 2021, 5:49pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/200 "2021-06-21T17:49:06Z")

</div>

You don’t need to register a package to use it.

---

<div class="post-metadata">

**Author:** ![Roger-luo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roger-luo/32/3399_2.png) [@Roger-luo](https://discourse.julialang.org/u/Roger-luo)\
**Post date:** [June 21, 2021, 5:49pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/201 "2021-06-21T17:49:57Z")

</div>

But I need to register the package using these sub modules to use it then I will have to register all the sub modules no matter they are mature or not

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [June 21, 2021, 5:50pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/202 "2021-06-21T17:50:46Z")

</div>

I have read the whole thing, and don’t get it. Why not `include` once, and then `using`/`import` multiple times?

You can make a module inside a module, how is this related to `include`? Python doesn’t appear to even have submodules, only subpackages.

[Previous page](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646.md?page=9)

[Next page](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646.md?page=11)