# Implicitly loaded modules in the future?

**URL:** <https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646>\
**Category:** Internals & Design\
**Tags:** question, module, code-organization\
**Created:** [June 9, 2021, 8:58pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646 "2021-06-09T20:58:00Z")\
**Posts on this page:** 20\
**Page:** 8

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [June 14, 2021, 10:12am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/141 "2021-06-14T10:12:44Z")

</div>

Does the content of `utils.jl` get executed twice in your proposed version, or is it cached somehow?

---

<div class="post-metadata">

**Author:** ![patrick-kidger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrick-kidger/32/20378_2.png) [@patrick-kidger](https://discourse.julialang.org/u/patrick-kidger)\
**Post date:** [June 14, 2021, 10:23am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/142 "2021-06-14T10:23:52Z")

</div>

Only once; it’s cached.

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [June 14, 2021, 10:39am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/143 "2021-06-14T10:39:01Z")

</div>

It’s it optional, or everyone must follow it (i.e. current way become unavailable)?

---

<div class="post-metadata">

**Author:** ![patrick-kidger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrick-kidger/32/20378_2.png) [@patrick-kidger](https://discourse.julialang.org/u/patrick-kidger)\
**Post date:** [June 14, 2021, 10:48am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/144 "2021-06-14T10:48:33Z")

</div>

I don’t think anyone is advocating removing the old way, if only for backward compatibility.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [June 14, 2021, 10:50am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/145 "2021-06-14T10:50:12Z")

</div>

Btw, you would not be allowed to modify foo and bar in another package from outside of the package without using eval.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [June 14, 2021, 11:02am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/146 "2021-06-14T11:02:55Z")

</div>

My point is that a lot of the comparisons and sources of pain discussed in this thread are from moving from python. It appears that making that move from python easy for people is a major motivator for these changes.

It may not be yours, but you’re not OP here. OP specifically references python in the first post, as have others.

> in which `A.jl` and `B.jl` implicitly depend upon `utils.jl` .

Also, is there a real package where this happens and is a problem? please link. Do you mean that capitalised A and B are modules?

---

<div class="post-metadata">

**Author:** ![liuyxpp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/liuyxpp/32/9870_2.png) [@liuyxpp](https://discourse.julialang.org/u/liuyxpp)\
**Post date:** [June 14, 2021, 12:21pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/147 "2021-06-14T12:21:54Z")

</div>

Indeed, with current `include`, it is so painful to read other’s package. I am always find myself locating where does this keyword/funciton/variable comes from by going through all source files. It really can drive someone crazy going this cycle over and over again!

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [June 14, 2021, 2:13pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/148 "2021-06-14T14:13:02Z")

</div>

I wonder, have you seen this: [Implicitly loaded modules in the future? - #72 by PetrKryslUCSD](https://discourse.julialang.org/t/proper-modules-in-the-future/62646/72)? It happens to mention your code. My point is that what is accomplished with FromFile macros could have been done with modules.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [June 14, 2021, 4:44pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/149 "2021-06-14T16:44:24Z")

</div>

> [@liuyxpp](#):
>
> Indeed, with current `include` , it is so painful to read other’s package.

Doesn’t that depend on the quality of the design of the package? Bad design is bad design, the language cannot prevent that.

---

<div class="post-metadata">

**Author:** ![heliosdrm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/heliosdrm/32/3851_2.png) [@heliosdrm](https://discourse.julialang.org/u/heliosdrm)\
**Post date:** [June 14, 2021, 5:06pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/150 "2021-06-14T17:06:06Z")

</div>

> [@patrick-kidger](#):
>
> The hope is to switch this out for a syntax looking something like:
> 
> ```julia
> # master.jl
> import "A.jl": foo
> import "B.jl": bar
> 
> # utils.jl
> common = 1
> 
> # A.jl
> import "utils.jl": common
> foo = common + 2
> 
> # B.jl
> import "utils.jl": common
> bar = common + 3
> 
> ```

Since `import` is meant to be an operation with modules, wouldn’t this be clearer if it required `A.jl` and `B.jl` to be written as “module files” (i.e., its contents being wrapped in `module A ... end`, etc.)?

That would mean typing a few more characters in each file, but it would make that new way of using `import` more understandable in my eyes: it would be more like a shorthand for `include("A.jl")` + `import .A` except for avoiding issues of multiple `include`s. Also, as soon as I see a file like `A.jl` in a project, I would clearly understand that `foo` belongs to the file-module `A`, so it won’t be writable by code from any other file.

This thread is long, and I have not looked thoroughly into all the answers or links to other posts or Github issues, so maybe the advantages or disadvantages of my suggestion have already been discussed elsewhere. If that’s the case, please don’t repeat the arguments; some pointers to relevant comments would be enough, thank you.

---

<div class="post-metadata">

**Author:** ![patrick-kidger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrick-kidger/32/20378_2.png) [@patrick-kidger](https://discourse.julialang.org/u/patrick-kidger)\
**Post date:** [June 14, 2021, 5:19pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/151 "2021-06-14T17:19:01Z")

</div>

Yep, this is a minor point that’s come up once or twice. I don’t think there’s _particularly_ strong feelings on this either way. (Perhaps apart from @PetrKryslUCSD?)

The module-less approach has less boilerplate, and the `import` statement can be understood to wrap a file into a module.

The module-full approach makes it harder to accidentally misuse any `some_obj` functionality defined outside the file, if it is unintentionally `include`-d, as plain `some_obj` usages would fail.

(Personally I favour the module-less approach. Less boilerplate is nice, and if `import` becomes the default approach then the same safety is attained. But I see the counterargument, especially during a transition period.)

---

<div class="post-metadata">

**Author:** ![Orbots](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/orbots/32/3392_2.png) [@Orbots](https://discourse.julialang.org/u/Orbots)\
**Post date:** [June 14, 2021, 5:21pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/152 "2021-06-14T17:21:30Z")

</div>

Not reading this entire thread, but linking my old thread as support for OP. [Organization of multiple modules in same package](https://discourse.julialang.org/t/organization-of-multiple-modules-in-same-package/17287)

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [June 14, 2021, 5:26pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/153 "2021-06-14T17:26:43Z")

</div>

> [@patrick-kidger](#):
>
> (Personally I favour the module-less approach. Less boilerplate is nice, and if `import` becomes the default approach then the same safety is attained. But I see the counterargument, especially during a transition period.)

I think the boilerplate is minimal, while the loss of legibility of the code/ability of the programmer to grok the code due to the missing module declaration may well defeat the purpose of this whole effort.

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [June 14, 2021, 6:03pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/154 "2021-06-14T18:03:40Z")

</div>

@PetrKryslUCSD thanks; indeed I did read your post as well as @gustaphe and @kristoffer.carlsson’s points about using submodules instead. I think it will be a bit more straightforward for everyone if you think of FromFile.jl as simply a macro on submodules, which it is (and the point of macros being to reduce unnecessary code).

> I think the boilerplate is minimal, while the loss of legibility of the code/ability of the programmer to grok the code due to the missing module declaration may well defeat the purpose of this whole effort.

I think this is a bit of wishful thinking on programmers being as disciplined as yourself (your project looks very nicely organized!). For instance: me - I knew about the existence of submodules when I started my SymbolicRegression.jl project, but I did not use them. Why would I put every file in its own submodule when the `include(...)` is already enough? So that is what I did. But FromFile.jl makes it convenient and attractive enough to the point where I actually write down all the dependencies, so I actually do it! That is the main difference that I see for this efficient syntax, compared to manually writing out the submodules. [_I don’t know if anybody has read “Atomic Habits” by James Clear - but this would essentially follow his point of making a desired habit “attractive and easy”_]

Cheers,  
Miles

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [June 14, 2021, 6:35pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/155 "2021-06-14T18:35:40Z")

</div>

Sorry, one more question. If it is optional, why not to use `FromFile.jl` or any other analogue to achieve necessary result? Is there anything in proposed approach/approaches which can’t be implemented in standalone package?

I mean, this is a good thing, that lots of ideas can be implemented as packages without being part of `Base`. If it can live outside of `Base` it seems that it’s better to stay this way. Just use package that you like and everyone is happy and those who support this idea and those who opposes it.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [June 14, 2021, 6:40pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/156 "2021-06-14T18:40:27Z")

</div>

For me it is useful to understand the main differences between the proposed syntax and the way I deal with organizing the Julia files. I `include` every file (with the files wrapped in modules) to the main package module and then making relative `import`s to them. The result seems similar to what you achieve, the main difference are:

1. Some boilerplate is removed (i.e., you do not need to wrap every file in a module).
2. The main package module will not have every submodule directly inside it (so it can be relatively imported by the other submodules).

I am not entirely sure if my approach is dependent on order or not, I do not remember anymore. If it is, then this would be probably the main difference, the DAG vs strict sequence is where more seems to be gained.

I like the current system, and my only problem with it is the lack of internal modularity. I would like the possibility of having a small set of “sub-packages” inside the package folder directory (only be visible to the main package). This way you could organize internal dependencies the exact same way you organize external ones. All the machinery would already be there, and it allows for DAGs already.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 14, 2021, 6:56pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/157 "2021-06-14T18:56:11Z")

</div>

> [@Henrique\_Becker](#):
>
> I like the current system, and my only problem with it is the lack of internal modularity. I would like the possibility of having a small set of “sub-packages” inside the package folder directory (only be visible to the main package). This way you could organize internal dependencies the exact same way you organize external ones. All the machinery would already be there, and it allows for DAGs already.

+1 for this

Each package could have its own mini-registry. And open the corresponding subpackages to the world by putting them in the general registry. We would need to define which character would be used to separate the main package from its subpackages

```julia
] add BigPackage_SubPackage

using BigPackage_Subpackage

```

to install that one independently. Of course `.` is the character of dreams, but I fear that one is problematic.

I see this solutions as really a gain-gain. Not breaking. Allows complete modularity and allows this modularity to be exported to the world. And completely optional.

---

<div class="post-metadata">

**Author:** ![lamorton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lamorton/32/21115_2.png) [@lamorton](https://discourse.julialang.org/u/lamorton)\
**Post date:** [June 14, 2021, 7:27pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/158 "2021-06-14T19:27:01Z")

</div>

Someone suggested an addition to the “Noteworthy differences” page. Here’s a draft that could be further wordsmithed. [Comment on `include` pattern in Julia vs. Python by lamorton · Pull Request #41227 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/41227)

In particular, perhaps one might add something like: “To recreate Python’s module/file equivalence when structuring a package, do the following. (1) Wrap the contents of each file with a `module MySubmodule`, (2) `export` all the variable names from that module that should be made available in the main module, (3) do `include("MySubmodule.jl"); import .MySubmodule` within the main module.”

Or, maybe make a reference to “FromFile” or “PatModules”, if that is considered acceptable.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [June 14, 2021, 7:43pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/159 "2021-06-14T19:43:12Z")

</div>

Another datapoint: a very similar idea of “include-and-wrap-into-module” was suggested and implemented by @fonsp in the context of Pluto.jl: see [include file containing structs · Issue #115 · fonsp/Pluto.jl · GitHub](https://github.com/fonsp/Pluto.jl/issues/115#issuecomment-705187793) and around.

---

<div class="post-metadata">

**Author:** ![liuyxpp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/liuyxpp/32/9870_2.png) [@liuyxpp](https://discourse.julialang.org/u/liuyxpp)\
**Post date:** [June 15, 2021, 3:21am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/160 "2021-06-15T03:21:46Z")

</div>

I am not the package author, so it is difficult to figure out where does one function come from if it is defined in one of included files, such as

```julia
include("dir1/A.jl")
include("dir1/Ax.jl")

include("dir2/Bx.jl")
include("dir2/B.jl") # some_func is defined in this file
include("dir2/By.jl")

include("dir3/C.jl")

# in file D.jl
f(x) = some_func(x)

```

Do you mean this is a bad way to organize codes? But I see a lot of prominent packages organized in this fashion, e.g. [Optim.jl](https://github.com/JuliaNLSolvers/Optim.jl/blob/master/src/Optim.jl). When I first see `some_func` (I may visit this particular place by google searching), how can I tell it is defined in file “dir2/B.jl”?

[Previous page](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646.md?page=7)

[Next page](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646.md?page=9)