# Implicitly loaded modules in the future?
**URL:** https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646
**Category:** Internals & Design
**Tags:** question, module, code-organization
**Created:** [June 9, 2021, 8:58pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646 "2021-06-09T20:58:00Z")
**Posts on this page:** 20
**Page:** 7
### Author:  [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)
#### Post date: [June 13, 2021, 2:41pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/121 "2021-06-13T14:41:42Z")
> [@lamorton](#):
>
> Does anyone ever write conditional `include` statements?
I do not use conditional `include` statements, but I use `Requires.jl` that may be seem as something similar (but I think it does conditional `eval`s instead of requires).
The use case is simple. I have a command-line tool provided by my package and that uses the functionality the package provides. However, my package/script depends on a solver to work, only one, of about 4 it supports. I do not want to force the user to have a specific solver, or the four of them (some are paid and cannot be installed easily), so `Requires.jl` allows me to only load the code that interfaces with some solver if the solver is loaded in the environment the script runs. And the only thing the user must do to use the tool is to enter the tool environment a single time and add the solver package they intend to use. Or, if they want to write their own script using my package, they only need to load one of the four solver packages for things to work seamlessly.
I think that looking at `Requires.jl` use cases will probably reveal which packages depends on something similar to conditional `include` statements.
---
### Author:  [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)
#### Post date: [June 13, 2021, 2:51pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/122 "2021-06-13T14:51:45Z")
> [@jzr](#):
>
> Could someone say why it’s useful to have a single module split between multiple files? If a module is a single logical unit, what’s the point of using multiple files? If the other files _aren’t_ part of the same logical unit, I can use multiple submodules.
I use this feature heavily. I have a big package that provides multiple mathematical formulations for a specific problem. So each mathematical formulation has their own submodule. However, for each formulation module I do not have only the code for model creation but also for plotting the results (that change with each formulation), etc… I do not create a second-level of submodules (`FormulationName.Build`, `FormulationName.Results`, …) because it is overkill, the `Results` method need to know the auxiliary structs used by `Build` and I would need some extra imports between submodules that serve the same formulation and have no conflicts with each other. So what I did to better organize my code was to split these different features in files, and include directly into the submodule by `require`. I am happy with the results, I find what I want faster now, without any overhead of headers and imports.
---
### Author:  [@lamorton](https://discourse.julialang.org/u/lamorton)
#### Post date: [June 13, 2021, 6:49pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/123 "2021-06-13T18:49:52Z")
> [@Raf](#):
>
> You’re thinking in Blub 😉
I was afraid of that! 😅
A lingering question I have: is it ever the intent for the definition of `subcalc` upstream from `calculate` to be different in two different places where `calculate.jl` gets included? For instance, does `subcalc` do `sqrt` in one context and `x^2` in another, or can I always assume when I read the text of `calculate` that `subcalc` will refer to the same function?
For example, like this:
`distance.jl`
```julia
subcalc(x) = sqrt(x)
include("calculate.jl")
```
`norm.jl`
```julia
subcalc(x) = x^2
include("calculate.jl")
```
If something like this happens, then the meaning of `subcalc` in `calculate.jl` is ambiguous. As far as I can tell, nobody is suggesting that there is a valid use case for something like this. It sounds like the _intent_ is always to have the same context wherever `calculate` gets included – even if the `include` happens in multiple places, it is assumed that all the missing identifiers are the same. But there’s no _guarantee_ that the context will be the same. By putting a namespace around `calculate,` you provide that guarantee. IMO, unless the intent is indeed to change how `calculate` works in different contexts, then `including` instead of `using` leaves the door open to undesired effects.
I played around a bit, and it looks like Julia generates warnings for pathological cases (ex: including both `norm.jl` and `distance.jl` into the primary module, or even if I turn those into submodules and try to export `calculate` from both.) That covers _most_ of what I’m concerned about. If neither submodule `Distance` or `Norm` exports `calculate`, then there is no warning. That still bugs me a little, but I guess I can get behind it because the two versions of `calculate` are appropriately isolated in their own namespaces, so there’s no reason they should mean the same thing.
---
### Author:  [@doit](https://discourse.julialang.org/u/doit)
#### Post date: [June 13, 2021, 10:55pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/124 "2021-06-13T22:55:30Z")
Not to be too disrespectful because some questions are genuine but it seems to me that an aweful lot of people try to construct examples of misuse of includes and then cry about errors it would cause.
That behavior is equivalent to standing in the kitchen and figuring out that knives can’t just cut bread but also your arm. And then cutting your arm. And then proudly showing your cut arm. And then arguing knives should be prohibited.
How about not cutting your arm?
Just because Julia allows you the freedom to be a bad programmer doesn’t mean you have to be one. There is nothing wrong with includes. If you desire a simple way to import single files in their own namespace, then by all means, demand just that. But don’t equate that to includes being wrong or harmful.
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [June 13, 2021, 11:23pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/125 "2021-06-13T23:23:46Z")
I don’t really see how you can look at that issue and conclude that there is “no proper discussion” or that dissenting voices are “slapped down”. Moreover, how is this kind of comment constructive? It is dismissive of the work and thought that have gone into that thread from all parties — which apparently doesn’t warrant the label “proper discussion”. It is also discouraging to core devs like myself who have been trying to move it in the direction of an actual solution, since we are apparently “slapping down” dissenting voices. In general, this kind of meta-complaint is not only completely unhelpful, it’s also one of the more exhausting things about open source. Please don’t do it.
---
### Author:  [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)
#### Post date: [June 13, 2021, 11:42pm UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/126 "2021-06-13T23:42:10Z")
I am sorry if I offended. The comment was really intended to adress the last week or so, not all the work that went into this thread.
I offered some opinions in the issue thread in the past few days, which could be classified as “dissenting”. To my several posts I got a bunch of “thumbs down” with no response/discussion (ignored), and only two meaningful, politely disagreeing, replies. I don’t know, it seems like a critical voice is not welcome?
---
### Author:  [@mbauman](https://discourse.julialang.org/u/mbauman)
#### Post date: [June 14, 2021, 1:59am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/127 "2021-06-14T01:59:21Z")
I thumbsed down a few of your comments because they were steering the discussion away from what I saw was a rather productive brainstorm.
Personally I’m still not sure if I’m on board with a change — partly because I don’t even know what the change would be. I find the status quo perfectly sufficient and dislike python’s system, but I don’t even know what this will end up looking like in a possible future Julia. No one does. I’m curious to see where this goes.
Your view has long been abundantly clear and some of the successive comments weren’t adding much. It’s not that your view isn’t welcomed or that dissenting voices are silenced, it’s that we’ve already heard your voice loud and clear.
---
### Author:  [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)
#### Post date: [June 14, 2021, 2:06am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/128 "2021-06-14T02:06:54Z")
Thank you for your perspective. I am afraid I rather gained the impression that this was a done deal and raining on the parade was undesirable and hence discouraged with disdain and 👎.
---
### Author:  [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)
#### Post date: [June 14, 2021, 4:00am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/129 "2021-06-14T04:00:40Z")
Just to briefly chime in on the readability point of view, as a user of @patrick-kidger’s FromFile.jl approach: I’m finding it quite a bit easier to work on my SymbolicRegression.jl package after the switch at version 0.5.0. Here is a glimpse of the old code:
>
At this point in time I used `include(...)` in my main file. Even as the person who wrote the package, oftentimes I would lose track of which type or method was defined in which file (which perhaps is because I don’t use any IDEs to manage this, just vanilla vim). This happened enough times that I wrote a script to (approximately) help keep track of which file declares which token:
```julia
for f in $(find . -name '*.jl'); do echo $f && cat $f | vim - -nes -c '%s/#.*//ge' -c '%s/"""\_.\{-}"""//ge' -c '%v/^\S\+/d_' -c '%g/^\(end\|@from\|using\|export\|import\|include\|begin\|let\)\>/d_' -c '%g/.*/exe "norm >>"' -c ':%p' -c ':q!' | tail -n +2; done | less
```
But after making the switch, it’s much easier to step through the codebase:
>
Of course, I could have written some documentation about the file dependencies, or an IDE to keep track of these, and would use fewer lines of code sticking to the `include()`, but I do like how this system keeps explicit track of dependencies like this.
I should mention that I do still use `include()` for smaller scripts where I only want to split up a few hundred line codebase into component files, but I think when the codebase grows to a certain size and complexity, it is really helpful to have this hierarchical approach available.
Cheers,
Miles
---
### Author:  [@gustaphe](https://discourse.julialang.org/u/gustaphe)
#### Post date: [June 14, 2021, 4:22am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/130 "2021-06-14T04:22:48Z")
That looks like a great use case for submodules.
---
### Author:  [@Raf](https://discourse.julialang.org/u/Raf)
#### Post date: [June 14, 2021, 7:10am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/131 "2021-06-14T07:10:34Z")
I haven’t surveyed the ecosystem, but anecdotally I have never included the same file in two modules or seen it done after working on a lot of packages. I don’t know if this ever actually happens?
I guess you _can_ include the same file in different modules, in which case the code in it may have a different meaning. While I’m not sure why you would do that, you would also be fully aware that you had done it for some reason. And even your example is highly unusual. Because we have multiple dispatch, `subcalc` would nearly always be methods of the same functions dispatching on different types.
So, I can agree that something odd is vaguely possible to do in very unusual code. The question is then how much harm does this cause. If it doesn’t cause any, does it matter? Other languages are riddled with worse foot guns lying around that people just work around, this is more like a foot water-pistol hidden in a box somewhere, that only python people seem to look in.
---
### Author:  [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [June 14, 2021, 8:21am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/132 "2021-06-14T08:21:07Z")
> [@MilesCranmer](#):
>
> But after making the switch, it’s much easier to step through the codebase:
That looks quite similar to
>
To me, it looks like you basically just added something that looks like submodules and liked the result.
---
### Author:  [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [June 14, 2021, 8:22am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/133 "2021-06-14T08:22:49Z")
> [@lamorton](#):
>
> Does anyone ever write conditional `include` statements? Or nested includes?
Yes, it is common (for some definition of common), especially when dealing with platform differences. For example:
>
Nested includes are also quite common. For example when you have a submodule that you have split up into multiple files.
---
### Author:  [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [June 14, 2021, 8:28am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/134 "2021-06-14T08:28:25Z")
Also, it is kind of possible to use files as modules by tweaking the `LOAD_PATH`:
```julia
shell> ls
A.jl B.jl
shell> cat A.jl
module A
import B
print_a() = print("A")
print_b() = B.print_b()
end
shell> cat B.jl
module B
print_b() = print("B")
end
julia> push!(LOAD_PATH, ".")
4-element Vector{String}:
"@"
"@v#.#"
"@stdlib"
"."
julia> using A
[Info: Precompiling A [top-level]
julia> A.print_b()
B
```
This doesn’t really work from packages where you can’t mutate the `LOAD_PATH` but it could be something to think about if something along those lines would be useful.
---
### Author:  [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)
#### Post date: [June 14, 2021, 8:48am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/135 "2021-06-14T08:48:54Z")
@gustaphe @kristoffer.carlsson I think this is close to how it works; here is the FromFile.jl source (only 50 SLOC!): [https://github.com/Roger-luo/FromFile.jl/blob/master/src/FromFile.jl](https://github.com/Roger-luo/FromFile.jl/blob/master/src/FromFile.jl). The differences are: (1) no need to manually wrap each file in a `module ... end`, (2) no `include(...)` statements. But I yield to @patrick-kidger for more…
---
### Author:  [@lamorton](https://discourse.julialang.org/u/lamorton)
#### Post date: [June 14, 2021, 8:58am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/136 "2021-06-14T08:58:05Z")
> [@Raf](#):
>
> this is more like a foot water-pistol hidden in a box somewhere, that only python people seem to look in.
Ha! 😃
> [@Raf](#):
>
> While I’m not sure why you would do that, you would also be fully aware that you had done it for some reason.
Sure, if I had done it, _I_ would know, but how would _you_ know if I had done it? Try to put yourself in the shoes of someone who has to pick up and maintain a legacy codebase. I know Julia isn’t old enough to have much legacy code yet, but it’ll get there. Most Julia devs are doing greenfield work right now, which seems to be reflected in prioritizing features that make it easy to write, but possibly harder to maintain. I always find myself on brownfield projects where I wish the language didn’t have quite so many [attractive nuisances](https://en.wikipedia.org/wiki/Attractive_nuisance_doctrine) for science grad students who taught themselves just enough programming to be dangerous.This experience colors the way I view the tradeoff between power and safety.
To give a concrete example, I once worked with an IDL code where a student had written a single run-on procedure of several thousand lines. Then they broke it up into separate files by using a single `common` block as a ‘God object’ to schlepp all the variables in the namespace from one file to another. When I see a long list of include statements in a package, I start having flashbacks to that nightmare.
> [@Raf](#):
>
> I don’t know if this ever actually happens?
> The question is then how much harm does this cause. If it doesn’t cause any, does it matter?
That’s what I was trying to reason out. As I said above, though, it looks like Julia has safeguards against including the same thing twice into the same namespace. Python has no warning mechanism if you use `exec` to overwrite definitions, which I think goes a long way toward explaining why it’s considered dangerous by folks with Python backgrounds.
> [@Raf](#):
>
> And even your example is highly unusual. Because we have multiple dispatch, `subcalc` would nearly always be methods of the same functions dispatching on different types.
That’s a good point about multiple dispatch. It reinforces my point, which is that one rarely needs the full flexibility that `include` offers.
---
### Author:  [@patrick-kidger](https://discourse.julialang.org/u/patrick-kidger)
#### Post date: [June 14, 2021, 9:01am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/137 "2021-06-14T09:01:24Z")
> [@Raf](#):
>
> So, I can agree that something odd is vaguely possible to do in very unusual code.
The essential argument being made here is that _typical_ code is not as readable as it could be.
> [@Raf](#):
>
> that only python people seem to look in
I think it’d be preferred to avoid this sort of comparison. This isn’t an attempt to make Julia look like Python.
The details being discussed are pretty different to Python.
---
### Author:  [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)
#### Post date: [June 14, 2021, 9:33am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/138 "2021-06-14T09:33:06Z")
I’ve been looking at this discussion from the side, and frankly, I’m confused. What are the actual benefits from the possible change, whatever it is?
---
### Author:  [@patrick-kidger](https://discourse.julialang.org/u/patrick-kidger)
#### Post date: [June 14, 2021, 9:50am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/139 "2021-06-14T09:50:19Z")
In brief: the current approach to organising several files worth of code is something like:
```julia
# master.jl
include("utils.jl")
include("A.jl")
include("B.jl")
# utils.jl
common = 1
# A.jl
foo = common + 2
# B.jl
bar = common + 3
```
in which `A.jl` and `B.jl` implicitly depend upon `utils.jl`.
Being implicit in this way [introduces various issues](https://discourse.julialang.org/t/proper-modules-in-the-future/62646/44). For example it’s hard to tell, whilst reading `A.jl`, what `common` is, where it came from, what it does. (Much of this discussion has focussed around the task of reading or maintaining existing code.)
The hope is to switch this out for a syntax looking something like:
```julia
# master.jl
import "A.jl": foo
import "B.jl": bar
# utils.jl
common = 1
# A.jl
import "utils.jl": common
foo = common + 2
# B.jl
import "utils.jl": common
bar = common + 3
```
to make the dependency structure between files explicit. The main motivation being to try and address the drawbacks previously stated.
---
### Author:  [@gustaphe](https://discourse.julialang.org/u/gustaphe)
#### Post date: [June 14, 2021, 9:57am UTC](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646/140 "2021-06-14T09:57:05Z")
`include`ing the same code twice is bad, but so is typing the same code twice, and there’s nothing to stop that. Somewhere there must be a line where you’re _allowed_ to be a bad programmer.
[Previous page](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646.md?page=6)
[Next page](https://discourse.julialang.org/t/implicitly-loaded-modules-in-the-future/62646.md?page=8)