# Modular local projects and \`include(...)\`

**URL:** <https://discourse.julialang.org/t/modular-local-projects-and-include/95781>\
**Category:** New to Julia\
**Tags:** question, modules, code-organization\
**Created:** [March 9, 2023, 3:23am UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781 "2023-03-09T03:23:28Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![tgross35](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tgross35/32/38591_2.png) [@tgross35](https://discourse.julialang.org/u/tgross35)\
**Post date:** [March 9, 2023, 3:23am UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/1 "2023-03-09T03:23:29Z")

</div>

Are local files/modules ever immediately importable with `using` , or is `include` always required?

For example, with a structure like this:

```julia
Project.toml
src/ # Source for a local library `MyLib`
    MyLib.jl # contains `module MyLib`
    SomeModule.jl # contains `module SomeModule`
    SomeModule/
        Foo.jl
        Bar.jl
runnable_file_1.jl # these two rely on MyLib.jl
runnable_file_2.jl

```

Is all of the following necessary?

- `runnable_file_1` and `2` have `include("src/MyLib.jl"); using MyLib`
- `MyLib.jl` has `include("SomeModule.jl"); export SomeModule`
- `SomeModule.jl` has `include("SomeModule/Foo.jl"); export Foo` and the same for bar

It seems to me that there might be a way for directory structure to create module structure without manually including then exporting everything - such that a top level runnable file could simply `using MyLib.SomeModule.Foo` without needing the `include` statement at all (ala Python with ` __init__.py` or Rust with `pub mod`).

Is something simpler possible? Alternatively, is there a better structure for a local package than what I describe?

---

<div class="post-metadata">

**Author:** ![patrick-kidger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrick-kidger/32/20378_2.png) [@patrick-kidger](https://discourse.julialang.org/u/patrick-kidger)\
**Post date:** [March 9, 2023, 4:16am UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/2 "2023-03-09T04:16:31Z")

</div>

This question comes up a lot. The “best” way to do this is a contentious topic around here.

Personally I recommend [FromFile](https://github.com/Roger-luo/FromFile.jl), which does what you’re after.

(Others prefer some `include` + `using` chicanery, which has a lot of boilerplate and suffers from a number of footguns. You can tell which side of this war I’m on.)

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [March 9, 2023, 8:35am UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/3 "2023-03-09T08:35:44Z")

</div>

> [@tgross35](#):
>
> `runnable_file_1` and `2` have `include("src/MyLib.jl"); using MyLib`

No, your scripts in `runnable_file_?.jl` should merely do `using MyLib`, and be executed with `MyLib` being the “active” project. I.e in order to run `runnable_file_1.jl`, you can

- from the command line, if the current directory is the “top directory” of the project:

```julia
shell> julia --project runnable_file_1.jl

```

- from within a julia REPL:

```julia
julia> ] # press `]` to enter Pkg mode
(@v1.8) pkg> activate .
(MyLib) pkg> # press <backspace> to exit Pkg mode
julia> include("runnable_file_1.jl")

```

> [@tgross35](#):
>
> `MyLib.jl` has `include("SomeModule.jl"); export SomeModule`

In the standard way of doing things, `MyLib.jl` should definitely have `include("SomeModule.jl")`. What else it has depends on what you want to do. Assuming `SomeModule` exports a function `foo`:

- if you do nothing:
  - `MyLib` can refer to `foo` as `SomeModule.foo`
  - `runnable_file_1.jl` can refer to `foo` as `MyLib.SomeModule.foo`

- if `MyLib` does `using .SomeModule` (mind the `.` in front of the submodule name):
  - `MyLib` can refer to `foo` as simply `foo`
  - `runnable_file_1.jl` can refer to `foo` as `MyLib.foo`

- if `MyLib` does `export SomeModule`
  - it does not change anything for the code written in `MyLib.jl`
  - `runnable_file_1.jl` can refer to `SomeModule` as simply `SomeModule` (instead of `MyLib.SomeModule`)

> [@tgross35](#):
>
> is there a better structure for a local package than what I describe?

I’m not sure what “better” means, but I’ll say that a lot of packages only define the mandatory top-level `module` (`MyLib` in your example), but no submodule at all. Although most packages are sufficiently large that splitting their source code in multiple files, possibly grouped into subdirectories. Julia allows the two notions (file/directory structure on the one side, and module/submodule hierarchy on the other side) to be entirely decoupled, which allows package developers to adopt a lot of different structures.

Using `FromFile` is perfectly fine. Another way of doing things, relying only on standard Julia features and provided that you don’t absolutely need to insulate the various parts of your project in different namespaces, would look like:

```julia
Project.toml
src/
    MyLib.jl
    feature1.jl
    feature1/
        foo.jl
        far.jl
runnable_file_1.jl

```

with:

```julia
#file MyLib.jl
module MyLib
export a_user_facing_function # this export statement could have been placed somewhere else, for example right before the definition of `a_user_facing_function` in `feature1/foo.jl`

include("feature1.jl")

```

```julia
#file feature1.jl
include("feature1/foo.jl")
include("feature1/bar.jl")

```

```julia
#file feature1/foo.jl
a_user_facing_function(x) = 2*an_internal_function(x)
an_internal_function(x) = x + 1

```

```julia
#file runnable_file_1.jl
using MyLib

# can use a_user_facing_function directly because it was exported by MyLib
println(a_user_facing_function(42))

```

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [March 9, 2023, 9:25am UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/4 "2023-03-09T09:25:12Z")

</div>

> [@ffevotte](#):
>
> - from the command line, if the current directory is the “top directory” of the project:
> 
> ```julia
> shell> julia --project runnable_file_1.jl"
> 
> ```
> 
> - from within a julia REPL:
> 
> ```julia
> julia> ] # press `]` to enter Pkg mode
> (@v1.8) pkg> activate .
> (MyLib) pkg> # press <backspace> to exit Pkg mode
> julia> include("runnable_file_1.jl")
> 
> ```

Another way is to start the runnable script with

```julia
using Pkg
Pkg.activate(@ __DIR__ )
Pkg.instantiate()

```

The third line is optional but means that dependencies will be automatically installed if they are missing, e.g. when running this the first time on another machine.

---

<div class="post-metadata">

**Author:** ![tgross35](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tgross35/32/38591_2.png) [@tgross35](https://discourse.julialang.org/u/tgross35)\
**Post date:** [March 9, 2023, 8:25pm UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/5 "2023-03-09T20:25:24Z")

</div>

Interesting, `FromFile` and the specification it mimics sounds like at least a syntax that’s easier to follow than the status quo. Thanks for the link!

Just for curiosity - was there ever a proposal for anything that builds the module tree off of paths, rather than needing the `module Foo` within the included file? I’d personally be partial to something similar to rust - e.g. `module SomeModule` / `export module SomeModule` creates a module from file `SomeModule.jl`, and anything in a same-named folder could be used as submodules (if also declared). Kinda similar to Python, but without needing ` __init__.py` and with easier reexporting

---

<div class="post-metadata">

**Author:** ![tgross35](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tgross35/32/38591_2.png) [@tgross35](https://discourse.julialang.org/u/tgross35)\
**Post date:** [March 9, 2023, 8:53pm UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/6 "2023-03-09T20:53:14Z")

</div>

Thanks for the thorough response. Your example is kinda sorta what I’m currently doing, but it just seemed very clunky to need to use `include` + `export` + `module` (with the same file name) to build project structure.

> Julia allows the two notions (file/directory structure on the one side, and module/submodule hierarchy on the other side) to be entirely decoupled, which allows package developers to adopt a lot of different structures.

This is probably the context that I was missing, coming from languages that typically have them closely coupled. [this page](https://docs.julialang.org/en/v1/manual/modules/) does sort mention that:

> Files and file names are mostly unrelated to modules; modules are associated only with module expressions. One can have multiple files per module, and multiple modules per file.

But doesn’t talk about files/import again, and doesn’t clarify why it’s “mostly unrelated” and not completely unrelated.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [March 9, 2023, 9:07pm UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/7 "2023-03-09T21:07:55Z")

</div>

> [@tgross35](#):
>
> Just for curiosity - was there ever a proposal for anything that builds the module tree off of paths, rather than needing the `module Foo` within the included file?

> [@tgross35](#):
>
> it just seemed very clunky to need to use `include` + `export` + `module` (with the same file name) to build project structure.

Yes, while the currently available (or default) design patterns have served many Julia users well, I think they are a bit of a sore point for a sizeable chunk of other users. There are quite a few GH issues with a long & active debate on what exactly would be the best design moving forward. I am optimistic that \_\_eventually\_\_ there will be a resolution which will mean users write a lot fewer `include(...)`s, but I wouldn’t hold your breath 🙂

---

<div class="post-metadata">

**Author:** ![tgross35](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tgross35/32/38591_2.png) [@tgross35](https://discourse.julialang.org/u/tgross35)\
**Post date:** [March 9, 2023, 10:15pm UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/8 "2023-03-09T22:15:33Z")

</div>

Well… I’m just glad to know I’m not the only one who struggles a bit 🙂 I’m just a bit surprised these answers and the “best practices” are a bit tricky to track down.

---

<div class="post-metadata">

**Author:** ![patrick-kidger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/patrick-kidger/32/20378_2.png) [@patrick-kidger](https://discourse.julialang.org/u/patrick-kidger)\
**Post date:** [March 10, 2023, 2:59am UTC](https://discourse.julialang.org/t/modular-local-projects-and-include/95781/9 "2023-03-10T02:59:00Z")

</div>

> [@tgross35](#):
>
> Just for curiosity - was there ever a proposal for anything that builds the module tree off of paths, rather than needing the `module Foo` within the included file?

For clarity (in case this is what you were thinking), FromFile doesn’t need a `module Foo` inside the include file.

Meanwhile as others have commented, the alternate approach to this usually involves creating just a single `module` for the whole project, and then just `include`ing everything into that one giant namespace. So this again doesn’t involve creating any extra modules inside files.

In terms of discussions: the main one on this topic is [#4600](https://github.com/julialang/julia/issues/4600).