# Restructuring code from modules to package

**URL:** <https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819>\
**Category:** General Usage\
**Tags:** packages\
**Created:** [October 15, 2021, 11:45am UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819 "2021-10-15T11:45:24Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 15, 2021, 11:45am UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/1 "2021-10-15T11:45:25Z")

</div>

I have developed an application that makes use of 20-odd modules. Coming from Matlab, I have ensured that Julia finds my code by adding relevant folders to the `LOAD_PATH`. The user uses my application by `using` relevant modules. The dependencies between my modules follows a (relatively complex) direct acyclic graph.

The time has come to _package_ the code to make is distributable. I have the right folder structure, with `/src`, `/test`, `Project.toml` and so on, and I “`] dev`” the project.

Trouble appears when I zero the `LOAD_PATH`: my primary module (which has the same name as the project) does not find the secondary modules (that are neatly stored in `\src`).

Whence my questions:

1. Can a package make more than one module available for `using` (by both the user and other parts of the package)?
2. Specificaly, is there a way, to make the package, on compilation, to add the relevant folder to the `LOAD_PATH`? Edit: _for the purpose of making the secondary modules available_.
3. `include`ing modules as submodules into the primary module and exporting them would be a way (with small API changes, no problem) to make secondary modules available to the user. I have not tested (lazy) but I am concerned that making a secondary module access another secondary in this way (through the primary module) creates a cycle in the dependency graph (the primary module uses or includes ‘all’ secondary modules), and so I can not quite see that working.
4. Am I persisting in unJulian ways, and must think one GIT repo==one package== one module, and boil things down to 3-4 packages with dependencies (some work for me to get my code there…)

Thank you in advance!

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [October 15, 2021, 12:34pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/2 "2021-10-15T12:34:21Z")

</div>

I think some more info about the structure of the code is needed to make useful suggestions.

However, 20+ modules seems like a lot. I suspect that you would be better off combining most of them into a much smaller set of modules.

If you want multiple modules in one package, the usual approach would be submodules.  
The main file would contain something like

```julia
module MyPackage

include("sub1.jl");
include("sub2.jl");
using .Sub1, .Sub2

end

```

and in `sub2.jl`:

```julia
module Sub2

using ..Sub1

end

```

With 20 modules, this gets hard to follow pretty soon.

Typically, there is either a subset of modules that are of independent use. These make natural candidates for a package (with 1 module). Or there isn’t a natural second package. Then the question is: should you simply have one module? What would be the drawback of this?

Another structure is to generate a module that contains `struct` definitions and stubs for methods. Other modules then use this module and extend the methods. Whether that fits your use case is hard to say without knowing more about the structure of the code.

On your second question: once you made a package, you add it to your environment and that takes care of code loading (including all dependencies). So the `LOAD_PATH` is no longer something you need to worry about.

On the fourth question: there is now a way of having multiple packages in one repo, but I think that’s a separate issue from code organization.

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 15, 2021, 12:43pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/3 "2021-10-15T12:43:47Z")

</div>

Hi,

Well, my code structure… coming from Matlab I started off with one module== one object, not that I insist any more.

I have edited my question #2: would the loadpath be a way _of making the secondary modules available_

Separate question, but a pointer to the doc on having multiple packages in a repo would interest me very much.

But yes, I do see how “one package==one module” constrains… correction: gently guides towards a modular design.

😀

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 15, 2021, 1:01pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/4 "2021-10-15T13:01:19Z")

</div>

Since you asked, here is the `using` graph (but I do not expect anyone to bother with this list).

Lithe is a “just add your element” FE code. My idea is a that I new element family will be programmed - you know me - as a new module.

```julia
# Demo code for element developer, could thus be left out of the package 
Use(:SimpleTetragonElement, [:Lithe, :Espy, :Dialect,:Elemental, :Dots, :Materials])
Use(:SimplestTetragonElement, [:Lithe, :Espy, :Dialect, :Dots])
Use(:VolumeElement, [:Lithe, :Espy, :Dialect,:Elemental, :Dots, :Materials])
Use(:VolumeMaterial, [:Lithe, :Espy, :Dialect, :Dots, :Adiff, :NewtonRaphson, :Materials])

# Core functionality, solve, display, export results
Use(:ElementTestBench, [:Lithe, :Dialect, :Espy])
Use(:LitheGraphics, [:Lithe, :Dialect])
Use(:GraphicMesh, [:Dialect])
Use(:Lithe, [:Dialect, :Adiff, :Espy])

# Metaprogramming operations on element code, Espy and Express could be merged.  
Use(:Espy, [:Dialect, :Express])   
Use(:Express, [:Dialect])             

# components the element developer might want to use
Use(:NewtonRaphson, [:Dialect, :Adiff, :Dots])
Use(:Elemental, [:Dialect])
Use(:Materials, [])

# used everywhere
Use(:Adiff, [:Dialect]) # could be a package, my take on automatic differentiation
Use(:Dots, [:Dialect])     
Use(:Dialect, [])

# used when writing the input file (Julia script) of a FE analysis. 
Use(:MeshReader, [:Dialect]) # thin wrap around AbaqusReader
Use(:Unit, [:Dialect]) # could be a package

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 15, 2021, 1:02pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/5 "2021-10-15T13:02:45Z")

</div>

> [@Philippe\_Maincon1](#):
>
> I have edited my question #2: would the loadpath be a way _of making the secondary modules available_

This is maybe a question, maybe an answer, but doesn’t something like

```julia
module Main
    include("./module1/Module1.jl")
    include("./module2/Module2.jl")
    using .Module1, .Module2
    ...
end

```

just works if `Main` is in `src` and ` module1` and `module2` are subdirs of ` src` ?

But in Julia this is more of a headeach. Modules are more useful to separate a set of structures and functions all that define a more or less closed set of functionalities.

You probably find the answer to the second question here:

> [@Usage of subdirectories to store multiple packages in a single repo](https://discourse.julialang.org/t/usage-of-subdirectories-to-store-multiple-packages-in-a-single-repo/55534/2):
>
> Many/most of these questions can probably be answerd by this: Packages in subdirectories are just regular packages. They just happen to live in the same git repo. Yes they need to be packages on their own with Project.toml, src/Package.jl entrypoint etc. Probably this structure makes the most sense: $ tree . . ├── PackageA │ ├── Project.toml │ └── src │ └── PackageA.jl └── PackageB ├── Project.toml └── src └── PackageB.jl 4 directories, 4 files or perhaps $ tree …

though I think it is unlikely that this will be a good alternative to that of simply flattening the package, unless your package is already really big.

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 15, 2021, 1:08pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/6 "2021-10-15T13:08:43Z")

</div>

Hi, thanks,

I was thinking in the same direction

```julia
module Main
    include("./module1/Module1.jl")
    include("./module2/Module2.jl")
    using .Module1, .Module2
    export Module1, Module2
    ...
end

```

so the user can do

```julia
using Main
Module2.foo()

```

which is a nice syntax.  
But now: Module2 uses Module1.

```julia
module Module2 
using Module1
end

```

does not work (not on the path and not an installed module/package and

```julia
module Module2
using Main
end

```

which I haven’t tried, smell of cyclic dependency.

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 15, 2021, 1:09pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/7 "2021-10-15T13:09:40Z")

</div>

And thank you for the link to the multiple packages in a repo!!! 😀

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 15, 2021, 1:11pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/8 "2021-10-15T13:11:44Z")

</div>

> [@Philippe\_Maincon1](#):
>
> which I haven’t tried, smell of cyclic dependency.

Definitely that is not a good idea.

If you think the user can use `Module1` without ` Main`, than make of `Module1` a package and add it as a dependency of `Main`.

Also, it is possible that you are overusing modules to avoid name conflicts in a way that is not needed by Julia because of multiple dispatch, one should not think of a module as an “Object”.

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 15, 2021, 1:14pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/9 "2021-10-15T13:14:11Z")

</div>

I clearly _am_ overusing modules, I now see 😁

20odd modules will become 4odd packages, living, to begin with, in the same repo. I have something to look forward to over the week end!!!

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [October 16, 2021, 3:38am UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/10 "2021-10-16T03:38:24Z")

</div>

Unfortunately, I don’t fully understand your issue, but I believe I can point you to some useful resources.

I recently wrote a high-level overview of modules & packages in an attempt to alleviate confusion:  
 → [Could we make first-class support for packages that are "just files" and not repositories? - #20 by MA\_Laforge](https://discourse.julialang.org/t/could-we-make-first-class-support-for-packages-that-are-just-files-and-not-repositories/36931/20)

Somewhat similar (but less well explained in my opinion - read this last - if at all):  
 → [How to add a folder path in your .jl script - #8 by MA\_Laforge](https://discourse.julialang.org/t/how-to-add-a-folder-path-in-your-jl-script/61901/8)

Creating your first package (incl. file organization)  
 → [How does the module system actually work? - #3 by MA\_Laforge](https://discourse.julialang.org/t/how-does-the-module-system-actually-work/53548/3)

Tips for include vs import/using  
 → [Proper way of organizing code into subpackages - #3 by MA\_Laforge](https://discourse.julialang.org/t/proper-way-of-organizing-code-into-subpackages/52835/3)

Tips for directory structure/`LOAD_PATH`  
 → [Proper way of organizing code into subpackages - #4 by MA\_Laforge](https://discourse.julialang.org/t/proper-way-of-organizing-code-into-subpackages/52835/4)

Link to more relevant threads:  
 → [Proper way of organizing code into subpackages - #5 by MA\_Laforge](https://discourse.julialang.org/t/proper-way-of-organizing-code-into-subpackages/52835/5)

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [October 17, 2021, 1:22pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/11 "2021-10-17T13:22:05Z")

</div>

I think the key questions are:

- What is the benefit of having more than 1 module in your case? Name conflicts? Easier to understand code logic?
- Will users use the sub-modules independently? Or will they always use the main module?

If users use sub-modules independently, this looks like a case for a separate package.  
Given your description of complex interdependencies of the modules, it sounds like having multiple modules complicates reasoning about the code. Then I would likely get rid of the sub-modules entirely.

I actually went through something quite similar to your experience when I transitioned from matlab to Julia. I started with lots of modules (easier to reason about, I thought). Then I ended up with confusing `import`s all over the place. Finally, I figured out what part of the code was truly independent of the rest and factored that out into a couple of separate packages (which will never be used independently; it’s all one big model). But the packages help reasoning about the code and also with editing and testing (testing is faster).

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 18, 2021, 5:28am UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/12 "2021-10-18T05:28:08Z")

</div>

MA\_Laforge, I will definitely read this, thank you!

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 18, 2021, 5:36am UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/13 "2021-10-18T05:36:59Z")

</div>

> I actually went through something quite similar to your experience when I transitioned from matlab to Julia. I started with lots of modules (easier to reason about, I thought). Then I ended up with confusing `import` s all over the place.

This is a good summary of my own story!

Name conflicts was not, generally, pushing me to create modules.

When cleaning up, I will keep some packages separate: they provide functionality that may be useful outside my current project. My project will reexport relevant functionality from these packages.

My project has the peculiarity of having two types of users: those that will create new finite element types, and those that will use project+new elements. To avoid cluttering the later users namespace, I am considering segregating the functionality that supports element development in its own submodule (or simply not exporting it…).

Thank you Hendri for your kind help!

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [October 18, 2021, 1:07pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/14 "2021-10-18T13:07:56Z")

</div>

I would make the code that is independently usable into a package rather than a sub-module. It’s easier for the user and (I think) also easier for the code author.

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [October 18, 2021, 1:17pm UTC](https://discourse.julialang.org/t/restructuring-code-from-modules-to-package/69819/15 "2021-10-18T13:17:51Z")

</div>

Yep, I see that.  
I might be doing a mistake, but I think of going through an intermediate step with a single Package, get that to work, and then split into packages to maximize re-usability.