# What is the best workflow when juggling with many modules and files?

**URL:** <https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067>\
**Category:** General Usage\
**Created:** [January 27, 2021, 5:45pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067 "2021-01-27T17:45:33Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![agatheLG](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@agatheLG](https://discourse.julialang.org/u/agatheLG)\
**Post date:** [January 27, 2021, 5:45pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/1 "2021-01-27T17:45:33Z")

</div>

Hi, I’m a bit unsure about what would be the best way to import a module with `using`. I understand, looking throught past julia topics, that if you want to import a local module, let’s say Users.jl, you should do `include(Users.jl)` and then you can do `using .Users` to get it back and indeed it works. However I saw, again in a past julia topic, that if “you wish to access the features of a Julia _ **package** _ within your code, you must either call `import` or `using` (as opposed to `include`)”. I indeed only want to use features of module however if I don’t `include` it beforehand I have an error saying that Users is undefined. I tried to `push!` to LOAD\_PATH the path of the folder in which the Users.jl module is defined but it did not work and anyway as I rade it doesn’t seem like it’s an ideal solution… In my real case scenario I have a lot of modules, each one being defined inside different folders. LOAD\_PATH seemed initially an ideal way as I thought I wouldn’t have to include one by one all my modules and would just be able to push to it the top level folder containing every subfolders, and then be able to access them every where in the code by doing `using myModule`. But did not work… Right now the code is working but if you have any idea of what the ideal way to proceed would be, I would really appreciate it.  
This is my code structure :  
 ![code_structure](https://global.discourse-cdn.com/julialang/original/3X/7/4/743b55dfa102791de6a2f6dcfb43ea11cd6ba2fa.png)

This is my code :

```julia
#main.jl
cd(@ __DIR__ )
function main()
    include(joinpath("app", "App.jl"))
    Base.eval(Main, :(using Revise))
end; 
main()
using App.Users
using Revise

#App.jl
module App
function main()
    include(joinpath("config", "config.jl"))
end; main()
end

#config.jl
include(joinpath("..", "resources","users","Users.jl"))
#This is what I tried with the LOAD_PATH variable
# const ROOT_PATH = pwd()
# LOAD_PATH = ["@", "@v#.#", "@stdlib", ROOT_PATH]
# appPath = joinpath(ROOT_PATH,"app")
# resourcesPath = joinpath(appPath,"resources")
# usersPath = joinpath(resourcesPath,"users")
# LOAD_PATH = ["@", "@v#.#", "@stdlib",ROOT_PATH, appPath, resourcesPath, usersPath]
#println(LOAD_PATH)
include(joinpath("..","services","service1.jl"))
using .Users

#Users.jl
module Users
export User
mutable struct User 
  id::Integer 
  name :: String
end
end

#service1.jl
using .Users
myUser = User(123,"agathe")
println(myUser)

```

There is different `using` a bit every where in the code to help me understand how to import module depending on where I am but the main goal is to use in _services_ files, modules defined in the _resources_ folder (many modules in real case scenario) , and to include this _services_ files (many files in real case scenario) into the _config.jl_ file. Furthermore this config.jl file is called from an App.jl file via `include`. By mimetism (from other projects) I turned this App.jl into a module that is called from a main.jl file but I think I don’t really get the advantages of doing so… what do I get from my App being a module ? Finally last small precision, I run this code from julia REPL by writing `include("main.jl")`.  
My apologies in advance if my questions are unclear or if I put too many questions in one topic, don’t hesitate to tell me, I will do better next time. Thank you !

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 27, 2021, 5:51pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/2 "2021-01-27T17:51:57Z")

</div>

I don’t think that you can include a file that defines a module inside a function. Modules are top-level construct.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 27, 2021, 6:28pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/3 "2021-01-27T18:28:49Z")

</div>

With Julia 1.5.3 the code you posted did not work. This works:

```julia
#main.jl
cd(@ __DIR__ )
using Revise
include(joinpath("app", "App.jl"))

using .App
using .App.Users

```

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [January 27, 2021, 7:06pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/4 "2021-01-27T19:06:12Z")

</div>

Perhaps the bigger picture question is: do you really need that many modules?

One drawback of having lots of modules is precisely the issue that you are running into now: one has to issue lots of `import` and `export` statements.

Even large packages are often written as a single module (not always). Unless there are name conflicts, dividing the code into separate modules does not really accomplish anything. Unless you plan to reuse a module somewhere else, of course. But then it should be a package.

---

<div class="post-metadata">

**Author:** ![pixel27](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pixel27/32/8902_2.png) [@pixel27](https://discourse.julialang.org/u/pixel27)\
**Post date:** [January 27, 2021, 9:25pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/5 "2021-01-27T21:25:52Z")

</div>

When not using modules for code reuse (i.e. create module X that is used by program A and B) I find that modules are best used to avoid name collision.

That said with Julia you can happily have multiple methods with the same name as long as they have different parameters. If you are not running into name/parameter collisions then you probably don’t need to put everything in modules. However is different people are responsible for different “modules” or directories then maybe modules will help just to avoid accidental name collision.

Regardless of if you do modules or not the way I tend to organize my code is:

```julia
src
     module1
        module11
            _module.jl
            m11_file1.jl
            m11_file2.jl
        _module.jl
        m1_file1.jl
        m1_file2.jl
     module2
         _module.jl
         m1_file1.jl
         m2_file2.jl
    file1.jl
    file2.jl

```

The \_module.jl files really only contain the includes for all the other Julia files in that directory and include the \_module.jl file(s) in the lower directories. Sometimes they define some “high level” objects that the other julia files in that directory need. This way if I add a file to a module the file to add the include too is “near” it.

If I am worrying about name collision, I can then just wrap the includes for a particular \_module.jl file inside a module block and possible put a `using` statement after it i.e.:

```julia
module Module1

include("module11/_module.jl")
include("m1_file1.jl")
include("m2_file2.jl")

export find, path

end

using .Module1

```

That way when I include `Module1/_module.jl` the module is defined in it’s own namespace and the “public” methods are exported automatically.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 27, 2021, 10:13pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/6 "2021-01-27T22:13:58Z")

</div>

Correction: it is possible to `include` inside a function. This may define a module, but it does that in the global scope. So I don’t think that is what you wanted.

---

<div class="post-metadata">

**Author:** ![agatheLG](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@agatheLG](https://discourse.julialang.org/u/agatheLG)\
**Post date:** [January 28, 2021, 9:05am UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/7 "2021-01-28T09:05:19Z")

</div>

I am not sure I understand when you say “It does that in the global scope”, which scope are you talking about ? The module Main or the module in which the function is called ? And either way I am having a hard time understanding why that wouldn’t be something I want, what does it imply ?

---

<div class="post-metadata">

**Author:** ![agatheLG](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@agatheLG](https://discourse.julialang.org/u/agatheLG)\
**Post date:** [January 28, 2021, 9:05am UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/8 "2021-01-28T09:05:59Z")

</div>

Oh yes sorry I forgot to mention it but I am using julia 1.3.1

---

<div class="post-metadata">

**Author:** ![agatheLG](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@agatheLG](https://discourse.julialang.org/u/agatheLG)\
**Post date:** [January 28, 2021, 9:19am UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/9 "2021-01-28T09:19:02Z")

</div>

Good question…Maybe I indeed don’t need that many module. I thought I did as I thought it would be lighter to have small modules and import only the small module I would need in a particular file. Plus each module right now defines a specific object with a struct defining it and functions to access it, that’s why it made more sense to me to have one module for one object. But maybe I just don’t really understand how to use modules yet… Would you suggest that I should have different files, one for each object (my current modules) and then include those files in one big module, which would mean that I would have to import everything everytime I need something from one of those files, but maybe that’s not really an issue. And I don’t think I will have name conflict as I want each module to define a different object. The struct and functions to handle that struct will thus have different names. However I am not sur what you mean by “unless you plan to reuse a module somewhere else”. I indeed plan on reusing the code as I want to use the structs and functions created in my modules in different places. But if you mean in a different project than mine then no I don’t plan on reusing those modules in another project.

---

<div class="post-metadata">

**Author:** ![agatheLG](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@agatheLG](https://discourse.julialang.org/u/agatheLG)\
**Post date:** [January 28, 2021, 9:47am UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/10 "2021-01-28T09:47:49Z")

</div>

I don’t understand why your Module1 would look like that. If “The \_module.jl files really only contain the includes for all the other Julia files in that directory and include the \_module.jl file(s) in the lower directories” then I would have thought to have something like that :

```julia

#_module.jl (the one that module1 sees)
include("m1_file1.jl")
include("m2_file2.jl")
include("_module11/_module.jl")

#module1
module Module1
include("_module.jl")
export ....
end

#file1.jl
using .Module1

```

What do you have in you Module1/\_module.jl ? And what does it imply that “the module is defined in it’s own namespace”

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [January 28, 2021, 12:57pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/11 "2021-01-28T12:57:49Z")

</div>

> [@agatheLG](#):
>
> which would mean that I would have to import everything everytime I need something from one of those files,

That depends on what you call “every time I need”. If that is within the package you are developing, just don’t use the multiple-modules and you do not need to `import` anything, the usual layout is:

```julia
module MyPkg
  export MyStruct, myfunc
  include("./MyStruct.jl")
  include("./myfunc.jl")
end

```

where in `MyStruct.jl` you just define the structure (without any module) and in `myfunc.jl` the function, or functions. How to split the code into files is more or less at will, that depends on how you think the code is better organized (I use to put one function in each file, but that is not the most common pattern here, apparently).

If, on the other side, you want, for example, that `MyStruct` is available to other packages, or independently for being used without `myfunc`, you can define a package only for that. For example:

```julia
module MyStructsPkg
  export MyStruct
  include("./MyStruct.jl")
end

```

Then, the previous one would be

```julia
module MyPkg
  using MyStructsPkg
  export myfunc
  include("./myfunc.jl")
end

```

and you could have another package that uses `MyStructsPkg` and does something else, or you could just load `MyStructsPkg` in the REPL and use those structs directly. That is pretty common, actually, there are many packages that define custom structures (Data Types). In general they define the functions that allow the most common operations that one user would want to do with that data type.

But that splitting in packages is only interesting if your new type is really interesting outside the context of the package you are developing.

---

<div class="post-metadata">

**Author:** ![pixel27](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pixel27/32/8902_2.png) [@pixel27](https://discourse.julialang.org/u/pixel27)\
**Post date:** [January 28, 2021, 2:47pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/12 "2021-01-28T14:47:34Z")

</div>

When I organize things a module only calls sibling or children, it doesn’t access children of siblings (directly). Sometimes a sibling might “re-export” a method or structure of it’s child, but that it. This is just the way I end up organizing, it might be because of my Java background, I don’t really know. If a module needs to be accessed by an uncle or grand parent then I usually move it up in the hierarchy.

Any function/structures defined in a module are in their own namespace, meaning their function/structure names won’t conflict with the functions/structures in other modules. So Module1 and Module2 can both define their own version of a structure Foo without any issues. If Foo is exported by Module1 and Module2 then you might start having issues, but as long as Foo is only used internally nothing bad happens.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 28, 2021, 4:28pm UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/13 "2021-01-28T16:28:37Z")

</div>

As the manual [states](https://docs.julialang.org/en/v1/base/base/#Base.include):

 ![image](https://global.discourse-cdn.com/julialang/original/3X/b/1/b103f5f5b04c280bfb8560a8d27bd2f1d51c9f15.png)

So if you have function `main` and that function includes a source file, whenever you call that function, the source file will be evaluated in the global scope. It seems to me that what you wanted was to run the application from within `main`, not to prepare the source code for execution. (I may be wrong, of course.)

The way I restructured your code above, the source code was evaluated once, making the modules available for use.

---

<div class="post-metadata">

**Author:** ![agatheLG](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@agatheLG](https://discourse.julialang.org/u/agatheLG)\
**Post date:** [January 29, 2021, 11:50am UTC](https://discourse.julialang.org/t/what-is-the-best-workflow-when-juggling-with-many-modules-and-files/54067/14 "2021-01-29T11:50:09Z")

</div>

All right thank you for all your advices. Lots to think about…