# Dependencies of src files inside a package

**URL:** <https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144>\
**Category:** General Usage\
**Tags:** dependencies\
**Created:** [July 15, 2020, 7:44pm UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144 "2020-07-15T19:44:26Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [July 15, 2020, 7:44pm UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/1 "2020-07-15T19:44:26Z")

</div>

I am struggling with dependencies of several files inside a project (MWE package M). I have illustrated the current structure (of a MWE), which reflects the natural separation of different parts of the real project:  
 ![dependency](https://global.discourse-cdn.com/julialang/original/3X/5/7/57084697df1dfbf7afa5c646367e3e49120d24f9.jpeg)

For the MWE the package is named M with top file M.jl.  
It calls a function `dfunc` defined in d.jl.  
`dfunc` calls `bfunc` and `cfunc` from b.jl and c.jl.  
A common data type is `Astruct`, defined in a.jl and used directly in b.jl,c.jl and d.jl.

The complete MWE package can be installed with  
`add https://github.com/oheil/M.jl`  
Currently every file (a.jl to M.jl) defines a separate module (A,B,C,D,M). I include the files and do a `using` where its needed, e.g.:

```julia
include("a.jl")
using .A

```

With this setup, when I try to load the package M I get:

```julia
julia> using M
[Info: Precompiling M [d58019f5-37a3-4da3-a62f-a442f1098211]
ERROR: LoadError: MethodError: no method matching bfunc(::M.D.A.Astruct)
Closest candidates are:
  bfunc(::M.D.B.A.Astruct) at C:\Users\oheil\.julia\dev\M\src\b.jl:8

```

Other setups, like only defining sub-module A, and just including other files do result in other errors or warnings, e.g.  
`cannot replace module A during compilation`

I have tried several different setups, e.g. only sub-moduling A, but nothing gave an error or warning free working call to `dfunc()`.

How do I solve this dependency problem? What is the proper way of sub-moduling, including and using? (I didn’t tried with import)

I hope I don’t have to rearrange everything, because the real project is much more complex and the above structure, as already said, reflects the different logical entities of the project.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [July 15, 2020, 8:48pm UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/2 "2020-07-15T20:48:26Z")

</div>

The problem is that you’re `include()`ing the same file multiple times. Each time you `include("a.jl")`, you create a _new_ module named `A`, with new, incompatible, types and functions. You need to ensure that each file is included exactly once.

For example, you could do:

```julia
## M.jl
module M

include("A.jl")
using .A
include("B.jl")
using .B

end

## A.jl
module A
  ...
end

## B.jl
module B

using ..A

end

```

Note that we can still _use_ module `A` within module `B` without _including_ it again there.

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 16, 2020, 3:17am UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/3 "2020-07-16T03:17:21Z")

</div>

Technically, I agree with @rdeits. However, I’m not sure that’s really the solution you are looking for.

### My experience

I have experienced the same confusion myself, and I have noticed that others seem to be stuck in the same mindset. I think it might stem from our ingrained C/C++'s include/header file methodology (assuming that’s your background).

In most cases, I don’t really want to break things up into different modules - especially inside a single package. Typically, when I need to break things up into separate modules, I really end up making _ **one package per module** _.

If I’m not mistaken, there is a good chance that what you consider to be a package might actually be some sort of _**repository that collects somewhat related import-able modules (called “packages” in julia)**_. Don’t get me wrong, `Module`s still exist in Julia, but they seem to correspond more to namespaces in C++.

### Suggested structure, given your MWE

I tend use my .jl files (as you showed) just to separate out different functional blocks, and keep my files to a more (mentally) manageable size.

So yes, I would define files a.jl, b.jl, c.jl, and d.jl as follows:

a.jl:

```julia-auto
struct Astruct
   #...
end

#more related code ...

```

b.jl:

```julia-auto
bfunc(a::Astruct) = dosomething(a)

#more related code ...

```

c.jl:

```julia-auto
cfunc(a::Astruct) = dosomethingelse(a)

#more related code ...

```

d.jl:

```julia-auto
function dfunc()
   a = Astruct()
   bresult = bfunc(a)
   cresult = cfunc(a)
   return dosomethingwith_bandc_result(bresult, cresult)
end

#more related code ...

```

Finally, I would `include` all these files in the master M.jl file.

Note that `[pkg_loadpath]/[PkgName]/src/[PkgName].jl` is the file Julia automatically loads when the caller imports your module with either `import PkgName` or `using PkgName`.

See [Code Loading · The Julia Language](https://docs.julialang.org/en/v1/manual/code-loading/#Package-directories-1) for more details on how packages are loaded.

Well, that’s at least true if you add `[pkg_loadpath]` to Julia’s `LOAD_PATH` variable, or if your CWD is in `[pkg_loadpath]`. See [Constants · The Julia Language](https://docs.julialang.org/en/v1/base/constants/#Base.LOAD_PATH) for more details on `LOAD_PATH`.

In any case, this is what my M.jl would look like:

```julia-auto
#But yes, your "Julia package" code needs to be wrapped inside
#a Module of the same name:
module M

#Import some "Julia packages":
#(Which you probably typically think of as "modules")
using FFTW
using SomeOtherPackage1
using SomeOtherPackage2

#include the all the functionality broken up in separate files:
include("a.jl")
include("b.jl")
include("c.jl")
include("d.jl")

function run_the_fancy_calculation
   #setup constants
   result = dfunc()
   #more computations
   return result
end

#more related code ...

end

```

### Hope this helps

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [July 16, 2020, 11:28am UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/4 "2020-07-16T11:28:21Z")

</div>

Thank you, both solutions work (at least for the MWE).

Trying the [version using modules](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/2) is still not satisfactory and I remember now, why I initially started with sub-modules in the real project. The reason why I am still unhappy with this is that VSCode complains about missing references:  
 ![vscode](https://global.discourse-cdn.com/julialang/original/3X/1/5/151c8948332dde31c1cb09b7a4af33bb84d1ff5d.jpeg)  
which is annoying despite that the complete package is compiling and running fine.  
Thats what I got in the real project after some increasing complexity and I solved it using sub-modules and later on I ran into the dependency problem above.

When I try the [includes only solution](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/3) VSCode doesn’t complain at all (for the MWE), which is fine, but now I don’t understand why it doesn’t complain and why I had the issue in the real project and why I found that sub-modules help to make the LanguageServer know about the symbols.  
Of course you can’t tell what happened in my real project.

But maybe someone know why the LanguageServer doesn’t complain about `::Astruct` in the case, when no sub-modules exist and e.g. `b.jl` looks just like:

```julia
#module B
#export bfunc
#include("a.jl")
#using ..A

function bfunc(::Astruct)
end

#end

```

Hovering over `::Astruct` shows the correct definition but I can’t “Go to definition”.

For now I try to make my real project running without sub-modules. Therefor, if you don’t mind, I am going to mark the includes only solution as this.

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 16, 2020, 3:42pm UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/5 "2020-07-16T15:42:04Z")

</div>

Unfortunately, I haven’t tried VSCode yet. I should: I hear it works well.

But from what I can gather, it sounds like the Julia language definition used in VSCode has a few limitiations.

In the “version using modules” problem, it almost sounds like the Language Server Protocol definition does doesn’t know about relative module paths. Note that I say this only because _ **relative** _ module paths is the least-typically-used construct in Julia – at least it is given the simplicity of your MWE.

In the “includes only solution”, I’m much more surprised. Maybe the Language Server Protocol implementation still has issues tracking `struct` definitions across files???

### Raising an issue

You might need to raise an issue here: [GitHub - julia-vscode/LanguageServer.jl: An implementation of the Microsoft Language Server Protocol for the Julia language.](https://github.com/julia-vscode/LanguageServer.jl)

Of course, I suppose the limitation might be in the implementation on the VSCode side, but if that’s the case, the maintainers of LanguageServer.jl will probably try to get that resolved if you inform them.

### Thanks for that @oheil !

I didn’t even know about this Language Server Protocol, or that it was supported by VI!

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [July 16, 2020, 4:08pm UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/6 "2020-07-16T16:08:44Z")

</div>

After cleaning up my original project, all is working now as expected. Actually I still have mixed sub-modules and includes and all is good.

> [@MA\_Laforge](#):
>
> `Module` s still exist in Julia, but they seem to correspond more to namespaces in C++.

Viewing modules (in my case a package internal sub-module, and the main module same as the package name) as namespaces helped a lot to do it right and to understand the former error with

```julia
bfunc(::M.D.A.Astruct)

```

vs

```julia
bfunc(::M.D.B.A.Astruct)

```

This is now totally clear as

```julia
## b.jl
module B
include("a.jl")
using .A

end

```

introduced the `...B.A.Astruct`, which wasn’t intended to exist.

> [@MA\_Laforge](#):
>
> ### Raising an issue
> 
> You might need to raise an issue here: [GitHub - julia-vscode/LanguageServer.jl: An implementation of the Microsoft Language Server Protocol for the Julia language.](https://github.com/julia-vscode/LanguageServer.jl)
> 
> Of course, I suppose the limitation might be in the implementation on the VSCode side, but if that’s the case, the maintainers of LanguageServer.jl will probably try to get that resolved if you inform them.

I think I missinterpreted something, so it’s more an issue on my side than at Language Server. Except for the “Go to definition” not working, but displaying the proper definition while hovering. After doing a “Find all References” the “Go to definition” worked again as expected. So, there seems to be a minor bug, I will investigate further to make it really reproducible, and if I succeed surely will open an issue.

Thanks again to you @MA_Laforge and @rdeits !

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 16, 2020, 5:05pm UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/7 "2020-07-16T17:05:16Z")

</div>

> [@oheil](#):
>
> Thats what I got in the real project after some increasing complexity and I solved it using sub-modules and later on I ran into the dependency problem above.

Ah, so here it sounds like you actually _ **were** _ trying to create a multi-“module” solution, and its just that the above MWE was a toy example.

### Building a ~~Multi-Module~~ Multi-Package Solution

First, I want to remind you that a Julia “Module” is actually more akin to a C++ namespace.

So if you wish to break down your solution into what is typically known as “modules” in computer science-talk, you need to start talking of Julia “packages”:

| Computer  
Science | Julia | C++ |
| --- | --- | --- |
| Arbitrary  
Scope | `begin` … `end` | `{` … `}` |
| Function-level  
Scope | `function x()` … `end` | `int x() {` … `}` |
| Module-level  
Scope | `module MyMod` … `end` | `namespace MyNS {` … `}` |
| Module | Package | .h header file  
w/compiled library |

### Package directory

So the first thing you need to do is find a location to develop your packages. Then, you need to tell Julia where to find them by adding it to the `LOAD_PATH` global variable.

You can automatically specify the package location either by setting the shell environment variable `JULIA_LOAD_PATH`, or by directly adding to `LOAD_PATH` in your `~/.julia/config/startup.jl` file.

- See [Constants · The Julia Language](https://docs.julialang.org/en/v1/base/constants/#Base.LOAD_PATH) for more details on `LOAD_PATH`.
- See [Code Loading · The Julia Language](https://docs.julialang.org/en/v1/manual/code-loading/#Package-directories-1) for more details on package directories and code loading, in general.

### Structure of package “library”

The basic structure of a package “library” is as follows:

```julia-auto
[path/to/shared/package/root]
├── A
│ └── src
│ └── A.jl
├── B
│ └── src
│ └── B.jl
├── C
│ └── src
│ └── C.jl
├── D
│ └── src
│ └── D.jl
└── M
    └── src
        ├── M.jl
        ├── logicalunit1.jl
        ├── ...
        └── logicalunitn.jl

```

Where the `logicalunit?.jl` files simply allow you to break down your package implementation with more manageable file sizes.

To access these packages, you would register the above “library” with your `~/.julia/config/startup.jl` file:

```julia-auto
push!(LOAD_PATH, "path/to/shared/package/root")

```

### Package definitions & `using`/`import`

In the toy MWE you gave, there is no real need to break things down into `logicalunit?.jl` files. So it would now be important to point out that what I previously showed you to was how `include` multiple `logicalunit?.jl` _ **files** _ to form a single _ **package** _.

However, when you wish to access the features of a Julia _ **package** _ within your code, you must either call `import` or `using` (as opposed to `include`). This is illustrated in the sample files below.

A/src/A.jl:

```julia-auto
module A #Package code must be wrapped inside a "Module"

struct Astruct
   #...
end

#more code ...

#Export symbols you don't want the user to
#explicitly qualify with A.Astruct, etc.:
export Astruct
end

```

B/src/B.jl:

```julia-auto
module B
using A #That's right, you don't "include" a package.

#You could also "import A" if you are ok with
#explicitly qualifying "exported" symbols, like
#"A.Astruct" each time!!!
#import A

bfunc(a::Astruct) = dosomething(a)

#If you don't "export" your functions explicitly, there
#is no risk of name collisions, so you could even call
#your function "f" in all your packages or modules!!!!
f(a::Astruct) = dosomething(a)

#more code ...
export bfunc
end

```

C/src/C.jl:

```julia-auto
module C
using A

cfunc(a::Astruct) = dosomething(a)
#more code ...
end

```

D/src/D.jl:

```julia-auto
module D
using A
using B
using C

function dfunc()
   a = Astruct()
   bresult = bfunc(a)
   #Must explicitly qualify cfunc here. It was not exported:
   cresult = C.cfunc(a)
   return dosomethingwith_bandc_result(bresult, cresult)
end

#more code ...
end

```

M/src/M.jl:

```julia-auto
module M

#Here, you don't need to call "using" on A, B, or C
#because M never accesses those packages directly:
using D

#But you might want to break down package "M"
#across multiple files to make things more manageable:
include("logicalunit1.jl")
#...
include("logicalunitn.jl")

function run_the_fancy_calculation
   #setup constants
   result = D.dfunc()
   #more computations
   return result
end

#more code ...
end

```

### That’s it for the most part!

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 16, 2020, 5:35pm UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/8 "2020-07-16T17:35:19Z")

</div>

I’m starting to think the Julia Docs might be lacking a bit with regards to “packages”, `module`s, `include`, `using`, `import`, etc.

I still see similar questions arising on Discourse, every now and then.

I remember I had a hard time creating package “libraries” when I first started. The naming convention used in Julia somewhat clashes with what I was previously familiar with in other programming environments. That seems to have set me down the wrong path at first.

In fact, I still have trouble not calling a “library of packages” a “repository of packages” - because I’m afraid people might confuse these concepts with how packages are registered in “General” from Git “repositories” on Github. FYI: Julia’s package registration system was designed to only have one “package” per Git “repository”. It appears Git repositories were never designed to provide multiple Julia “packages”.

The worse part of all this is that you are in no way new to Julia, @oheil. I can see you have made significant contributions to our community for many years.

I have to wonder how many more people are having trouble, and how we can help them get more efficient.

I don’t like the idea we might be loosing out on potential adopters simply because they can’t figure out how to create/use their own ~~module~~ package “library”. I almost gave up on Julia myself when I struggled with the module/package concepts a while back (and still do sometimes).

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [July 16, 2020, 7:15pm UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/9 "2020-07-16T19:15:50Z")

</div>

> [@MA\_Laforge](#):
>
> The worse part of all this is that you are in no way new to Julia, @oheil. I can see you have made significant contributions to our community for many years.

This is true, but until now I didn’t had very complex projects. I am not so good in digesting documentations when I don’t have a real problem to work on and to use what I read.  
But your explanations are great and very helpfull! Thank you.  
Currently I am on the “Module-level Scope”-complexity (more because I want it so not because I really am) and when I think “module” I mean the Julia construct `module .... end`. I am not sure how to improve documentation, maybe it is better served as a separate document, like a blog or something. Actually your answers here (and @rdeits) are perhaps already sufficient? It was for me. Whenever these question come again I will link to this thread.

Actually Julia is quite easy to start with and first results are mostly impressive. But mastering Julia is quite a task. The learning curve is somehow exponential and I don’t see the plateau yet 🙂

> [@MA\_Laforge](#):
>
> I don’t like the idea we might be loosing out on potential adopters simply because they can’t figure out how to create/use their own ~~module~~ package “library”. I almost gave up on Julia myself when I struggled with the module/package concepts a while back (and still do sometimes).

I think people come and go, thats natural. I don’t like the idea, that programming languages/paradigms are somehow religions, where people have to be locked in. But wording like e.g. “evangelists” do imply that. The words form the thoughts. Programming languages are tools, they have to be used appropriately according to a problem. But enough of philosophy.

If some early adopters are leaving, than just because they didn’t ask here. We can’t force them to do that. But we can help those who ask. And thats what we do. Right on @MA_Laforge 👍

---

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [July 17, 2020, 12:55am UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/10 "2020-07-17T00:55:42Z")

</div>

> [@MA\_Laforge](#):
>
> FYI: Julia’s package registration system was designed to only have one “package” per Git “repository”. It appears Git repositories were never designed to provide multiple Julia “packages”.

Just FYI, we do support having multiple Julia packages in a single Git repository.

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 17, 2020, 1:44am UTC](https://discourse.julialang.org/t/dependencies-of-src-files-inside-a-package/43144/11 "2020-07-17T01:44:46Z")

</div>

> [@dilumaluthge](#):
>
> Just FYI, we do support having multiple Julia packages in a single Git repository.

Sorry, I should have said “originally”, I suppose. I’m still figuring out how to use those in an effective manner.

I hope that one day soon, there will be an fast, simple, elegant way to bundle multiple Julia packages into a simple Git repository, while simultaneously allowing developers to maintain minimum dependency requirements.

I have heard others suggesting it is already possible, but I personally seem to get snagged on little issues which I do not appear to communicate effectively.