How to Import dependency from one file to another inside of a package?

New to Julia
, ,
1

Hi, I am writing a package in Julia. But can not figure it out how to import some particular function from one file to another. Here is the latest version of the package: https://github.com/DataPsycho/ProjectFlow.jl

Lets say,
I have following files in src directory utils.jl, manager.jl, ProjectFlow.jl, profiler.jl. The utils.jl, manager.jl and profilers.jl is included in the ProjectFlow.jl using include() but,

Case 1: I have a general function make_dir in utils.jl file and I want to import that function inside of the manager.jl file and I have a function called load_profiles in profiler.jl and I want to import it manager.jl what is the best way to do it? In python I could do relative import for example in manager.py file i can write from .profiler import load_profiles

Case 2: If I want to import the whole file but not just a function how to do that in julia way? For example I want to import * from utils.jl in to manager.jl How I can do that?

2

Generally,

using SomeModule: some_function

or with import (if you want to define new methods). See

https://docs.julialang.org/en/v1/manual/modules/

Conceptually, the right level of granularity is a module or a package. Even with the above, the whole of SomeModule will be loaded, it’s just that only some_function will be in your namespace.

3

If I understood you correctly, just include the files in correct order here: https://github.com/DataPsycho/ProjectFlow.jl/blob/5e8609e9fd2babb72a85985541989927556b45ab/src/ProjectFlow.jl#L6, because everything is inside the same Module.

Edit I mean change the order that way it works I.e. utils.jl is included first instead of last. Include is equivalent of copy-pasting the code from the file to right there.

4

Ok. Then if I want to use some function from utils.jl into lets say manager.jl then is that a right way to do that: https://github.com/DataPsycho/ProjectFlow.jl/blob/5e8609e9fd2babb72a85985541989927556b45ab/src/manager.jl#L1

Or there is someother way if I want to import some function from utils.jl or whole utils.jl in to manager.jl

5

I had the same questions when I developed my own package.

The most important things to retain are:

  • the logical structure of your program can be decoupled from the actual division in files (i.e. no reason to have necessarily one module == one file like in Python)
  • you can have one top module and several submodules if you wish, although this is in general discouraged
  • there is no issues in calling import x or using x several times in the code, BUT include("x.jl") should be called only once. include is like copying everything from the included file in the point where it has been included.
  • you can use the reexport package to automatically reexport the functions/objects you imported from an other module

In my case I have:

BetaMl.jl file:

module BetaML
include("Utils.jl")
using .Utils
include("Nn.jl")
using .Nn
include("Perceptron.jl")
using .Perceptron
include("Clustering.jl")
using .Clustering
end # module

Nn.jl file: (similar for the other files)

module Nn
using [list of packages I need for Nn functionality]
@reexport using ..Utils # note the two dots in front to go one level up, as these two modules are siblings
export [list of Nn module provided functions]
[...code...]
include("Nn_default_layers.jl")
[...code...]
include("Nn_default_optalgs.jl")
[...code...]
end # end module

In the “Nn_default_layers.jl” and “Nn_default_optalgs.jl” file I then have:

using [list of packages I need in this "section" of the Nn module ]
export  [list of functions/objs provided in this "section" of the Nn module ]
[...code...]

An user of the package can then access its functionality either by using the specific submodule of interest and then directly the provided functionality or by using the root module BetaML and then prefix with BetaML. each object/function she/he want to use, e.g.:

using BetaML.Nn
myLayer = DenseLayer(2,3)

or, equivalently,

using BetaML
res = BetaML.kernelPerceptron([1.1 2.1; 5.3 4.2; 1.8 1.7], [-1,1,-1])
1 Like
6

Thanks a lot. That helps to understand how to architect packages in Julia.

7

Can you say more about why submodules are discouraged? It seems like your example uses them?

I’ve heard advice like this before but also have seen it authoritatively contradicted.

8

I think we enter a bit of subjectivity here. I believe what is discouraged is the a priori decision to organise a package in submodules, rather than after having reasoned that that is its best organisation, for example among the criteria of @Tamas_Papp in the thread you cited…
It is a matter of fact that many (most?) large and heavily used packages are not organised in submodules, so, as minimum, you have less community experience (hence help) if you run into troubles…

1 Like
9

The key question when using any feature is: Why? What is purpose of using modules? Modules provide separate namespaces for global names in Julia. So the question is: do you need separate name spaces in your own code? If the answer is yes, then absolutely use modules since that’s what the language feature is for. If the answer is no, then why use modules?

From my own experience, I tried using submodules for code organization in Pkg, and it ended up being kind of a mess that I now regret. It didn’t really help with clarifying the code organization in any way and it makes refactoring code in Pkg much more annoying than it would be if we didn’t use any submodules. If anything, it actively makes organizing the code harder since you can’t always put definitions where it makes logical sense to put them because you may have to define them in a different module and therefore in a different file.

While I think there are good use cases for submodules in a Julia code base, it’s probably less necessary than some other languages.

8 Likes