When to define a `module`

When developing a package (call it MyPackage) one usually has a “main” function src/MyPackage.jl. This file is a module with a bunch of using, export and include statements.

The export statements establish the API of the package, i.e. the set of functions, variables, types, etc. which will be made available through the call using MyPackage. The include statements typically import .jl files which contain the actual code of our package.

For instance, if we are developing a package for time series analysis, the included files might be time_series.jl (defining a struct and basic functionalities), fourier.jl (containing FFT and power spectrum estimation functions), wavelets.jl implementing wavelet analysis, etc.

Programming principles encourage us to make the files of our package as independent as possible from one another. For instance, fourier.jl might operate entirely on native vectors, without needing to import anything from time_series.jl.

Should such independent portions of code (fourier.jl in our example) be modules as well? And what about less isolated, or less independent, files? In short, when should a set of functionalities be wrapped around a module, and when not?

I don´t think there are clear rules for that, and taste and good sense are important.

Nevertheless, recently a developer (not me) of ChunkSplitters.jl reorganized the code separating all the internals of the package into a module: ChunkSplitters.jl/src at main · JuliaFolds2/ChunkSplitters.jl · GitHub

That way, it is hard to access the internals inadvertently, and what is API and what’s not is very transparent. I found that organization neat. One can extend that idea in having the API defined at the top level module of the package, and split all the remaining code into submodules that import the API from the main module and then define the internals providing the functionalities.

4 Likes

VSCode does not support module based development well. So I am hardly using modules any more, only packages that are split up with includes. Not perfect, but kind of OK.

1 Like

What do you mean specifically?

1 Like

This bug, which is open since 7 years and I am not very optimistic that it will ever get fixed unless their is funding to hire another developer for the julia-vscode plugin.

1 Like

Hi,

I am not sure if this is entirely what you mean, but I seem to have VS Code working for me, while developing with modules.

If you like at my source repo here:

I have a module in src/SPHCellList.jl:

module SPHCellList

export ConstructStencil, ExtractCells!, UpdateNeighbors!, NeighborLoop!, ComputeInteractions!, RunSimulation

using Parameters, FastPow, StaticArrays, Base.Threads, ChunkSplitters
import LinearAlgebra: dot

using ..SimulationEquations
using ..SimulationGeometry
using ..AuxillaryFunctions
using ..SimulationMetaDataConfiguration
using ..SimulationConstantsConfiguration
using ..SimulationLoggerConfiguration
using ..PreProcess
using ..ProduceHDFVTK
using ..TimeStepping
using ..OpenExternalPrograms

Which refers to other modules (and not their filepaths directly). If I hover about it in VS Code:

And I can to jump to using crtl+left click.

Of course I have a main file named after the package, SPHExample.jl where I must include all files etc.

It took me quite some time to get this work flow up and running from Julia, but I like it that it is module based and I can quite easily import/export modules between different files, without ever referring to direct file path (only once in main)

Hope it helps, if I understood you correctly

Kind regards

3 Likes

This only affects you if you rely on LOAD_PATH, right? I’m not sure what reason there is to rely on a custom load path. Private packages, but no time for a local registry?

Well, I am used to write code without using packages. If code shall not be reused by other projects, then why write a package?

If you define a file that contains a module in the src folder the easiest way to make use of this module is to add the src folder to the LOAD_PATH.

I do not understand the using ..ModulName syntax. What does it do?

  • Dependency management
  • Tooling support (this topic)

I don’t think that’s any easier than just dev-ing a package. It’s easy to create a package (interactively!) by:

using PkgTemplates: generate
generate()
  • using .N makes the names of the module N from this namespace available in this namespace
  • using ..N makes the names of the module N from the parent namespace available in this namespace
  • and so on

I never dev a package. If I do that the source code ends up at a location which is below the .julia folder, and I delete that folder now and then.

You can change that by modifying ENV["JULIA_PKG_DEVDIR"] to a location which you prefer IIRC

You’re thinking about dev RegisteredPackage, I think, but we’re not talking about registered packages anyway. When I said dev I meant dev /path/to/your/source.

FTR I, too, don’t keep the development sources of any packages under .julia, but I dev many packages.

1 Like

So what is that good for?

It makes the package available to Julia and tooling, while tracking all changes to the source files. So you wouldn’t have to modify the load path.

Nice. But this does not work for modules, only for packages. And modules are the main topic of this thread.

So I repeat my statement that modules are not well supported by the Julia tooling. Which means I cannot do modular programming in the way I would like to do it, to create code units that are equal to a file and understandable by reading just that one file.

A package is a module. Just a module with a Project.toml. And that, adding a Project.toml/making a package, is how you make modules “well supported”. I’m literally giving you the solution to your problems, while your dismissal seems quite arbitrary.

It is NOT a solution of my problem. A package can and should consist of multiple modules. It is a different level of granularity. And to say, because packages are well supported (well, even that is debatable, goto definition often does not work in examples or tests), to say that because of the tooling support for packages modules are well supported is just plain wrong.

Yes, and this is well supported. As already noted above by @Ahmed_Salih.

This contradicts my experience.