Multiple modules in single package

I found that a rather short function I use in the unit tests of a package would be useful somewhere else. If I don’t want to make a separate package for it, is it possible to put it in a separate module in the same package?

Simply putting it in src/MiniModule.jl in the same package does not make using MiniModule find it it.

1 Like

It would need to be a submodule, I think. To get it using MyPkg.MiniModule ain’t so bad.

1 Like

That’s unfortunate: MiniModule is not really related to MyPkg, other than being useful in its unit tests.

It seems that, there is an implicit 1:1 mapping between modules and packages for many things.

How would you want it to be accessible? As soon as you think about that, it sounds like a julia package.

Packages con be small and minimal. What is the exact issue you are worried about?

Making a package for <10 LOC. But I see your point.

You can also do push!(LOAD_PATH, @__DIR__) in src/Module.jl to make MiniModule findable. Then users can do using Module; using MiniModule

At least if registered it does put additional burden on METADATA maintainers (and on Tamas too).

There was some discussion on this topic here: https://github.com/JuliaLang/Juleps/issues/8

I’m really hoping to see this feature before 1.0; I don’t think having to use a separate git repository for every chunk of code that forms a standalone module is really scalable for huge projects.

4 Likes

Sorry for necroing this thread, but I assume that those who replied may be interested.

I think found a Pkg3-compatible solution. Let’s call the package ThePackage and the auxiliary module AuxModule.

  1. Have the auxiliary module in a subdirectory of the main project, eg assets/AuxModule/src/AuxModule.jl.
  2. pkg> activate the main project.
  3. pkg> dev ./assets/AuxModule. This will generate an UUID for it, even if there is no Project.toml file in assets/AuxModule.

Then it appears that when the package is activated again, using AuxModule etc will work fine. One just needs to make sure that the project was activated.

This solves my problem, but I am unsure if it is something I can count on to keep working, or an abuse of the Pkg3 setup. Comments welcome.

2 Likes

I think this is how it’s intended to work, judging by this comment: Pkg3: package namespaces · Issue #8 · JuliaLang/Juleps · GitHub

2 Likes

Note that my solution is without making AuxModule a submodule.

1 Like

Necroing the thread again. Does this solution still work? I’m getting

ERROR: expected a `uuid` entry in project file at...

and, even if I ignore it, I have no success.

Did a “generate”, got a uuid, and all seems to be well. The Manifest.toml says exactly the right thing. However, if I put the Manifest.toml on github, will that break CI?

It seems to be trickier than that. When I try to get the AuxModule to load from a notebook, IJulia can’t find it. Nor does it work in the REPL after a restart of Julia.

The item in @Per’s post
https://github.com/JuliaLang/Juleps/issues/8#issuecomment-400829383
seems to say that I need to load it as a separate package from the same repo. I’d prefer not to do that because it will be confusing for my users.

Related to this, I think the FastAI.jl package contains multiple modules (not submodules) in it:

  • FastAI
  • FastMakie
  • FastTabular
  • FastText
  • FastTimeSeries
  • FastVision

Only the FastAI module is defined in the top src/ directory of the package; the remaining modules are defined in their own directories, and they all have their own Project.toml and src/ directories.

I’m not sure how this is done; I wonder if @holylorenzo could explain how.

[Update] It seems misleading to say that the FastAI.jl package contains multiple modules. It seems that all these modules (FastVision, FastTabular, etc) are all packages on their own, because after I install FastAI and try using FastVision, the package manager asks me to install FastVision (even though FastAI is already installed). So, it seems more correct to say that the single GitHub repository at GitHub - FluxML/FastAI.jl: Repository of best practices for deep learning in Julia, inspired by fastai contains multiple packages.

Another, probably more familiar case where multiple packages are included in a single repository: Makie.jl. It contains several packages like Makie.jl, CairoMakie.jl, GLMakie.jl.

So, putting multiple packages in a single Git repository seems like a widely used practice at this point.

1 Like