How to add a Directory Package to an existing Project?

New to Julia
1

I am trying to using Julia 1.6.1 for application development.
I like to make a separation between “library code” and the main program.

In my mind, a directory layout close to the following might work:

  • MyApp ← the project created with generate MyApp
    • Manifest.toml
    • Project.toml
    • src/
      • MyApp.jl ← module MyApp; include(“included.jl”);
      • MyApp ← runnable from bash, using MyApp; MyApp.main()
      • included.jl
    • Lib/
      • src/
        • Lib.jl ← No dependency on MyApp.jl
    • test/
      • runtests.jl
      • Manifest.toml
      • Project.toml
    • notebooks/

When I add using Lib to MyApp.jl, I get Package MyApp does not have Lib in its dependencies.

Doing add Lib results in:

(MyApp) pkg> add Lib
[ Info: Resolving package identifier `Lib` as a directory at `~/Projects/MyApp/Lib`.
ERROR: Did not find a git repository at `Lib`

Typically, edits are being made to MyApp.jl while Lib.jl changes slowly.
The package Lib would only ever be used by MyApp.jl; it makes sense for Lib.jl and MyApp.jl to live in the same git repo.

Is there a canonical way to add a Directory Package to a Project?

2

A bit more digging, I have found that I can use “dev” as mentioned in the reply linked here:
https://github.com/JuliaLang/Pkg.jl/issues/1251#issuecomment-647115302

Cloning ma-laforge’s PkgTestRoot.jl repo, I confirmed that I could allow sibling dependencies in subpkgs using “dev” instead of “add” –
https://github.com/ma-laforge/PkgTestRoot.jl

I would still like to know if this is a common approach.

3

It is a reasonable solution for certain cases, I use it sometimes, especially if I want to distribute unregistered packages within a repo.

1 Like
4

I’m not sure I understand why you don’t want the MyApp code and the MyLib code to live in the same project/package? (For example, the code defining MyLib could be put into some subdirectory of src, and maybe put in a separate submodule if you want to guarantee it doesn’t depend on outer MyApp code)

5

Thanks ffevotte,

I’m not sure I understand why you don’t want the MyApp code and the MyLib code to live in the same project/package?

What I have done so far is to use include("...") to spread my code among many files. Perhaps because of my experience with C++ templates, I trend to view large include graphs as something to be wary of, particularly as the application grows larger. There seems to have been some lively discussion about using relative imports – but some proposals amount to something like include + using – which feels like it makes pre-compilation an impossibility. Is there a way to use submodules without using include or guarantee pre-compilation?

My understanding is that pre-compilation occurs at the package level, hence the desire to factor out stable code into packages (using using Lib not include) keeping all within the same git repo for MyApp.

From what I have read, one package per project per repo is a valid way to factorize development of larger applications. But I haven’t warmed to this as it seems like overkill and increases coordination cost between components; for packages used in one code base, git branches suffice for my needs. There may be a way forward by manipulating LOAD_PATH outside of the Pkg REPL; if so I would like to hear if/how others do it in v1.6+.

6

I’m not following the specifics of your problem exactly, but you should use dev instead of LOAD_PATH for sure.

3 Likes
7

Yes, if Lib has long precompilation times, it is a good reason to factor it out in a separate package, in order to ensure that it will be pre-compiled independently. (To be clear, if Lib was a submodule of MyApp, it would still get precompiled, but the precompilation would happen again each time MyApp changes).


Please don’t manipulate LOAD_PATH yourself, and use Pkg.dev instead (giving it a relative path to your library).

Note that if you want to get the full benefits of having Lib be a fully fledged package, it should have its own Project.toml, that list its own dependencies.

2 Likes