Understanding package directories and entry point files

New to Julia
1

Dear Community,

I am currently restructuring some existing code, which is set up to be a package. When moving around some files into a (for me) logic new structure, I got entangled with problems related to Package directories. I think this question belongs in “New to Julia” because I am most interested in understanding why my approach does not work and good practices for setting up my code structure.

A minimal working example looks like this: I considered it sensible to implement the following package structure:

  • MyPkg
    • src
      • pkg
        • ‘MyPkg.jl’
        • …
      • examples
      • …
    • Project.toml

Hence, I wanted to create a subfolder pkg, which holds the module file with ‘MyPkg.jl’ to separate it from other code that is not necessarily invariant to user inputs.

The Project.toml file looks like this:

name = "MyPkg"
uuid = "d6f8d3e6-0b6f-11e9-4e3f-3b5f3d3f3b3e"
version = "0.1.0"

[deps]
Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"

I included the dependency of Printf for no particular reason. The MyPkg.jl file just contains a very simple function hello().

module MyPkg

export hello

using Printf

function hello()
    @printf "Hello, world! \n"
end

end # module

When I now try to use the package, by first running ] activate . followed by ] instantiate, I get the error ERROR: expected the file `src/MyPkg.jl` to exist for package `MyPkg` at `SomeDirectory\MyPkg`.

If I delete the uuid and the version in the Project.toml file, I do not have any issue with the instantiate.

After reading a bit into The Manual regarding Code loading I understand that this should not work.

First, I am curious to understand why moving the ‘MyPkg.jl’ file with the module into a subfolder does not work and potentially is not supposed to work. Second, and conditional on “maybe it can/should work”, is there a way to get around this, or is this simply not desirable?

Thank you very much for helping me understand.

Files of the MWE are attached.

Project.toml (135 Bytes)
MyPkg.jl (107 Bytes)

2

Why it does not work is simply a matter of not matching the code loading conventions. Those could potentially have been designed to be more flexible but you always have various tradeoffs from that.

For what it’s worth, the usual layout is

  • MyPkg
    • src
      • ‘MyPkg.jl’
      • …
    • examples
    • …
    • Project.toml

but you could also consider

  • MyPkg
    • pkg
      • src
        • ‘MyPkg.jl’
        • …
      • Project.toml
    • examples
    • …
3

Thank you for the quick reply @GunnarFarneback.

I am especially curious about the “Why” behind the code loading conventions. Can you elaborate on the tradeoffs that were considered when setting the conventions the way they are?

4

No, I wasn’t involved in setting the conventions. But the most obvious tradeoffs if you don’t have a fixed place to look for the code is that you either need some way to configure where it is, which adds to both documentation and code complexity, or the code loading needs to search for the code, which adds to code complexity and time for the search.

5

Just be absolutely clear: These aren’t just conventions, but (up to a point) requirements for how the files in a Julia packages have to be organized and named.

I think this (the pkg subfolder) can work, but makes registration unnecessarily complicated. I very strongly suggest sticking to the standard layout where src is a direct subfolder of MyPkg. The examples folder should be either directly under MyPkg, or it could also be in docs or docs/src, depending on whether these examples are stand-alone or a part of a (Documenter) documentation.

As for why the decision was made for the particular layout required for a Julia package: No idea, since I wasn’t part of that discussion either. It seems sensible, though.

6

Thank you for your answers.

To summarize: First, no workaround allows the use of an arbitrary file organization, but the structures you mentioned are requirements (and customization is not desired). Second, another file organization potentially can work (the pkg subfolder), however in general it is better to stick to the standard structure.

1 Like
7

It does work (one example is GitHub - GunnarFarneback/GoGameGraphs), and the complications amount to having to specify a subdir.

The more interesting difference between the layouts is that in the standard layout, examples etc. are included when you Pkg.add the package and they are distributed by the package servers, whereas in the alternative layout they are not. Often you want to include examples but you also may prefer not to, e.g. if they are very large.

1 Like