Errors in creating a small, multi-file project with one module

New to Julia
,
1

Hello!

I am a new Julia user, using v1.10, and am struggling a bit to fully understand how to get multiple code files to work together. Specifically, I am finding that editing files can throw errors, and my using statements do not always import necessary code. The documentation seems pretty confusing to me, and I haven’t found julia examples that use multiple files like this, so I thought I would ask for help here.

I have a filesystem with three files. The system looks like the following.

Module1.jl
ObjectFolder
⤷Object1.jl
RunnerFolder
⤷Runner1.jl

Let’s go through each of the files now.
Object1.jl is a file that simply contains a struct definition. Later, I might add some functions that use this, but for now I simply care about defining a new data type:

struct Object1
    variable1::Vector{Int64}
end

Module1.jl is a file containing a simple module definition. Currently, all it does is include Object1.jl:

module Module1
    include("ObjectFolder/Object1.jl");
end

Finally, Runner1.jl is intended on using the code in Module1 inside a coherent script. The concept is that all my helper code is in the module, and then my Runner actually uses the code:

using Module1

Alice = [1,2,3];
print(Alice);
Bob = Object1(Alice);
print(Bob)

For the purposes of this post, I have changed the name of the files, but the contents are correct. I have added the parent folder containing Module1.jl and every subfolder to the LOAD_PATH so that it is recognized by the “using” statement correctly.

When I run this, I get the following error:

[1, 2, 3]ERROR: UndefVarError: `Object1` not defined
Stacktrace:
 [1] top-level scope
   @ FILEPATH\Runner1.jl:4

So, I know that the “using” statement does not throw an error, because Alice is successfully printed. However, the “Module1” module is not being imported by the using statement, as the error shows. That, or the module is not actually functioning as I would expect, and is not importing the struct code from Object1.jl.

I have a few questions, for anyone who would be willing to help me out here:

  1. Why is the Object1.jl code not being imported by the module? Am I not understanding how modules are supposed to function?
  2. How do I get two code files to work together like this? I am explicitly trying to avoid defining all of my code in one file, because I will soon have to create a project that has too much code to keep track of in one place.
  3. Are there any “Best Practices” for organizing code in multiple files like this? How would you approach this problem?
2

I think you are just missing the export Object in your module, do make it accessible outside when using it.

2 Likes
3

I haven’t tried reproducing the same file hierarchy to test everything, but most probably it does not work as it is because Object1 is not exported by Module1. You have two possibilities:

  • either add export Object1 somewhere in Module1 (either inside Module1.jl or in ObjectFolder/Object1.jl. This sets things up so that using Module1 brings Object1 into scope in the client code.
  • second possibility: in the client code (Runner.jl), refer to Object1 with a fully qualified name: Module1.Object1.

I’d say you seem to have grasped how include works, and this is the correct tool to use here. If you have some code that could be put in the same file, but is too large, simply put part of it in another file, and include() that file in its place.

I’d say you have it almost right, but the only missing part to follow “best practices” would be to put your “library code” inside a package. This is a bit stronger than a module, in that every package defines a top-level module, whereas not every module corresponds to a package. A package is an “atomic” piece of code in Julia, destined to be re-used elsewhere (in your case, in the Runner code).

The whole project could look like this:

MyProject
+- src
|  +- MyProject.jl # defines a module, can include other code
|  +- object1.jl
+- Project.toml    # contains meta-data about the project
+- runner.jl       # code living outside the module, but using it

You can initially create this directory structure (including the Project.toml) by running

julia> import Pkg
julia> Pkg.generate("MyProject")

Now you put your library code in src/MyProject.jl. If it gets too large you can include() from there other source files in the src subfolder.

Now the good part is this: instead of fiddling with LOAD_PATH in order to be able to use your code, all you need is to activate the project. Assuming you run julia from the directory where your project is defined:

  • either use something like import Pkg; Pkg.activate(".") in an interactive session, or
  • use an IDE such as VScode that does this for you automatically, or
  • pass the --project command line option when you start julia. For example, to run your script non-interactively: julia --project=. runner.jl.
8 Likes
4

I suggest using PkgTemplates.generate() instead. It’s more comprehensive, customizable and interactive.

IMO it’d be better to put the runner.jl script outside the package directory.

My rules are:

  1. each source file in a package’s src/ directory must start with module FileName
  2. only use include in the top-level module
2 Likes
5

On the whole, I prefer the suggestions of @ffevotte

For a beginner in a case such as this, Pkg.generate is simpler and perfectly adequate.

@ffevotte s suggestion is simpler and I use it myself sometimes, especially if it is an informal package that is purely for personal purposes and still under development.

In my opinion, your rule no. 1 is too restrictive, I feel that @Spydercrawler s way of using include is sensible and is what I use as well.

5 Likes
6

IMO there’s no reason to use Pkg.generate, and I would especially recommend PkgTemplates.jl to a beginner.

1 Like
7

Well, our opinions differ

5 Likes
8

I would like to let you know that this is an incredible response, and completely answers my questions.

Most of the time, I am quite scared to go onto any type of programming forum, because people can sometimes be quite snarky, especially towards newcomers. This answer completely blew all of my expectations out of the water, so thank you very much.

5 Likes
9

I have writtem a tutorial on modules, packages and, eventually, how to document, test and register a package:

3 Likes
10

Thanks! I think you’ll find that there are lots of “incredible responses” here. In any case, I’ve personally never found a community as talented as the people you’ll see here if you stick around, yet welcoming at the same time. Welcome!


Regarding some of the other comments, I could perhaps have made a better job of separating what-you-should-do-because-that’s-pretty-much-the-one-recommended-way from the objectives-everyone-agrees-you-should-strive-for-but-there-are-multiple-ways-to-get-there.

I’d say everybody will agree that it’s a good idea to separate “library code” that defines/declares functions/types/etc from “scripts” that actually run/execute code. You had that one completely right!

  • I think there’s a strong consensus that library code should go into packages. There are however several tools allowing one to build a package; which one to use will depend on personal preferences, the size of the project, the type of QA you’re aiming for, etc.

    • I’ve mentioned Pkg.generate above: it’s really easy to use (which is why I mentioned it, because I could explain you how to use it in one line), but will only create a bare-bones project.
    • PkgTemplates.jl is probably the other end of the spectrum : depending on what you ask, it will set things up so that you can document, test your project with various CI systems and more. Highly customizable, which also implies you have to know what you want
    • inbetween those two, you’ll find tools like PkgSkeleton, which provides a more opinionated, no-questions-asked way to create fully-fledged projects that integrate well with Github tools

  • I also think everybody will agree that “scripts” should live outside the packages sources (i.e. outside the src directory). Exactly where will depend on a broader context and/or personal taste, though. Scripts can actually live pretty much everywhere; you just have to remember to activate the right project environment when you run them.

6 Likes
11

Another standard piece of advice for developing packages is to use Revise.jl. I think most people add it to their base project (the one you get when you run julia without a --project argument), so you don’t have to add it to every project that you want to use it in. Then, just run using Revise before loading your package, and any changes you make to the package after that will be automatically loaded into your running session.

I believe VS Code with the julia extension does this automatically, so a superseding piece of advice might just be to use VS Code.

1 Like