Problems regarding test and module imports


#1

Hello,

I’m having some trouble structuring my project so that i can easily run tests.

Right now I have the following structure:

Project/
    src/
        MyStruct.jl
    test/
        MyStruct-test.jl
        runtests.jl
    Project.toml
    Manifest.Toml

Inside of MyStruct.jl I define a struct MyStruct and some methods for it. Then, on MyStruct-test.jl I have some functions related to testing the correct funcionality of the struct and finally on runtests.jl I have different @test macros using this functions.

The top of my runtests.jl file looks like this:

include("../src/MyStruct.jl")

using Test, MyStruct

include("MyStruct-Test.jl")

...

However when I try to run runtests.jl or >pkg activate . ; >pkg test I get an error like this:

ERROR: LoadError: LoadError: UndefVarError: MyStruct not defined

I think this is because inside of both runtests.jl and MyStruct-test.jl I use the struct defined in src/MyStruct.jl but it is not importing it correctly.

The only thing that seems to work is hardcoding the path via push!(LOAD_PATH, "/path/to/MyStruct.jl but this feels very hacky and if I work on this on a different computer I would have to modify it.

What is the proper way to structure the project so that using works and I can just use Pkg and have it run the tests?


#2

First of all, you cannot have a struct with the same name as a module, so you’ll need to change either your module name or your struct name to avoid conflicts.

Second, your runtests.jl should never need to include() any files from src. The easiest way to make sure a package you’re working on is available to Julia is to simply activate its Project.toml environment. Activating an environment just means running pkg> activate /path/to/Project. Or, even more simply, you can cd into Project before launching Julia and run julia --project instead of julia. That should ensure that the MyStruct package is available without needing to include() anything.


#3

Hello,

The name of the File is actually MyStructs.jl and the name of the struct is MyStruct, I made a mistake in writing the example.

Secondly, MyStruct is not a package, it is just a source file in which I have defined a struct and a number of functions for that struct. So when I remove the include calls from runtests.jl it still tells me that MyStruct is not defined.

Do you just have to make a package for every module you want to define?

Note: If i add a using MyStructs to the runtests.jl file, I get an error saying

ERROR: LoadError: ArgumentError: Package ZPField not found in current path
...

#4

Why not use a relative path?

push!(LOAD_PATH, realpath("../src"))

This is no more “hacky” than include("../src/MyStruct.jl")


#5

Without a package, a package manager is not of much use :wink:.
Have you read: https://julialang.github.io/Pkg.jl/v1/creating-packages/?

For package Foo, a barebones runtests.jl would be:

module FooTests
using Foo
end

#6

The top level is indeed a package, it just felt stupid to have to define a package for every file you want to split your package into.

I finally was able to solve it for my case. For any other users having problems with this, this is what I ended up doing:

  1. Go to some directory and run pkg> generate Example

Say you want to define a module Module1 in a file called MyModule.jl. The contents of MyModule.jl are the following:

module MyModule

export MyStruct, myfun

struct MyStruct
    a::String
end


function myfun(x::MyStruct)
    print("TEST")
    return true
end

end # module

Now, what you want to do in Example.jl to be able to use this file is

module Example

include("MyModule.jl")
using .MyModule
export MyStruct, g
end # module

Note the . before MyModule. This is because include() basically copy-pastes one file into the calling one, so the modules you define there end up being submodules of Example.

Then, from Example I export MyStruct and g so I can do the following in runtests.jl

using Test, Example


a = MyStruct("ASD")

@test g(a)

And now I can just run pkg> activate . ; pkg>(Example) test and it will run the tests correctly.

There is probably a more elegant way to do this but this works for now. Still, I feel like there should be a way to split modules into files and import them without having to use include() to basically copy and paste source files.


#7

IIUC, you problem regards the structure of a large “Application” (as defined in the Pkg glossary), into several (more-or-less) independent modules.

I too used to find it odd that only one top-level module could be defined per package (and only this module could be imported without having to mess with LOAD_PATH). But eventually I realized that most of the time you can work with sub-modules in the same way as you would work with top-level modules.

Here is what I do now:

sh> tree MyApp
MyApp
├── Manifest.toml
├── Project.toml
├── src
│   ├── A.jl
│   └── MyApp.jl
└── test
    └── runtests.jl

File src/MyApp.jl:

module MyApp

include("A.jl")

end # module

File src/A.jl:

module A
export greet

greet() = "Hello from module A!"

end # module

File test/runtests.jl:

using Test
using MyApp.A  # Note that submodule A can be imported without problems

@testset "A" begin
    @test greet() == "Hello from module A!"
end

All tests work; the only thing is that you have to use using MyApp.A instead of just using A as you would have liked (IIUC).

(MyApp) pkg> test
   Testing MyApp
 Resolving package versions...
Test Summary: | Pass  Total
A             |    1      1
   Testing MyApp tests passed 


Note that the modules hierarchy has almost nothing to do with files: any of the files above could be split into several files wich include each other. And conversely both the module MyApp and the submodule MyApp.A could have been defined in the same MyApp.jl file by simply replacing include("A.jl") with the contents of A.jl.

AFAIK, the only constraints imposed on the file hierarchy are:

  • module MyApp has to be defined in file src/MyApp.jl (I think just MyApp.jl would be OK too)
  • tests have to be defined in test/runtests.jl (at least if you want Pkg to be able to run them)