Run tests on src file change: Best practice

New to Julia
1

What is the best practice to have Julia run unit-tests automatically when the src code changes?

A. Using 3rd party packages.
B. Using only Base or Stdlib packages.

2

I’ll endeavor to revise the selected solution to point to any better alternatives proposed…

Current best practice appears to be set-out in this SO answer by anowacki:

  1. Launch a separate shell and Julia REPL.
  2. Activate your package dev or test environment.
$ cd ~/src/your-package
$ julia --project=.

A. Using 3-rd-party Packages (Blocked by Watcher issue #2)
Add test related packages to Project.toml

[extras]
Test = "<uuid-here>"
Watcher = "<uuid-here>"

[targets]
test = ["Test", "Watcher"]
  1. Cut-and-paste the following:
$ julia -e "using Watcher"

B. Using only Base or Stdlib

NOTE:
The following does not trigger test runs when you create or delete a file. Only when you add/remove content to a file. To delete a file: first remove all content (a test run will be triggered), then delete the empty file.

  1. Cut-and-paste the following:
$ julia --project=.
julia> import Pkg; import FileWatching: watch_file

julia> @async while true
           event = watch_file("src")
           if event.changed
               try
                   Pkg.pkg"test"
               catch err
                   @warn("Error during testing:\n$err")
               end
           end
       end
5 Likes
3

If you want to catch file creation or deletion, you probably need to do a watch_folder on the parent folder, since those types of changes modify the folder data structure, not the file data structure.

An alternative is to a use a tool like entr to watch the specified set of files you care about and then trigger your tests based on that. I haven’t used it before, so I’m not sure of its limitations, if any.

1 Like
4

Thanks for taking the time to make the suggestion.

I had tied that, unfortunately that was throwing this error when submitted in the REPL:

ERROR: type Pair has no field changed

Did watch_folder work for you?

With time I might dig in but right now a partial solution is as above.

5

Jive v0.1.5 supports watch folders.

~/.julia/dev/Jive/test $ cat runtests.jl
using Jive # runtests
runtests(@__DIR__)

~/.julia/dev/Jive/test $ julia --color=yes -q -i runtests.jl jive/s
1/3 jive/skip/skip-calls.jl
    Pass 2  (0.29 seconds)
2/3 jive/skip/skip-functions.jl
    Pass 4  (0.02 seconds)
3/3 jive/skip/skip.jl
    Pass 4  (0.01 seconds)
✅  All 10 tests have been completed.  (0.61 seconds)
julia> watch(@__DIR__, sources=[normpath(@__DIR__,"..","src")]) do path
           @info :changed path
           runtests(@__DIR__)
       end
watching folders ...
  - jive/skip
  - ../src

when saving any files in the watching folders, it automatically run tests.

julia> ┌ Info: changed
└   path = "jive/skip/skip.jl"
1/3 jive/skip/skip-calls.jl
    Pass 2  (0.00 seconds)
2/3 jive/skip/skip-functions.jl
    Pass 4  (0.01 seconds)
3/3 jive/skip/skip.jl
    Pass 4  (0.01 seconds)
✅  All 10 tests have been completed.  (0.15 seconds)

to stop watching

julia> Jive.stop(watch)
stopped watching folders.
6 Likes
6

Thanks for Jive.jl: it has really changed the way I develop Julia code!

I’ve had a bit of trouble making Jive.watch work the way I want, though. In particular, when strictly following the documentation (or your post above), tests are correctly re-run for every source file change, but only changes to the test cases are correctly accounted for. As opposed to changes in the package source code itself, which don’t seem to propagate to the environment in which tests are re-run.

I’ve eventually made things work by adding Revise into the mix:

using Revise
using Jive

watch($dir; sources=[normpath(joinpath($dir, "..", "src"))]) do path
    @info "File changed" path
    revise()
    include("runtests.jl") # uses Jive.runtests() to run tests
end

Did I miss something? Or is this one of the ways Jive is meant to be used? In the latter case, I’d be happy to make a PR to document this if you think it useful.

3 Likes
7

you’re right. I’ll update for it in Jive v0.1.6
thank you very much :slight_smile:

8

Is there any update on Jive or any other tool that helps one to implement a TDD workflow to run tests on save? I have looked at Jive - being totally new to Julia, it looks a bit overwhelming…

9

I think nowadays Revise.entr provides enough features to re-run tests when sources change. Something like this could be a good starting point:

using Revise
using YourProject

# Re-run tests when something changes in the `test` directory
# or the sources of `YourProject`
entr(["test"], [YourProject]) do
    @info "Running tests"
    try
        include("test/runtests.jl") # or Pkg.test()
    catch e
        showerror(stderr, e)
        print("\n")
    end
end
3 Likes
10

I use the following script: Script for watching and running tests or certain files (using Revise.jl to avoid full re-compilation). Useful if you're taking a test-driven development. Requries the following packages to installed in the global environment: - `ArgParse` - `Revise` - `TestEnv` · GitHub

Found it to work extremely well in practice

1 Like