For languages without classes tests are in my opinion a great way to document functions. For example, when I have the function
function str2int(letter::String)
letter = uppercase(letter)
@assert occursin(r"[A-Z]+", letter)
ch = letter[1]
i = Int(ch) - 64
end
then the goal is immediately clear when I read
@test str2int("C") == 3
Unfortunately, all languages I know of tend to put these test in a file far away from the definition of the actual function. When we define these tests right below these functions then any time we load the module all tests will be run, which is not useful in practice either. To solve this I wrote a little macro (thanks to the Julia language for allowing that).
module DTest # `Delayed test`, or `define test`, whatever you like
all_dtests = Expr[]
export all_dtests
macro dtests(ex)
push!(all_dtests, ex)
# Returning last evaluated expression to get "Test Passed".
esc(:(dtest() = eval.(all_dtests)[end]))
end
export @dtests
end # module
Then to use it I write
using Test
include("dtest.jl")
using .DTest
function str2int(letter::String)
letter = uppercase(letter)
@assert occursin(r"[A-Z]+", letter)
ch = letter[1]
i = Int(ch) - 64
end
export str2int
@dtests begin
@test str2int("C") == 3
end
Now I can still load the module Helpers and its functions without running the tests. To run the tests I call Helpers.dtest().
So far the small macro has been very convenient for me. I am wondering whether it is something I should make a Julia package, or even whether it is a nice addition to the standard library?
BTW, I think that because of the way precompilation works, @dtest wouldnât work in other packages, because the push! wouldnât happen inside of __init__
Without thinking about the technical implications or problems with implementation of such a package, I like the idea and reading your post immediately convinced me that this is a good idea.
Not so much for the users of code, but for the subsequent developers who try to add functionality or fixing bugs.
Tamas_Papp Tero_Frondelius
Both your suggestions are true. However the reason I came up with this is out of convenience. Reminds me of the Hacker News comment where someone claimed that Dropbox was useless, because it can easily be created by just combining some Linux tools.
@cstjean@PetrKryslUCSD
Awesome. Did not know that existed. I think it would solve my problems indeed! Thanks
Are you sure about that ? My understanding was that they are run only on precompilation, and that only the __init__() function is run when using the package.
Fair. This depends on how you would define âloadâ. Iâve checked and indeed the tests trigger upon include("some_module.jl") and not on using .SomeModule.
Anyway we can mark this topic as solved. I asked whether it would be interesting and apparently we can use doctests.
Personally, Documenter has always kind of intimidated me and it seemed like far too much work to wade through itâs documentation to learn how to use it. Itâd be nice if the doctest functionality could be separated out from the rest of the package and made to work on individual functions.
Iâm imagining something like
julia> begin
"""
f(x)
add `1` to `x`
```jldoctest
julia> f(1)
2
```
"""
f(x) = x + 1
end
f
julia> @doctest f
â Info: Testing the documentation for f: total 1 test
â Test passed!
true
instead of requiring all that formal structure required by documenter. You could then integrate it into your test-suite like so:
Another location I like for test files is in âsrcâ next to the file under test, e.g. âsrc/something.jlâ and âsrc/something_test.jlâ. Tradeoff being less navigation between src and tests, but still less âclutterâ navigating source code than if tests were dominating those files.
With a reasonable editor, quick navigation between files in various directories and their buffers should not be an issue.
While one can of course always deviate from the standard file layout of Julia packages, I am not sure this should be done without a good reason, as it makes cooperation more difficult.
Seems to me that depends on the user and a bit on how source is organized. I think Vim and VSCode plus plugins are considered reasonable, maybe itâs the user whoâs lacking here.
My understanding is that the standard specifies that there should be a âtest/runtests.jlâ, not where each test file should be, which I see varying across the ecosystem even when within âtestâ. Thatâs not to say I go off-script in projects I think could end up in General, but in terms of âmaking cooperation more difficultâ the option I presented here is about as innocuous as it gets â it seems it has immediate comprehension benefits for some, and couldnât be as hard to adapt to for anyone as, say, what subset of the language to use, or what âquality of lifeâ dependencies to take on.
I.e. seems reasonable to not like it yourself, but it may be convenient to others and I donât buy the âbarrier to collaborationâ claim in this particular instance, though I may be missing something.
I would love to see a âdelayed testingâ type of module, where some lazy tests and examples can be defined, but not yet executed, right next to method definitions. I also feel like this is well suited for a macro (rather than a docstring).