Is there any chance of getting a Test Adapter to allow automatic unit testing in Julia with the Test Explorer extension?
In theory, maybe, but I don’t think anyone is working actively on that right now…PRs certainly welcome!
@ZacLN sent me this example of using JuliaCST parser tech (which is the brains of the VS code plug in)
to detect testsets:
https://gist.github.com/ZacLN/86f05bdf307974e28f58d437cdddc0ba
You can use this kinda stuff to find testsets and then also to workout which testsets can be run independently.
That would form the basis of a great unit testing UI system.
idk if that is what Test Explorer does.
2 Likes
Test Explorer is a Node.js program, and the Julia plugin would probably have to be at least partly in javascript to work with the Node.js code.
Large parts of the julia extension are written in TypeScript, really only the language server and some other smaller parts are written in julia.
Is there any progress here? I would like to have a UI in VS Code for running tests, especially for running a single test, similar to Python:
I’m new to Julia, came from Swift/JavaScript. I could not find a way to run a single unit test, which is crucial for test driven development (I don’t want to run all tests after each key stroke). Is it why there is no unit test integration in VS Code? 
1 Like
No support for that. I also wouldn’t know how to do this in general with the current Julia test design: most test suites are designed as one long script, not as a compilation of individual tests that could be run e.g. out of order.
I see, thanks. Do you think it will be useful for us to create a test runner, similar to pytest in Python or rspec in Ruby?
What if we created a test runner that comes with a jutest executable? Suppose we have our code structure like this:
Animals/
├-- possum.jl
├-- possum_test.jl
├-- dolphin.jl
â””-- dolphin_test.jl
Notice that the test files are placed together with code, and not in a separate test directory, to make it easier to find/write tests.
Now we can run all our tests from Linux terminal, which will automatically run files that end with “_test.jl”.
> jutest
Or, we can run an individual file:
> jutest dolphin_test.jl
Or, like in pytest or mocha, we can run individual test sets that match the given text:
> jutest -grep "like to play"
Something like this :). Will this be useful to anybody?
The ability to test single units of tests in Julia would be tremendously helpful.
Most package that I contribute to have a top-level runtests.jl file that includes a bunch of small test unit files, which in theory could be tested separately. Right now because of the current design of Test, we can’t just test a single file without also loading the runtests.jl file.
As already mentioned, the ability to test a single file would be very useful in test-driven development. I often see myself copying snippets of code to my test files manually. Being able to just write down a script with tests, and then include it in runtests.jl later would be great.
2 Likes
If the test runner provides us better control of what to test (filtering) and/or how to test (out of order execution etc.) then it’s absolutely useful for me. Oh, implementing file level filtering in test/runtests.jl is rather easy (like this) so what I expect is more precise control like -k of pytest (-grep in your example )
By the way, it might be better to provide a way to execute the runner by executing test/runtests.jl. Since the test command of Pkg REPL mode executes the file, I believe majority of projects will continue using the file as an entry point.
1 Like
You can easily make each individual test file standalone by adding the necessary using commands to each one.
1 Like
FWIW, a relatively easy solution is to have them included from separate files, then comment out what you don’t need, and revert everything from git before committing.
A bit more complicated system could communicate with the inferior process via an environment variable — if set, run only the tests that match, if not, run all. A simple withenv wrapper to Pkg.test would take care of the rest.
Yes, but that is sub-optimal. Ideally we would have a proper treatment of the issue.
That is what I currently do. I comment out all the other files in my runtests.jl, and then uncomment then when I am ready to include the new test unit. It works for small projects, but is sub-optimal.
If the test runner provides us better control of what to test (filtering) and/or how to test (out of order execution etc.) then it’s absolutely useful for me. Oh, implementing file level filtering in
test/runtests.jlis rather easy (like this) so what I expect is more precise control like-kofpytest(-grepin your example
Thanks for the link. I think we could write a very similar code, release it as a test runner package. And then call a method from the test runner in test/runtests.jl that will automatically locate and include all the unit test files. That’s for package creators.
At the same time, if I just want to write test for my code, I don’t even need to have test/runtests.jl. I create test files, put them wherever I like, and run selected tests with a simple command from a shell. Same way this is done in many other languages like Ruby or JavaScript.
And then we can create an extension for VSCode that will use the test runner to locate and run tests from the UI.
That’s the idea.
Would you like to see Jive.jl? that’s a package to help the testing.
4 Likes
I’d love to see support for more fine-grained test execution, but I think from the VS Code extension point of view we would wait until something appear in Base and then support that. Not a hard rule, but that is my gut feeling right now.
Thank you @wookyoung, I will take a look at Jive.jl, it is great.
@wookyoung could you maybe share an example of a top-level runtests.jl file with a collection of small test units in separate files that Jive can run one-by-one?
I understand that we can either run each test file separately or run the entire runtests.jl, is that correct?
OK. I’ve created another simple package, visit here
I understand that we can either run each test file separately or run the entire runtests.jl, is that correct?
yes you could.
2 Likes