Hey Folks. Is there a good “fixtures” library for unit testing in Julia? By fixtures I mean saving some data that I can use for different tests later.
Here is my idea on using fixtures. I am not sure how most folks handle testing in Julia. Do most folks just have a single test file and then they save data in the test file and use it across all tests?
In my case I have a bunch of different testing scenarios, so I would probably create different testing files–since one file will get hard to search through after a while. I would like to be able to share fixtures between files, so that I don’t have to keep duplicating the data between test files. That is the idea anyway.
I saw that there used to be a “Fixtures.jl” library, but I don’t think it is maintained anymore. I think there is also a PyTest.jl library, but I was not sure if this actually used some python pytest stuff, or if that was just the name. I believe this library has some fixture features, but did not want to use it until I checked for pure julia implementations.
From reading the Julia documentation it seems like I can also setup some variables and data in a testset so that all tests in the set can access that data. However, if I want to share the same data between different testsets, then I would probably be out of luck right?
Hmm, I think there are a few reasons to setup fixtures, but I am also not sure how Julia handles management of test cases. So say I write some test cases and then pull a CSV file in for some data. Now if the test crashes, how does the Julia test suite handle tearing down the data objects? When I say a test crashes, I mean that some critical error occurs and the kernel might panic, and then the test will fail.
I think the reason I worry about this is that when I use PyCharm with python, depending on the amount of data loaded, I definitely do see some problems with memory being released after tests. Like all of the memory consumed during testing is not released and I have to manually kill PyCharm in order to release the memory. So this might be a PyCharm thing, or it might be a pytest thing. Hence I was just wanted to know what to expect.
Worst case scenario is that the Julia test suite does not know anything about the data, and so the references to the data get lost but the data is not destroyed–so you have a memory leak. But then again the Julia garbage collector might catch this, but I just was not sure.
In the python world, pytest creates the testing data fixture. So the testing suite knows about the data and executes the tear down code if the tests crash.
I guess the same type of issue would happen if I had to connect to a database and pull some data. I could open a connection and pull the data from a function. But if the test crashes, then that data might persist in memory, and even worse the database connection remains open–because any disconnection code was not executed.
Is there a way to tell the test suite what to do in case of a test crashing, etc? Again, I am still new to Julia, so I was not sure what would happen in the cases mentioned. Would the Julia GC handle stuff like this, or are there ways to configure this in the current testing suite that might take care of these problems? I might just be overthinking this, but like I said I am still getting used to the environment. Thanks for your comments.
They would be garbage collected like everything else. But since tests are run in new processes when called with Pkg.API.test, this is not a concern even if you used manually allocated memory, eg within a C function.
It’s not clear what you mean by “crashing”, an uncaught error? If you need to release some resource in case of an error (not test failure), you can just use a finally clause.
@Tamas_Papp thanks for clarifying these things. Yes, that is what I needed. So any outstanding data/memory is released at the end of a test, even as the test is run inside of that Pkg.API.test. So that is great.
Yes, and good to know about the finally clause. I had not seen that, but seems like a good way to close database connections at the end of a test.