I’m writing a package to plot quantum circuits: QuantumCircuitDraw.jl but im not sure how to write testsets for this kind of code. How can I ensure its plotting the right thing without having to manually check the output plot ?
I don’t know any other solution but using reference images. We do that for Makie and AlgebraOfGraphics, for example. You store image examples of code that works and check against them in CI to make sure your code doesn’t regress. It can be difficult to cover all possible behaviors of your plotting code as the amount of possible composition in plotting tends to be really large and interactions are complex. For example, we cannot check in Makie that every way you could use a poly plot works with anti-aliased text rendered on top with all the possible kinds of fill options and polygon shapes etc. etc.
2 Likes
In other languages (MATLAB) I have tested plotting by inspecting and verifying the number or attributes of the children associated with a figure, axes, or app.
However, I am new to Julia and have wondered if an analog to that exists. So I am interested to know more too.
What is the distance metric used between stored test images and generated images?
1 Like
As one example, ReferenceTests.jl uses PSNR: Home · ReferenceTests.jl
2 Likes
There is VisualRegressionTests.jl, which automates the reference process. It does a pixel comparison (with some tolerance) and prompts you when there is a difference.
2 Likes
Another thing you might want to do is to separate any internal logic (like testing warnings and errors if inputted parameters don’t make sense) from plotting.
There is a nice article for c++ about it: “The Humble Dialog Box” by Michael Feathers. I am not sure whether there is a similar article for Julia.
1 Like