# Makie - How to add ReferenceTests when contributing to Makie
**URL:** https://discourse.julialang.org/t/makie-how-to-add-referencetests-when-contributing-to-makie/126092
**Category:** Visualization
**Tags:** makie
**Created:** [February 20, 2025, 12:23am UTC](https://discourse.julialang.org/t/makie-how-to-add-referencetests-when-contributing-to-makie/126092 "2025-02-20T00:23:24Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@mikeliux](https://discourse.julialang.org/u/mikeliux)
#### Post date: [February 20, 2025, 12:23am UTC](https://discourse.julialang.org/t/makie-how-to-add-referencetests-when-contributing-to-makie/126092/1 "2025-02-20T00:23:24Z")
I am not an experienced julia user, but I would like to contribute with a couple of minor implementations in Makie.jl. One of them is almost done, but I got stucked at one thing: I must implement a ReferenceTest.
This ReferenceTest is used to compare the output of a test code that produces an image with a given reference image.
Let us focus on my [PR #4800 here](https://github.com/MakieOrg/Makie.jl/pull/4800), which simply allows the user to pass the `alpha` (transparency) parameter to a tricontourf plot.
I have already implemented the change in a git branch and commited a PR. Furthermore, I tried to add the new test (without running it locally), by introducing this code into [Makie.jl / ReferenceTests / src / tests / examples2d.jl](https://github.com/MakieOrg/Makie.jl/blob/master/ReferenceTests/src/tests/examples2d.jl):
```julia
@reference_test "tricontourf alpha transparency" begin
dxy = 1.0;
x = [0.0, dxy, 0.0, -dxy, 0.0, dxy/2, -dxy/2, dxy/2, -dxy/2];
y = [0.0, 0.0, dxy, 0.0, -dxy, dxy/2, dxy/2, -dxy/2, -dxy/2];
@. f1(x,y) = x^2 + y^2;
z = f1(x,y);
f = Figure()
ax1=Axis(f[1,1], title = "alpha = 1.0 (default)")
ax2=Axis(f[1,2], title = "alpha = 0.5 (semitransparent)")
hlines!(ax1, [-0.5, 0.0, 0.5])
hlines!(ax2, [-0.5, 0.0, 0.5])
tricontourf!(ax1, x, y, z, levels = 3)
tricontourf!(ax2, x, y, z, levels = 3, alpha=0.5)
f
end
```
I added test this and committed the PR to the official Makie repository on GitHub, but in fact I don’t know how to run this test locally.
When I send the PR, the test is failing because there is no reference image, and I also don’t know how to add this image and upload it.
In [Makie.jl / CONTRIBUTING.md](https://github.com/MakieOrg/Makie.jl/blob/master/CONTRIBUTING.md), the following instruction is provided:
> Please ensure locally that your feature works by running the tests. To be able to run the tests, you have to `dev` the helper package `ReferenceTests` that is part of the Makie monorepo. `ReferenceTests` is not a registered package, so you have to do `]dev path/to/ReferenceTests`.
For my development, I am using these versions:
- julia:1.11.3
- Makie: 0.22.1
- git: 2.34.1.windows.1
- vscode: 1.97.2
I started my development like this:
`] add Revise`
`] dev Makie`
`] dev CairoMakie`
And then:
```julia
using Revise
using Makie
using CairoMakie
```
So I tried to add the package ReferenceTests.jl as follows:
`]dev . ReferenceTests`
In the documentation of [ReferenceTests.jl](https://github.com/JuliaTesting/ReferenceTests.jl), is says that the images with the missing references are created automatically. So I suppose I just have to run a test locally and the image will be created.
But I don’t have a folder with images in my locally cloned repo. When I check the source code of [`Makie.jl / .gitignore`](https://github.com/MakieOrg/Makie.jl/blob/master/.gitignore), I see that these paths are excluded and therefore not loaded into my computer:
```julia
WGLMakie/test/reference_images/
GLMakie/test/reference_images/
CairoMakie/test/reference_images/
```
That means, I won’t have the images of existing tests in my local repo, and if I create a new image, I think it wont be sent to GitHub.
Furtheremore, there is a file at [ReferenceUpdater / src / image\_download.jl](https://github.com/MakieOrg/Makie.jl/blob/65495fabda1f59cf2f567d9877c9aa8b511d619d/ReferenceUpdater/src/image_download.jl). This file downloads or uploads the images as required, (or so I think).
So, here are my questions:
1. What packages should I add and dev?
2. If I add ReferenceTests.jl, should that be a subfolder of Makie.jl?
3. Should I `add]` (or `]dev`?) all Makie backends?
4. How do I create and upload a reference image?
5. There are many tests in `examples2d.jl` and I don’t want to run them all. How do I run locally only the reference test that I created?
Thank you in advance.
---
### Author:  [@jules](https://discourse.julialang.org/u/jules)
#### Post date: [February 20, 2025, 8:37am UTC](https://discourse.julialang.org/t/makie-how-to-add-referencetests-when-contributing-to-makie/126092/2 "2025-02-20T08:37:53Z")
I think that documentation probably needs an update. For a basic PR that includes a new reference test, you basically just have to add the code creating a figure in one of the reference test files, wherever it fits best, give it a reasonable name, and test locally that if you run the code inside the macro, you get the figure you were expecting.
Then, when you make the PR, our CI will render the new image and create a note that a new reference image needs to be added. This will be done by a maintainer before merging the PR. You cannot upload a reference image, the set of reference images is managed by the maintainers because it is not part of the code and lives as a separate archive within the github repo under releases.
---
### Author:  [@mikeliux](https://discourse.julialang.org/u/mikeliux)
#### Post date: [February 21, 2025, 9:41am UTC](https://discourse.julialang.org/t/makie-how-to-add-referencetests-when-contributing-to-makie/126092/3 "2025-02-21T09:41:44Z")
> [@jules](#):
>
> Then, when you make the PR, our CI will render the new image and create a note that a new reference image needs to be added. This will be done by a maintainer before merging the PR. You cannot upload a reference image, the set of reference images is managed by the maintainers because it is not part of the code and lives as a separate archive within the github repo under releases.
Thank you @jules. I marked the PR as completed, since the reference test had already been added, it was just the new reference image that was missing.
For more complex PRs, i.e. when I change existing functionality, I need to make sure that it doensn’t break other tests, or their reference images. I suppose it is then need it to run the tests locally, which I am not sure how to do. I’d appreciate some clear instructions on this.
Best regards.
---
### Author:  [@jules](https://discourse.julialang.org/u/jules)
#### Post date: [February 21, 2025, 9:46am UTC](https://discourse.julialang.org/t/makie-how-to-add-referencetests-when-contributing-to-makie/126092/4 "2025-02-21T09:46:29Z")
To be honest, I don’t know because I always run the tests on CI and only test in a more ad-hoc kind of way locally. Maybe @sdanisch can elaborate on what the current workflow should be like for local tests
---
### Author:  [@icweaver](https://discourse.julialang.org/u/icweaver)
#### Post date: [July 9, 2025, 2:00pm UTC](https://discourse.julialang.org/t/makie-how-to-add-referencetests-when-contributing-to-makie/126092/5 "2025-07-09T14:00:40Z")
Also v interested in this