# Makie Observable subtlety with contourf using vectors

I have a dataset which is a 4xN array of numbers corresponding to field values at (z,y,x)-positions, sampled sparsely in z. I am trying to use `contourf!` in Makie to make a video of the evolution of these fields, which requires indexing into the array to find the subsets which correspond to each z-level.

The problem arises with the update, making use of `Observable`, and how to implement the plotting both performantly and correctly. Thus far I have only managed “very slow and correct” or “slow and incorrect”. Using PyPlot.jl, I can make an equivalent plot using `tricontourf` at 2 it/s, so my approach in CairoMakie is between 4x and 192x slower in this instance. I would really appreciate any insight about how to approach this a better way.

Code showing attempts below:

``````using CairoMakie, ProgressBars

const nts = 1:5
const zs = [1,24,50]

function generateObsData(; zs=[1,24,50])

obs = [];
for x in 1:5:200, y in 1:5:200, z in zs
push!(obs,[z, y, x, rand()])
end
obs = reduce(hcat,obs)

return obs

end

const obs = generateObsData(; zs=zs);

function main1()

iz = [findall(z.==obs[1,:]) for z in zs]
ys = [Observable(obs[2,z])  for z in iz]
xs = [Observable(obs[3,z])  for z in iz]
us = [Observable(obs[4,z])  for z in iz]

fig = Figure(resolution = (900, 400))
axs = [ Axis(fig[i, j], width = 200, height = 200) for i in 1:1, j in eachindex(zs) ]

for ax in axs
hidedecorations!(ax)
end

for j in eachindex(zs)
if length(us[j][]) > 0
contourf!(axs[1,j], ys[j], xs[j], us[j];levels=range(0,1,129),colormap=:Oranges,colorrange=(0,1),rasterize=true)
end
end
Colorbar(fig[1:1,length(zs)+1]; colorrange=(0,1), colormap=:Oranges, label=L"u")
tit = Label(fig[0,:], text = L"\$t = 0\$ []", textsize = 24)
resize_to_layout!(fig)

record(fig, "obsTest1.mp4", ProgressBar(nts); framerate=10) do n
obs .= generateObsData(; zs=zs);
for (j,z) in enumerate(iz)
ys[j][] = obs[2,z]
xs[j][] = obs[3,z]
us[j][] = obs[4,z]
end
tit.text[] = L"\$t = %\$(n-1)\$ []"
end

return nothing
end

function main2()

OO = Observable(obs)

fig = Figure(resolution = (900, 400))
axs = [ Axis(fig[i, j], width = 200, height = 200) for i in 1:1, j in eachindex(zs) ]

for ax in axs
hidedecorations!(ax)
end

for (j,z) in enumerate(zs)
iz = findall(z.==obs[1,:])
contourf!(axs[1,j], OO[][2,iz], OO[][3,iz], OO[][4,iz];levels=range(0,1,129),colormap=:Oranges,colorrange=(0,1),rasterize=true)
end
Colorbar(fig[1:1,length(zs)+1]; colorrange=(0,1), colormap=:Oranges, label=L"u")
tit = Label(fig[0,:], text = L"\$t = 0\$ []", textsize = 24)
resize_to_layout!(fig)

record(fig, "obsTest2.mp4", ProgressBar(nts); framerate=10) do n
obs .= generateObsData(; zs=zs);
OO[] = obs;
tit.text[] = L"\$t = %\$(n-1)\$ []"
end

return nothing
end

main1()	# ~48s/it; very slow, but correctly updating contourf's

main2()	# ~2s/it; still slow, but contourf's do not update

``````

For animations using `GLMakie` is a lot faster. If there is no particular reason to use `CairoMakie`, the first thing to do would be to try `GLMakie` The second thing is that the `Observables` should be vectors, not the individual values…

Couple of things here. First of all, it’s a bit tricky to get the observables update correct with multiple axes depending on the same data. Something like this rough attempt can work:

``````function main1()

observations = Observable(obs)

iz = [findall(z.==obs[1,:]) for z in zs]
ys = [Observable(obs[2,z])  for z in iz]
xs = [Observable(obs[3,z])  for z in iz]
us = [Observable(obs[4,z])  for z in iz]

on(observations) do obs
iz = [findall(z.==obs[1,:]) for z in zs]
for (i, z) in enumerate(iz)
ys[i].val = obs[2,z]
xs[i].val = obs[3,z]
us[i][] = obs[4,z]
end
end

fig = Figure(resolution = (900, 400))
axs = [ Axis(fig[i, j], width = 200, height = 200) for i in 1:1, j in eachindex(zs) ]

for ax in axs
hidedecorations!(ax)
end

for j in eachindex(zs)
if length(us[j][]) > 0
contourf!(axs[1,j], ys[j], xs[j], us[j];levels=range(0,1,129),colormap=:Oranges,colorrange=(0,1),rasterize=true)
end
end
Colorbar(fig[1:1,length(zs)+1]; colorrange=(0,1), colormap=:Oranges, label=L"u")
tit = Label(fig[0,:], text = L"\$t = 0\$ []", textsize = 24)
resize_to_layout!(fig)

# record(fig, "obsTest1.mp4", ProgressBar(nts); framerate=10) do n
record(fig, "obsTest1.mp4", nts; framerate=10) do n
observations[] = generateObsData(; zs=zs);
# for (j,z) in enumerate(iz)
# 	ys[j][] = obs[2,z]
# 	xs[j][] = obs[3,z]
# 	us[j][] = obs[4,z]
# end
tit.text[] = L"\$t = %\$(n-1)\$ []"
end

return nothing
end
``````

But this is still slow. However, you’re comparing `contourf` with `tricontourf` which is a different algorithm. When you use `contourf`, the z vector is collated into a matrix with NaN entries, so there’s more data to go over than in the sparse representation. Makie currently doesn’t have tricontourf, but there was some progress on this issue a couple months ago https://github.com/JuliaPlots/Makie.jl/issues/1744

Also, you’re using 130 levels, which is so many that I wonder if a normal heatmap wouldn’t do because it will be hard to see contour lines at that resolution. For example, when I do `levels=range(0,10,15)` it’s only 1.3 seconds overall (a lot of time is spent on the polygon/mesh calculations.

The best way forward would probably be to get tricontourf working.

Here’s an attempt of doing that:

Thank you for the thorough answer. Ultimately, I am interested in a continuous representation of the data, but the sampling positions (`ys` and `xs`) are relatively sparse and may vary with depth `z`. Your solution neatly handles this by updating `iz` when `observations` is updated. For clarity: is there a reason for updating `us[i][] = `… while `ys[i].val = `… and `xs[i].val = `…?
Using `heatmap!` is substantially faster, and passing `interpolate=true` provides a reasonable approximation to `PyPlot.tricontourf` with a large number of levels. Thank you for your help.

That’s great, thank you very much!

No that was just sloppy editing 