Makie Observable subtlety with contourf using vectors

General Usage
1

I have a dataset which is a 4xN array of numbers corresponding to field values at (z,y,x)-positions, sampled sparsely in z. I am trying to use contourf! in Makie to make a video of the evolution of these fields, which requires indexing into the array to find the subsets which correspond to each z-level.

The problem arises with the update, making use of Observable, and how to implement the plotting both performantly and correctly. Thus far I have only managed “very slow and correct” or “slow and incorrect”. Using PyPlot.jl, I can make an equivalent plot using tricontourf at 2 it/s, so my approach in CairoMakie is between 4x and 192x slower in this instance. I would really appreciate any insight about how to approach this a better way.

Code showing attempts below:

using CairoMakie, ProgressBars

const nts = 1:5
const zs = [1,24,50]

function generateObsData(; zs=[1,24,50])

	obs = [];
	for x in 1:5:200, y in 1:5:200, z in zs
		push!(obs,[z, y, x, rand()])
	end
	obs = reduce(hcat,obs)
	
	return obs
	
end

const obs = generateObsData(; zs=zs);

function main1()
	
	iz = [findall(z.==obs[1,:]) for z in zs]
	ys = [Observable(obs[2,z])  for z in iz]
	xs = [Observable(obs[3,z])  for z in iz]
	us = [Observable(obs[4,z])  for z in iz]

	fig = Figure(resolution = (900, 400))
	axs = [ Axis(fig[i, j], width = 200, height = 200) for i in 1:1, j in eachindex(zs) ]

	for ax in axs
		hidedecorations!(ax)
	end

	for j in eachindex(zs)
		if length(us[j][]) > 0
			contourf!(axs[1,j], ys[j], xs[j], us[j];levels=range(0,1,129),colormap=:Oranges,colorrange=(0,1),rasterize=true)
		end
	end
	Colorbar(fig[1:1,length(zs)+1]; colorrange=(0,1), colormap=:Oranges, label=L"u")
	tit = Label(fig[0,:], text = L"$t = 0$ []", textsize = 24)
	resize_to_layout!(fig)

	record(fig, "obsTest1.mp4", ProgressBar(nts); framerate=10) do n
		obs .= generateObsData(; zs=zs);
		for (j,z) in enumerate(iz)
			ys[j][] = obs[2,z]
			xs[j][] = obs[3,z]
			us[j][] = obs[4,z]
		end
		tit.text[] = L"$t = %$(n-1)$ []"
	end
	
	return nothing
end

function main2()
	
	OO = Observable(obs)
	
	fig = Figure(resolution = (900, 400))
	axs = [ Axis(fig[i, j], width = 200, height = 200) for i in 1:1, j in eachindex(zs) ]

	for ax in axs
		hidedecorations!(ax)
	end

	for (j,z) in enumerate(zs)
		iz = findall(z.==obs[1,:])
		contourf!(axs[1,j], OO[][2,iz], OO[][3,iz], OO[][4,iz];levels=range(0,1,129),colormap=:Oranges,colorrange=(0,1),rasterize=true)
	end
	Colorbar(fig[1:1,length(zs)+1]; colorrange=(0,1), colormap=:Oranges, label=L"u")
	tit = Label(fig[0,:], text = L"$t = 0$ []", textsize = 24)
	resize_to_layout!(fig)

	record(fig, "obsTest2.mp4", ProgressBar(nts); framerate=10) do n
		obs .= generateObsData(; zs=zs);
		OO[] = obs;
		tit.text[] = L"$t = %$(n-1)$ []"
	end
	
	return nothing
end

main1()	# ~48s/it; very slow, but correctly updating contourf's

main2()	# ~2s/it; still slow, but contourf's do not update
2

For animations using GLMakie is a lot faster. If there is no particular reason to use CairoMakie, the first thing to do would be to try GLMakie :wink:

The second thing is that the Observables should be vectors, not the individual values…

3

Couple of things here. First of all, it’s a bit tricky to get the observables update correct with multiple axes depending on the same data. Something like this rough attempt can work:

function main1()
	
    observations = Observable(obs)

	iz = [findall(z.==obs[1,:]) for z in zs]
	ys = [Observable(obs[2,z])  for z in iz]
	xs = [Observable(obs[3,z])  for z in iz]
	us = [Observable(obs[4,z])  for z in iz]

    on(observations) do obs
        iz = [findall(z.==obs[1,:]) for z in zs]
        for (i, z) in enumerate(iz)
            ys[i].val = obs[2,z]
            xs[i].val = obs[3,z]
            us[i][] = obs[4,z]
        end
    end

	fig = Figure(resolution = (900, 400))
	axs = [ Axis(fig[i, j], width = 200, height = 200) for i in 1:1, j in eachindex(zs) ]

	for ax in axs
		hidedecorations!(ax)
	end

	for j in eachindex(zs)
		if length(us[j][]) > 0
			contourf!(axs[1,j], ys[j], xs[j], us[j];levels=range(0,1,129),colormap=:Oranges,colorrange=(0,1),rasterize=true)
		end
	end
	Colorbar(fig[1:1,length(zs)+1]; colorrange=(0,1), colormap=:Oranges, label=L"u")
	tit = Label(fig[0,:], text = L"$t = 0$ []", textsize = 24)
	resize_to_layout!(fig)

	# record(fig, "obsTest1.mp4", ProgressBar(nts); framerate=10) do n
	record(fig, "obsTest1.mp4", nts; framerate=10) do n
		observations[] = generateObsData(; zs=zs);
		# for (j,z) in enumerate(iz)
		# 	ys[j][] = obs[2,z]
		# 	xs[j][] = obs[3,z]
		# 	us[j][] = obs[4,z]
		# end
		tit.text[] = L"$t = %$(n-1)$ []"
	end
	
	return nothing
end

But this is still slow. However, you’re comparing contourf with tricontourf which is a different algorithm. When you use contourf, the z vector is collated into a matrix with NaN entries, so there’s more data to go over than in the sparse representation. Makie currently doesn’t have tricontourf, but there was some progress on this issue a couple months ago https://github.com/JuliaPlots/Makie.jl/issues/1744

Also, you’re using 130 levels, which is so many that I wonder if a normal heatmap wouldn’t do because it will be hard to see contour lines at that resolution. For example, when I do levels=range(0,10,15) it’s only 1.3 seconds overall (a lot of time is spent on the polygon/mesh calculations.

The best way forward would probably be to get tricontourf working.

4

Here’s an attempt of doing that:

1 Like
5

Thank you for the thorough answer. Ultimately, I am interested in a continuous representation of the data, but the sampling positions (ys and xs) are relatively sparse and may vary with depth z. Your solution neatly handles this by updating iz when observations is updated. For clarity: is there a reason for updating us[i][] = … while ys[i].val = … and xs[i].val = …?
Using heatmap! is substantially faster, and passing interpolate=true provides a reasonable approximation to PyPlot.tricontourf with a large number of levels. Thank you for your help.

6

That’s great, thank you very much!

7

No that was just sloppy editing :slight_smile: