Asynchronous Makie

General Usage Performance
1

Hi,

I wrote a toy CFD solver computing pressure and velocity fields for successive timesteps and I use Makie for plotting the result and store a movie.

Although I only plot new frames every 200 timesteps, the plotting time becomes dominant (e.g. contour with 100 levels or streamplots).

I wonder if there is a simple way (or even better an example) to execute Makie computations on another cpu core ?

2

I’d guess that you could use Distributed and a RemoteChannel. Have the worker generate the data and pass it to the controller over the RemoteChannel. Then your code that feeds Makie would read the data from the RemoteChannel and pass it to Makie.

Things should run faster however if you generate data faster than Makie can process it (Makie is using more that 50% of the time in your current setup) then your generate process will end up idling waiting for Makie to catch up. However with this setup the total time should be controlled by how long it takes Mackie to process the data.

3

Thanx, I wonder if 1.3 threads could make this simpler…

4

streamplot & contourplot are probably some of the slowest Makie plot types, since they do most of the work on the CPU…So using other plot types could be an order of magnitude faster - if you share your code, we could take a look at it…
Threads won’t help yet, since makie currently isn’t thread safe. You could also consider starting makie in a different process and then sent the data to that process!

5

Hi Simon.
First a big thank for your wonderful job on Makie !

I did notice that contour was slower than other kind of plots. I also have the feeling that when I use a user defined level arrays like
contour(..., levels=[l1,l2,..,li])
contour gets even slower.

I did not study the contour algorithm so I don’t know if it could be easily adapted for GPU (it would be great).

My toy CFD is a bit too large (and messy) to be shared by now but I shall find some time to send a MWE.
In particular I wonder if my use of observable and animation is optimal.

I have a loop like

q=myMakiePlot(....) # where q is an array observable
record(scene,"test.mp4",framerate=10) do io
    for i=1:ntimesteps
        ...
       # compute a new array Q
       ...
      if mod(i,100)==0
          q[]=Q
      end
   end
end
function myMakiePlot(...)
     scene=Scene(resolution=(400,400),scale=false)
     qn=Node(rand(nx+1,ny+1))
 
     contour!(scene,x,y,lift(a->a,qn),linewidth=2,levels=100,fillrange=false,limits=limits,axis = (showticks = false, showaxis = false,))
   
     qn
end
6

You could try:

contour(rand(4, 4), fillrange = true)

Which should execute on the gpu, but will look different and maybe a bit more ugly :wink:

2 Likes
7

Thanks, I will try that.

8

@LaurentPlagne Very interested to find out more about your CFD solver. I worked for an F1 team and we used a commercial package - no not the obvious one.
I am asking though - you are doing the plots after every timestep? As you know better than me the usual workflow is to run the solver, then run a post processing step to produce the pressure plots, contours, and the movies etc.
Using Julia from start to finish with Makie does give you the facility to do it in real time I suppose. Wow.

2 Likes
9

Hi John,

This CFD “solver” is only a toy solver that we intend to use as a basis for a Julia lecture (just like I did with my toy 2D Shape collisionner Plot a circle with a given radius with Plots.jl - #8 by LaurentPlagne
https://discourse.julialang.org/uploads/short-url/tJmPIH9nmhtaudwxmZR8HexScwy.gif

I have translated the famous mit18086 from Matlab to Julia and replaced the cholmod solver by an home made geometric multigrid solver (that must be able to solve full neumann B.C.) translated from C++ to Jula.

The resulting Julia code is reasonably fast (5-100 times faster than the matlab original code depending on the mesh size)

For your question, I dot not use the classical post-processing workflow and the Makie plots are computed and displayed during the simulation (and the movie being recorded live) but I do not emit a frame by timestep: there is about 20k \Delta t and around 200 frames :wink:

I agree that visualization during simulation is rather convenient and do this only with Julia is very nice !

3 Likes
10

Wow, the speedup with fillrange=true is impressive (x10) !
Thank you again

Laurent

P.S. I would not say that the GPU contour is ugly… maybe stylish :wink:
Contour GPU
Contour on cpu
… the youtube compression is ugly :sleepy:

1 Like
11

Yeah this is a case where the contour algorithm on the GPU works out really nicely :slight_smile:
There are some others where it looks pretty odd compared to what people expect.

12

What does fillrange do? Could you add a blurb to the Contour docstring about it if possible?

13

I have just tried the newly introduced streamplot Makie feature : great !!
test
Thank you again !!

7 Likes
14

I was thinking about barking up this tree for a project I’ve been working on but can’t decide if Makie is the way to go or not. Is there a simple means to create a video stream from Plots in Julia? If so it seems that would reduce a lot of the headache for me.

15

“Simple” depends a lot on the user.
I made animations both with Plots an Makie. Makie is less “simple” to me because it is based on observables, lift and record( ) do constructs which are not yet very familiar to me. According to @ffevotte, the later is, on the contrary, very familiar to user with a native functional programming background. I suspect that the former (observables, lift) allows for interactive plots and In this case it allows for monitoring the animations during the fluid computation which I found very convenient (if not simple).
Hope it helps :wink:

1 Like
16

For @tyfdzt: here are the basic components extracted from my code with Makie streamplots.
Note that it is not a MWE. I hope it can help you. If not I can try to write a MWE.

#The Cartesian axis :
x = range(0,stop=lx,length=nx+1); hx = lx/nx;
y = range(0,stop=ly,length=ny+1); hy = ly/ny;
#Arrays for x and y velocity arrays that evolve via CFD simulation
Ue=zeros(nx+2,ny+2)
Ve=zeros(nx+2,ny+2)
.....
#Declare a Makie Node (once) for storing velocity arrays
uvn=Node([rand(size(Ue)...),rand(size(Ve)...)])
.....
#Update the velocity uvn Node for each video frames
# Note that this is not done for each time steps but only for each video frames
push!(uvn,copy!(to_value(uvn),[Ue,Ve]))
.....
#Helper function : the Makie streamplot! function take a function 
# for its first argument (not an array)
function make_velocity(uv)
     f_vx=interpolate(uv[1],BSpline(Quadratic(Reflect(OnCell()))))
     f_vy=interpolate(uv[2],BSpline(Quadratic(Reflect(OnCell()))))
  function velocity(x,y)
     nx,ny=size(uv[1])
     xi=1+x*(nx-1)
     yi=1+y*(ny-1)
     Point2f0(f_vx(xi,yi),f_vy(xi,yi))
  end
  return velocity
end

# The call to the Makie streamplot! function : 
#      this function is called each time the uvn Node is updated 
streamplot!(scene,lift(a->make_velocity(to_value(a)),uvn),
       x,y, colormap = :magma, arrow_size=0.02, gridsize=(20,20))

Let me know if this helps.

1 Like
17

I have drawn the streamline using PyPlot. Thanks all the same.

18

I asked this on an older thread, but here I see you did complete the geometric multigrid. I wonder if it would be worth it to write a fast general package for this.