How to parallelize computation and plotting?

Specific Domains Visualization
1

The title is a bit misleading, because I would like to get a sensor’s data and plot it immadiately (so not a result of simulation or something). I get the values via TCP connection (which I can setup and print the values to console), but I want to construct a simple plot with the latest values (probably with Makie). My problems: 1) the sampling frequency is relatively high (~100 Hz) and 2) there are Makie examples only for “limited datasets”, where one iterates through the values in a range (and none where the data comes from “external source”).
(Of course) I don’t expect a fully fledged solution, I am interested in concepts or examples (which I might have overlooked).

(I asked/proposed this question here, but this time reformulated it to be more general.)

1 Like
2

Just from a general computing perspective, you aren’t going to plot at 100Hz, it seems most likely you’d plot maybe 1 to 4 times per second, using a buffer to accumulate the data.

I think you could probably do this just fine with Julia Tasks, basically one task reads data from a queue and plots it, the other task reads data from the TCP connection and puts it into the queue for plotting…

1 Like
3

In terms of pushing data to a plot, it’s pretty easy in Makie - there are a couple of unpublished examples, but it ultimately boils down to pushing to an Observable array of Points.

1 Like
4

Thank you for the suggestions, I posted an MWE at the older post (link to repo).

I used tasks and I can update the plot 4 times a second (but not yet tested for larger number of updates).
I am just curious if my approach is correct.

5

Based on the manual, it seems like using a Channel object would be the usual method for sending data between tasks, rather than the heavier weight TCP socket route.

6

The value’s source is another PC, therefore the TCP. The sender side is only implemented here to give a full working example.

7

I see, you’re using low level thread sync primitives for this. It seems to me a Channel is a higher level abstraction that’s probably easier to use and more straightforward to understand. Basically you’d have two tasks, one does (in pseudocode)

while(true)
   read from socket
   send to channel
end

and the other does

while(true)
   read from channel
   plot
end
8

Oh I see. I did not considered using two threads (yet) as the plotting thread is the “main julia thread”.
I’ll consider this approach, thank you for the suggestion!