Julia for real time, worried about the garbage collector

General Usage Performance
1

Hello, totally new. I’ve read about Julia and the concept seems great.

I have a block diagram software (like simulink) that automatically creates a script, works well and it is using m-code where every block is a function.
The basic layout of the script is like this:

parameters
loop start
functions that take insignals from other functions and create outsignals to other functions and also sends its own parameters to itself
loop end

I want to do this with julia to reach top speed and want to be able to run it in real time.

My question is how to avoid the garbage collector to run and slow things down.

This is my idea:
Initiate the parameters before the loop, every function initiates in the first iteration of the loop and in this first iteration the parameters can change type and size. Run the garbage collector at the end of the first iteration.
Now the parameters will never change type/size allocation. Will this save me from the garbage collector?

Thanks, and nice work with this language.

3 Likes
2

example of the autocode generated:

gs0_0=tic;                    %LINE:1
addpath('/home/cr/eclipse/GoSim/functions');                    %LINE:2
addpath('/home/cr/eclipse/GoSim/user');                    %LINE:3
count_=1;                    %LINE:4
h_=0.01;                    %LINE:5
stop_=10;                    %LINE:6
a0_0=1;                    %LINE:7
a0_1=0;                    %LINE:8
a1_0=1;                    %LINE:9
a1_1=1;                    %LINE:10
a1_2=0;                    %LINE:11
gs0_1=0;                    %LINE:12
gs0_2=tic;                    %LINE:13
gs0_3=tic;                    %LINE:14
for i_=0:h_:stop_                    %LINE:15
[a1_0,a1_1,a1_2,z0]=Sine(a1_0,a1_1,a1_2,i_,h_,stop_,count_,'a1'); %a1                    %LINE:16
[a0_0,a0_1]=Scope(a0_0,a0_1,i_,h_,stop_,count_,'a0',z0); %a0                    %LINE:17
[gs0_0 gs0_1 gs0_2 gs0_3]=GoSimCom(gs0_0,gs0_1,gs0_2,gs0_3,i_,h_,stop_,count_,0);                    %LINE:18
count_++;                    %LINE:19
end                    %LINE:20
3

so basically, would this run your garbage collector? sorry I haven’t learnt your syntax yet:

p=0;
p1=0;
for loop start
p,z=function( p)
p1=function1(p1,z)
run garbage collector in the first iteration of the loop
for loop end

function/function1 initiate in the first iteration and p and p1 change to other types and matrices and need more memory. But after that they only change values and are of the same type.

Will this format save me from the garbage collector?

1 Like
4

Do you know for a fact that garbage collection will be a significant performance barrier for your program? I’m confused why you are so afraid of and concerned about gc.

3 Likes
5

I may be wrong, but I think those states are allocated and freed on the stack, so there is no GC unless you allocate vectors and mutable structs.

Also you can wrap state into one tuple or named tuple for clearer high-level code, like so:

state = (a1_0 = 0.01, a1_1 = 10)
out1, out2, state = function(in1, in2, state)

Also, you should not run that code into a main script for performance reasons - wrap it into some main_function, and then benchmark it using @time main_function(), that shows allocations and time.

6

Isn’t it pretty much no garbage, no collection? Just don’t allocate in your loop and use functional techniques.

2 Likes
7

I read about an example where one was running something at 300 Hz and when the gc kicked in it dropped to 80 Hz. This can not happen in a real time control scenario.

8

why would it be faster as a main_function?

9

It would be faster because if the variables are inside a function, the compiler has more information and guarantees about the types of the variables, and produce optimized code

10

Do you care about throughput or latency? If you care about latency, what are your deadlines? Put another way - do you care mostly about how often your function runs on average, or do you need a guarantee on the maximum amount of time between two iterations?

In general, Julia is not hard realtime-safe (i.e. not suitable for situations where missing a latency deadline is a safety issue). That said, in practice you can get pretty far by making sure you pre-allocate all your memory so you’re not creating any GC pressure in your loop, and also make sure to call all your functions at least once before your loop so that the JIT compiler has a chance to compile your functions before the loop starts.

There are some tools to inspect your memory allocation (see the @allocated macro as well as the --track-allocation command line flag).

11 Likes
11

I think this talk: JuliaRobotics: Making robots walk with Julia | Robin Deits - YouTube by @rdeits is a great example of someone showing how Julia can be made to handle very complicated realtime, low-latency systems quite well if you know what you’re doing.

I would recommend just trying to learn the language and get a solid handle on it before attempting to build a low-latency system with it though, otherwise you may feel like you’re shoving a square peg in a round hole.

Julia wasn’t really designed with low-latency stuff in mind, though it’s perfectly capable of it, it’s just that the standard performance tips, reading materials and libraries might not support you as well as you’d like.

8 Likes
12

latency is more important than throughput, in my words it would be minimal time jitter is more important than speed.

How often does the GC run? if nothing is allocated can I be safe that it doesn’t run?

13

i watched the slides in the talk, and they seem to be happy with not just allocating memory in their control loop.

This is my program running m-code in real time in 100 Hz. I use a command to “close” one CPU-core for the OS and then put my Octave process in that CPU-core to make it never be interrupted, isn’t that a guarantee :slightly_smiling_face:. This is on a standard ubuntu core and not the low latency core. 80% CPU load. The problem is that m-script is slow, If I can achieve the same performance (time jitter and repeatability) with julia I would be happy. The GC is the only thing that I am not sure about. I need to know when it runs.

x: time (sec)
y: sample time (sec)
red: moving average

octave_pause_new_80_load
octave_pause_new_80_load500×555 34.9 KB

1 Like
14

What platform are you running on and how do your control signals get to the real world?

I did some playing with gpios on a raspberry pi and could toggle a pin every 100 ns. Occasionally there would be dropped out sections, but garbage collection is not the only cause. If you are not running on an RTOS with some guarantees about execution times, you are also subject to the whims of the OS setting your task aside as it deals with other tasks, including low level stuff of servicing networks and other peripherals.

You might want to run a test of something simple like that and see what interruptions you are getting. Most real time systems tolerate some amount of jitter or even skipped cycles in the control loop. In cases where it really must be zero / highly consistent we sometimes move the control into digital logic (e.g FPGAs).

My main points: you can tolerate some jitter, zero is not usually a realistic goal, and the garbage collector is not the only source of stalling / jitter.

EDIT: I was typing at the same time as you - your recent post answered my questions and I now see that you’re addressing my concerns about the OS by “closing” one core to the OS.

1 Like
15

Julia does not make formal promises around what will and won’t cause allocations or trigger GC, but in practice if you run julia with the regular optimizations (i.e. not in an interpreter like JuliaInterpreter.jl or julia --compile=no), if you only use stack allocated data structure (or preallocate any heap allocated structures like Arrays), then you should never encounter heap allocations which means the GC should not run.

You need to do tests to make sure you’re doing things right and you didn’t hit a corner case like the compiler bailing out of inference and doing dynamic dispatch, but as long as you know what you’re doing and test rigorously, GC jitter should be completely avoidable in julia (definitely moreso than Octave!).

1 Like
16

Thanks for info, I will practice and see what I can achieve. Julia devs should have thought about real time performance, it would make it so much more industrially viable.
Every car has several computers in them…

But I’m optimistic.

17

Thanks for info, I will practice and see what I can achieve. Julia devs should have thought about real time performance, it would make it so much more industrially viable.
Every car has several computers in them…

But I’m optimistic.

18

I use Linux, I haven’t tried with the low latency core yet but it should be good.

My own trick is to separate one CPU-core from the OS and put my process in it so it can’t be interrupted. Works well.

19

Can you share what you did to close the core? I may want to try that later on the Raspberry Pi.

20

I assure you that the julia devs have thought more about julia’s use-cases and potential applications than anyone else.

2 Likes