Help with real-time performance needed

[Palli]

For soft-real time I would try out in Julia 1.9:

New option --heap-size-hint=<size> gives a memory hint for triggering greedy garbage collection.

Try different values, e.g. as low as you can go.

For hard-real time (i.e. any dropped package is catastrophic), you want NO allocations, and thus get rid of GC as a potential trouble.

Whatever you do, try disabling threads, it can slow down the GC a lot:

Very different performance in different computers due to garbage collection?

Thanks! How many threads are you using? Does the problem happen with one thread? In the desktop profile, we mostly see threads waiting for gc to finish, but it’s not clear whether that’s a cause or an effect. For example maybe using the CPU pause instruction in jl_safepoint_wait_gc is causing the kernel to de-schedule us for a long time on one system.

[Of course we want threads to work well with the GC, so hopefully that’s a temporary issue, maybe already fixed on 1.9?]

The GC in Julia isn’t hard-real time (such GC is available in e.g. Java, and maybe will be made later in Julia… but I wouldn’t hold by breath since, it, unlike HPC, not a priority, also easy to avoid the GC), but I’m not sure it’s too good (currently) even for soft-real time, at least with threading.

There’s ongoing work on the GC, I believe some already landed in 1.9 (with default settings), at least the option above. Historically Julia had no options for its GC (well, except for turning it off; temporarily, which isn’t a great idea), and I believe that new option is still the only option.

What you might want is set a high enough absolute (real-time) priority. Just be warned, on Windows I once set a frozen process to realtime, and predictably blocked the rest of the system running… and had to restart the machine:

What is the 'realtime' process priority setting for?

I don’t know how to set it programmably there, but I do know how to set priorities in Linux.

I believe the (historical) Linux/Unix priority is relative, not absolute (since a multi-user system), so this wouldn’t happen, with nice (or renice).

Nice is to be, well, nice to others, to lower priority, but you can use a negative value:

ps -eo uid,pid,rtprio,ni,pri,cmd |grep julia

IBM Developer

Niceness generally ranges from -20 to 19, with -20 being the most favorable

Absolute Priority (The GNU C Library)

Only a privileged process can change a process’ absolute priority to something other than 0. Only a privileged process or the target process’ owner can change its absolute priority at all.

POSIX requires absolute priority values used with the realtime scheduling policies to be consecutive with a range of at least 32. On Linux, they are 1 through 99. The functions sched_get_priority_max and sched_set_priority_min portably tell you what the range is on a particular system.

That’s for Linux/POSIX, but might work on Windows too? Since Windows claims POSIX compiliant, even before WSL2, what you might also want to try out.

You can monito in the task manager, or equivalently with top (or GUI tool) on Linux, e.g. these columns (some not on by default), from its man page:

  21. PR  --  Priority
      The scheduling priority of the task.  If you see `rt' in this field, it means the task is running under real time scheduling priority.

      Under linux, real time priority is somewhat misleading since traditionally the operating itself was not preemptible.  And while the 2.6 kernel can be made mostly preemptible, it is not always so.

  15. OOMa  --  Out of Memory Adjustment Factor
      The value, ranging from -1000 to +1000, added to the current out of memory score (OOMs) which is then used to determine which task to kill when memory is exhausted.

  16. OOMs  --  Out of Memory Score
      The value, ranging from 0 to +1000, used to select task(s) to kill when memory is exhausted.  Zero translates to `never kill' whereas 1000 means `always kill'.

I did also find at a former colleague (but not elsewhere, so that manual may be outdated…):
https://elias.rhi.hi.is/libc/Traditional-Scheduling-Intro.html