I have a package that both collects data and analyzes the data. These parts are separated, with the data acquisition function periodically collecting data and writing this data to a file. The analysis function looks for a new data file, and when it exists, then it reads and processes the data file. Both functions run continuously, with the acquire function running in a separate thread using the @spawn command.
These tasks are run on separate threads using the following code:
function main()::Cint
try
#separate thread for acquiring data
Threads.@spawn acquire()
# main thread to process the files acquired
autorun()
catch
Base.invokelatest(Base.display_error, Base.catch_stack())
return 1
end
return 0
end
The function is written with the intent to use it for the PackageCompiler, but that has not been tested yet.
The machine is a PI 4 with 8 GB of RAM. The acquire portion only takes a couple percent of the CPU. The main work of acquisition is performed by PI HATS and the main bottleneck in this is reading the data from the HAT and writing it to a file, in my case an Arrow file. If this fails, then there is a buffer overrun error message.
With some testing, the acquire function running on its own works well. However when it is acquiring data at the same time that the PI is processing another data file, I can get the buffer overrun error and in one case the program hung.
Linux has a way of setting process priority, is there a way I can do this in Julia for tasks? Or is there something else that I am missing?
