I have a local service that uses HTTP.jl to handle simple API requests running on the main task and then a background task (started with Threads.@spawn) that periodically loads data from a SQL database, does heavy computations using DataFrames and several optimization packages, and updates the data to be served by the API.
This generally works well, with HTTP response times (including some lightweight calculations ranging from 5-10 ms. But at a certain point when the heavy background calculations are being done, the response time will spike to 5-10 seconds (a 1000x increase). I am guess that this is because some of the packages like DataFrames will automatically try to run multiple tasks, which could use up all Julia threads and block the main HTTP task.
My question is how can I pin the HTTP task to a thread and prevent any other tasks from running on that thread?