How to run HTTP Post request as background task?

General Usage Performance
,
1

Hi everybody,

I am working on a wrapper for the Seq logger for Julia (see WIP project)
Using the REST API for posting raw events , it is possible to post log events to the Seq logging server.

My current implementation sends a POST request after every log event (i.e. @debug, @info, @warn, or @error).

The request takes some time to finish, therefoe, I’d like to run these requests as a background task.

Currently, I use the following approach

Threads.@spawn begin
     HTTP.request("POST", url, header, event)
end

which spawns a new task without waiting for the result. Performance-wise this works quite well, but it is hacky since I don’t wait for a potential error message if the POST request fails.

Is it possible to run this post request as a proper background task without blocking the execution of the other code but still receiving the return value of the request?

2

Create a pool of threads that post to the server. These post workers should read the post information from a channel. They can write the response if it is a failure to another channel which is read by a response worker task that handles the failure. The original code now writes the post information to the channel. Adjust channel and pool sizes so that nothing blocks.

1 Like
3

Thanks for your reply! Do you have any example code on how to create that thread pool and how to read and write to the channels?

Another solution I found is the package WorkerUtilities where you could call it like this, I guess:

worker = WorkerUtilities.@spawn begin
     HTTP.request("POST", url, header, event)
end
response = fetch(worker)
4

The manual has examples here Asynchronous Programming · The Julia Language. Since your HTTP.Post is not cpu bound, your workers can probably be async tasks rather than threads.

1 Like