Efficiency of channels?

General Usage
1

I’ve been attempting to use channels to implement a discrete time simulation and in my simple test code i have found that they seem to incur a lot of overhead.

just moving a single value into the producer channel and out of a consumer channel, 1 value at a time, at 10^5 iterations takes, about 0.3 s. i know this is an arbitrary number, but as a point of comparison the calculation that i would be performing on the accumulated values, adds 0.07s to that run-time.

I had sort of assumed that “under the hood” the channels would simply be some sort of array/buffer and an index, and so there would be very little over head. I’ve read through the documentation, and done the obligatory web search, and haven’t found any documentation pertaining to efficiency.

before i go re-inventing any wheels I’m wondering if there’s
1 a way to improve the efficiency of the channels
2 another data type in Julia that would be more suitable for what Im doing

Thanks!

2

Channels are used for asynchronous I/O etcetera, and so they have to go through libuv’s event loop. This adds a lot of overhead.

You shouldn’t normally use channels if you just want to pass data from one function to another. Just compose the functions directly.

4 Likes
3

ok.

the reason i was attempting to use channels was because eventually i’ll be operating on data coming in over UDP.

the idea was that the processing could stall until data was available.

however it occurs to me that i’m much better of handling that by having something that simply kicks off the calculations when enough data is present.

then i just have the one async interface point and the rest of the calculation can just proceed as normal.

thanks.

4

Right, or you could either @schedule a task to listen for UDP, blocking when there are no packets (though actually on the same worker process), or you could set up a separate parallel worker if you wanted to do the listening on a separate worker.