how costly it is to use this channel-based instant-scheduling mechanism in comparison to an iterator
@christianl I can’t really quantify this. In my case I was parsing a few bytes of JSON at a time and the overhead was a complete show stopper. In may cases the complier can completely inline an iterator method, so it’s hard to beat that. However, as you say, if you’re processing big chunks of data between each task switch, the overhead may be unimportant in your case.
There is one important insight about Julia’s unique capabilities that I realised while working on this stuff is the ability to return multiple values from a function.
Writing stateful data-stream processors in C, you tend to have to store all the state variables away in a struct, and that gets messy, so you end up wanting threads so that you can have a simpler readable function that just uses local variables for state.
In Julia, you can return multiple values from a function and the compiler will still do good optimisation (inlining, passing values in registers, etc). So in my code I have a pattern where the parsing functions take (s, i, c) and return (s, i, c), a string, an index and the current character. The code ends up being terse and easy to read because there is no struct unpacking/packing at the beginning and end of each function and performance is good because there are no mutable structs and no boxing of values and passing by reference.