Aah just realized tkf has not been around for a while, @yuyichao (sorry for the tag just thought of you), do you have an idea on this with your multi-threading experience?
Let’s consider that processing a line takes much longer than reading it, and that we cannot assume users will have NVMEs or SSD, so we can only use a single thread for reading.
I’m okay with the Channel but as I said above overloading the number of threads will suddenly degrade performance and I cannot really expect everyone to try to figure out the best number of threads to match their IO. ConcurrentCollections does not have that problem but it will flood memory as soon as there isn’t enough “CPU power” to keep up with reading