Hi
I would like to use Julia to process large logfiles piped into STDIN, in order to create reports.
The amount of data is too big to fit into RAM, so I need to process every line by line while reading the stream of data (rather than slurping all data into a variable before starting to process data).
Like this:
cat my_huge_logfile.log | reporting.jl
In Perl, I would use something like this:
while(<STDIN>) {
# regex matching on current line
# do some preprocessing
# remember selected data in hash/array
}
# reporting based on hash/array
What is the equivalent or better way do it in Julia?