Changing strings at the beginning of a large file

Specific Domains Data
10

If you have the space you need for rearranging the header line that’s fine. Otherwise there’s a big difference between reading everything at once into memory or just a line at a time. But if you want to improve on that I would do

open(outfilename, "w") do output
    open(infilename, "r") do input
        header = readline(input, keep = true)
        # ...transform header...
        write(output, header)
        while !eof(input)
            write(output, read(input, blocksize))
        end
    end
end

where you can tune blocksize depending on how much memory you want to handle at once and the characteristics of your file system.

Personally I wouldn’t bother with doing an in-place header change unless the file was gigantic, but in that case I just wouldn’t change it at all. Things like a column name with unexpected space padding will always come back to bite you at a later time.

1 Like