Skipping a lot of lines in CSV.read() allocates too much memory

General Usage Performance
,
34

Update:
I reckon this is the fastest solution I found so far;

using CSV, DataFrames

number_of_lines = 10^6
CSV.write("data.csv", DataFrame(rand(number_of_lines, 10), :auto))

steps = 10
chunk = round(Int, number_of_lines / steps)

# Preallocate arrays
raw_strings::Vector{String} = [" " for _ in 1:chunk]

open("data.csv") do io
    readline(io) #skip header

    for _ in 1:steps

        @time for i in 1:chunk
            raw_strings[i] = readline(io)
        end

        # split strings and convert to numbers or dataframe
        # do whatever you want with the numbers 
        # move to next block

    end
end

Again, thank you everyone for your inputs.