# Skipping a lot of lines in CSV.read() allocates too much memory

**URL:** <https://discourse.julialang.org/t/skipping-a-lot-of-lines-in-csv-read-allocates-too-much-memory/110121>\
**Category:** Performance\
**Tags:** csv, io\
**Created:** [February 12, 2024, 9:52pm UTC](https://discourse.julialang.org/t/skipping-a-lot-of-lines-in-csv-read-allocates-too-much-memory/110121 "2024-02-12T21:52:47Z")\
**Posts on this page:** 1\
**Showing post:** 57

<div class="post-metadata">

**Author:** ![aris](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@aris](https://discourse.julialang.org/u/aris)\
**Post date:** [February 18, 2024, 2:16pm UTC](https://discourse.julialang.org/t/skipping-a-lot-of-lines-in-csv-read-allocates-too-much-memory/110121/57 "2024-02-18T14:16:58Z")

</div>

> [@rafael.guerra](#):
>
> Could you post the final code using Steffen’s solution?

@rafael.guerra Here is the code I ended up using (which I corrected slightly from the [original post](https://discourse.julialang.org/t/skipping-a-lot-of-lines-in-csv-read-allocates-too-much-memory/110121/46)).

```julia
using CSV, DataFrames

open(filepath) do f
    while !eof(f)
        readuntil(f, "ITEM: BOX BOUNDS")
        boxsize = CSV.read(IOBuffer(readuntil(f, "ITEM: ATOMS")), DataFrame, header=false, skipto=2)
        positions = CSV.read(IOBuffer(readuntil(f, "ITEM: ")), DataFrame, header=false, skipto=2)

    end
end

```

@djholiver Here is a [link to a file using the lammps format](https://livemanchesterac-my.sharepoint.com/:u:/g/personal/aristarchos_mavridis_postgrad_manchester_ac_uk/EcYDPqZGnp1Ihl0LTslxb-IBKG7dK7ZLsXM4U3CO9nYpow?e=7TBRbp). For future reference, this is a OneDrive link and it will expire in one month if I’m not mistaken, sorry but I don’t know if I could keep the link active forever.

> [@djholiver](#):
>
> comment on the ecosystem

As for this comment, that’s just genuine appreciation for everyone who provides me with the tools to do my job in an efficient way, for free. I wouldn’t dare ask for something that’s more specific to the lammps format, I think what’s available already would be more than enough.  
This is why I made the original post using a generic MRE ([here’s the general solution](https://discourse.julialang.org/t/skipping-a-lot-of-lines-in-csv-read-allocates-too-much-memory/110121/34), for anyone who’s lost with all the posts in this thread), to find something that can be used by anyone with a similar problem.

Nevertheless, I really do appreciate how you guys offer to help with my specific example.

---

_[View the full topic](https://discourse.julialang.org/t/skipping-a-lot-of-lines-in-csv-read-allocates-too-much-memory/110121)._