# Issues reading big CSV file despite using CSV.Row
**URL:** https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395
**Category:** Data
**Tags:** memory, csv, rcall
**Created:** [July 18, 2022, 9:54am UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395 "2022-07-18T09:54:46Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@candidaorelmex](https://discourse.julialang.org/u/candidaorelmex)
#### Post date: [July 18, 2022, 9:54am UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/1 "2022-07-18T09:54:46Z")
I’d like to import a .tsv file which `CSV.read(file, DataFrame)` doesn’t like.
The error I get is
> ArgumentError: length argument to Parsers.PosLen (204996615) is too large; max length allowed is 1048575
I found an alternative that should work
```julia
csv_rows_file = CSV.Rows(file_path) # load file path
df = DataFrame([[] for i in csv_rows_file.names], csv_rows_file.names) # create a df with correct column names
for row in csv_rows_file # iterate over every row
push!(a, [row[i] for i in 1:ncol(a)]) # add new row by turning CSV.Row2 object values into a vector and pushing it into the df
end
```
I still get the same error!
> ArgumentError: length argument to Parsers.PosLen (204996615) is too large; max length allowed is 1048575
Shouldn’t my approach circumvent the memory issue? Is there a workaround for this?
Also, the limit of 1048575 seems quite low…
Thanks in advance for your help!
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [July 18, 2022, 9:59am UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/2 "2022-07-18T09:59:04Z")
This is not a memory issue, but an issue with strings in your file being too long, see:
[https://github.com/JuliaData/CSV.jl/issues/1009](https://github.com/JuliaData/CSV.jl/issues/1009)
---
### Author:  [@candidaorelmex](https://discourse.julialang.org/u/candidaorelmex)
#### Post date: [July 18, 2022, 10:08am UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/3 "2022-07-18T10:08:19Z")
I see.
If I understand correctly: at a given row, one of the string values has a length of 204996615 which is too long to be imported?
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [July 18, 2022, 10:11am UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/4 "2022-07-18T10:11:53Z")
Yes, currently only strings up to c. 100k characters are supported. You have a string which has roughly 205 million (!) characters.
If you didn’t expect this maybe the delimiter has been incorrectly identified by CSV.jl - I believe the first 10 rows are used to figure out the columns and the delimiter, so if your file has some other information in the first rows this might fail. Try setting the `delim` kwarg explicitly to see if that helps.
---
### Author:  [@bkamins](https://discourse.julialang.org/u/bkamins)
#### Post date: [July 18, 2022, 10:17am UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/5 "2022-07-18T10:17:53Z")
CC @quinnj
---
### Author:  [@candidaorelmex](https://discourse.julialang.org/u/candidaorelmex)
#### Post date: [July 18, 2022, 10:24am UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/6 "2022-07-18T10:24:16Z")
I didn’t scroll through each row but I’ve worked with this software’s (the one Im using to generate the tables Id like to look at) output tables quite a lot, 205million sound like 204.9999 million too much per cell…
Changing the delimiter didn’t work.
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [July 18, 2022, 10:57am UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/7 "2022-07-18T10:57:34Z")
You could also try limiting the number of rows you’re reading in like `CSV.read(file, DataFrame; limit = 10)` to first of all make sure it’s working for parts of the file and then hone in on the row where it fails and inspect that further.
---
### Author:  [@candidaorelmex](https://discourse.julialang.org/u/candidaorelmex)
#### Post date: [July 18, 2022, 12:00pm UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/8 "2022-07-18T12:00:56Z")
I tried that, the delimiter is correctly recognized by `CSV.read`. When I try to only import 1 column
```julia
csv_rows_file = CSV.read(file_path, DataFrame; select = [:Peptide])
```
which contains a maximum of 25 characters per cell I still get the error
> ArgumentError: length argument to Parsers.PosLen (204996615) is too large; max length allowed is 1048575
Does that mean there’s a bug?
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [July 18, 2022, 12:09pm UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/9 "2022-07-18T12:09:35Z")
My suggestion was to read in only the first few rows, not just one column - what do you get from that?
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [July 18, 2022, 12:23pm UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/10 "2022-07-18T12:23:43Z")
My guess is the most likely problem is that you have a string with a missing closing quotation mark. So hunt for that.
---
### Author:  [@candidaorelmex](https://discourse.julialang.org/u/candidaorelmex)
#### Post date: [July 18, 2022, 12:43pm UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/11 "2022-07-18T12:43:07Z")
That’s what I did. it worked just fine. Rows can be read up to row ~2.1 million, then the error pops up
---
### Author:  [@candidaorelmex](https://discourse.julialang.org/u/candidaorelmex)
#### Post date: [July 18, 2022, 12:45pm UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/12 "2022-07-18T12:45:27Z")
I get the same error when I ready just 1 column with floating point values, I doubt it’s an issue of quotation marks.
I managed to import the table with R’s `read.delim` function.
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [July 18, 2022, 12:56pm UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/13 "2022-07-18T12:56:04Z")
CSV still has to know where the delimeters are. If there is a missing ending quotation, it wouldn’t be able to keep track of columns enough to just read the one you selected.
---
### Author:  [@candidaorelmex](https://discourse.julialang.org/u/candidaorelmex)
#### Post date: [July 18, 2022, 1:13pm UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/14 "2022-07-18T13:13:15Z")
The software that generated the table has never shown an issue of generating a file where a string would have misplaced quotation marks.
R can import the file just fine. I counted the number of characters for each column and plotted them on histograms.
36 columns, so 36 histograms. none of them had any values beyond 100.
Unluess `read.delim` in R does some magic to randomly insert quotation marks we can exclude that possibility - good suggestion though!
---
### Author:  [@candidaorelmex](https://discourse.julialang.org/u/candidaorelmex)
#### Post date: [July 18, 2022, 1:48pm UTC](https://discourse.julialang.org/t/issues-reading-big-csv-file-despite-using-csv-row/84395/15 "2022-07-18T13:48:11Z")
I found a workaround through R, in case someone runs into the same problem:
```julia
using RCall
x = reval("read.delim('/folder/file_path')")
df = rcopy(x)
```
gives a dataframe as expected.
(substitute “/folder/file\_path” with your file’s file location. dont forget the ’ '!)