Alternative to DataFrame Readtable to read large data files with headers

Specific Domains Data
9

Hello!

Thank you all for your responses and suggestions. I’m following up with more details about my issues with CSV.read.

Here is a link to an example output file from the stellar evolution code that I mentioned. It has 61 columns and 177 rows of data, not considering all of the header data.

With readtable in DataFrame, which is now depreciated, I was able to read all of the data, and the entire file was imported as a data frame. Here is a screenshot

08%20PM
08%20PM1546×946 569 KB

In contrast, with read in CSV, it takes a noticeably longer time to read the same file, and it doesn’t do so correctly in either Julia 0.7 or Julia 1.0 Here is a screenshot, where the file is read as 177 rows and 141 columns rather than the 61 actual columns. There are also a lot of missing values, which are not actually missing in the file, nor were noted with DataFrame.readtable.

34%20PM
34%20PM1898×452 45.8 KB
I’ll note that I made datarow here 7 rather than skipstart=5 in DataFrame.readtable above because they are defined slightly differently.

I used validate in CSV to get detailed information about why CSV.read failed, and was met with the following error concerning too many columns.

43%20PM
43%20PM1952×710 113 KB

So going back to why I started this thread: How should I efficiently import a file with a large number of columns now that DataFrames.readtable has been depreciated? Is there a way to make CSV.read work for my data files? Is there a different package I should use?