JuliaDB won't open CSV file

General Usage
1

I have been using JuliaDB across my various OSX, Linux, and Windows computer to load data for a project. I am working off of my Linux computer currently, but JuliaDB doesn’t seem to want to open a single CSV file. It is stuck at the loadtable function.

The CSV files are barely 3.0 MB. Is there a way to correct this problem?

2

Hi @alphasig329, you might want to take a look at this thread and follow the guidelines. In particular, post MWE code, and share the error that you’re getting. Diagnosing CSV problems can also be tricky without seeing the file itself, since there are a lot of non-standard files that people call “csv”.

In the mean time, have you tried reading it with CSV.jl or CSVFiles.jl?

3

Yeah sorry! I am not getting an error to share because it is will not execute. Here is the sample code.

using CSV
using JuliaDB
using JuliaDBMeta
using IndexedTables

csv_files = filter(x -> occursin(r"^(IC|ic)20[0-1][0-9]_(ay|AY)(_rv)?.csv$", x) == true, readdir())
data_table = loadtable(csv_files[1], delim = ',', header_exists = true)

At the last line, the loadtable function will not load the data.

I will switch to CSV or CSVFiles.

Thanks!

4

Will not load in what way? It hangs? It could be that it’s just taking a really long time (if the file has many columns, speed is a known problem). Try making a file with just the first 3 or 4 rows and see if that hangs too.

Also, this is a quibble, but the filter line and all of the using ... other than JuliaDB are not (I’m guessing) necessary components of the MWE. If you pull out whatever the path is from csv_files[1] and just do

using JuliaDB

data_table = loadtable("whatever/path/file.csv", delim = ',', header_exists = true)

Do you have the same problem?

5

I apologize for including extra lines in the MWE. Yes the code hangs at the loadtable function. The file works when using CSV.read when calling using CSV. I am just trying to deduce when running this code. The CSV has 175 columns with mixed data types. JuliaDB works on other data import project which uses .txt files.

6

175 columns doesn’t seem like that much, but the mix of data types might be an issue. If you’re willing to share the file, I might open an issue with JuliaDB, and mention that CSV.jl loads it fine.

Another thing you could try is to load it and then save it again with CSV.jl. if there’s some weird formatting thing that CSV handles gracefully but trips up JuliaDB, this might solve it.

7

Also, no need to apologize! I know it’s a bit pedantic, especially in your case where there’s so little code it really didn’t make a difference. But it’s worth practicing how to write MWE’s, because for more complicated problems all the other stuff can really be a distraction and make it harder for folks to help.

1 Like