# Julia is unable to load CSV files from the Kaggle competition

**URL:** <https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830>\
**Category:** Performance\
**Tags:** question, package, csv\
**Created:** [August 2, 2018, 4:34pm UTC](https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830 "2018-08-02T16:34:18Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Altaf\_Sultanji](https://avatars.discourse-cdn.com/v4/letter/a/8797f3/32.png) [@Altaf\_Sultanji](https://discourse.julialang.org/u/Altaf_Sultanji)\
**Post date:** [August 2, 2018, 4:34pm UTC](https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830/1 "2018-08-02T16:34:18Z")

</div>

I am using Julia (v0.6.4) to load csv files. These files can be found at this link:

> **[Santander Value Prediction Challenge](https://www.kaggle.com/c/santander-value-prediction-challenge/data)**
>
> Predict the value of transactions for potential customers.

Julia is unable to load them (train.csv and test.csv). I tried to use the CSV package with method CSV.read.

While in Python using pandas it load in no time. All all working perfectly.

It appears there is a serious issue in parsing/loading of these files in Julia.

Did anyone try and succeed or have answer/tip to make this work ?

Thanks.

---

<div class="post-metadata">

**Author:** ![favba](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/favba/32/2735_2.png) [@favba](https://discourse.julialang.org/u/favba)\
**Post date:** [August 2, 2018, 4:36pm UTC](https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830/2 "2018-08-02T16:36:46Z")

</div>

I just google and it seems there is a Pandas wrapper for julia: [https://github.com/JuliaPy/Pandas.jl](https://github.com/JuliaPy/Pandas.jl)  
Maybe that will work for you?

---

<div class="post-metadata">

**Author:** ![John\_Gibson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/john_gibson/32/5321_2.png) [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)\
**Post date:** [August 2, 2018, 5:45pm UTC](https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830/3 "2018-08-02T17:45:30Z")

</div>

Works for me on some simple test data.

```julia
julia> using CSV

julia> d = CSV.read("benchmarkdata.csv", header=false)
10×3 DataFrames.DataFrame
│ Row │ Column1 │ Column2 │ Column3 │
├─────┼─────────┼─────────────────────┼──────────┤
│ 1 │ c │ iteration_pi_sum │ 27.369 │
│ 2 │ c │ matrix_multiply │ 72.068 │
│ 3 │ c │ matrix_statistics │ 4.52399 │
│ 4 │ c │ parse_integers │ 0.099092 │
│ 5 │ c │ print_to_file │ 9.93013 │
│ 6 │ c │ recursion_fibonacci │ 0.022726 │
│ 7 │ c │ recursion_quicksort │ 0.258923 │
│ 8 │ c │ userfunc_mandelbrot │ 0.07669 │
│ 9 │ fortran │ iteration_pi_sum │ 27.3692 │
│ 10 │ fortran │ matrix_multiply │ 83.5437 │

julia> versioninfo()
Julia Version 0.6.4
Commit 9d11f62bcb (2018-07-09 19:09 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-3960X CPU @ 3.30GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge MAX_THREADS=16)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, sandybridge)

```

```julia
gibson@sophist$ cat benchmarkdata.csv 
c,iteration_pi_sum,27.369022
c,matrix_multiply,72.067976
c,matrix_statistics,4.523993
c,parse_integers,0.099092
c,print_to_file,9.930134
c,recursion_fibonacci,0.022726
c,recursion_quicksort,0.258923
c,userfunc_mandelbrot,0.07669
fortran,iteration_pi_sum,27.369179
fortran,matrix_multiply,83.543703

```

You should post exactly what you tried in Julia and the resulting error message or incorrect output. Use triple backticks to quote the code blocks.

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [August 2, 2018, 5:55pm UTC](https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830/4 "2018-08-02T17:55:35Z")

</div>

You can also try to use [CSVFiles.jl](https://github.com/queryverse/CSVFiles.jl), it uses a different parser under the hood, so if you are lucky, it might be able to deal with those files. Syntax would be:

```julia
using CSVFiles, DataFrames

df = load("foo.csv") |> DataFrame

```

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [August 3, 2018, 5:25am UTC](https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830/5 "2018-08-03T05:25:12Z")

</div>

> [@Benchmarking ways to write/load DataFrames IndexedTables to disk](https://discourse.julialang.org/t/benchmarking-ways-to-write-load-dataframes-indexedtables-to-disk/8973):
>
> Update 2018-Feb-19: added R feather and Pandas; thanks to @zhangliye for the pandas code For Julia, JLD.jl has the fastest write-solution and I have used it via the ultra-convenient FileIO.jl. However for interop with other packages, the slightly slower Feather.jl is also a good choice, also it may be arguable that you read data more often than you write, so Feather.jl’s superior read-speed will be essential. However, R’s feather is faster than Julia’s. The read and write speed seem to scale …

Check out the post for “inspiration”, I often find that using R’s data.table’s `fread` is the fastest.

---

<div class="post-metadata">

**Author:** ![Altaf\_Sultanji](https://avatars.discourse-cdn.com/v4/letter/a/8797f3/32.png) [@Altaf\_Sultanji](https://discourse.julialang.org/u/Altaf_Sultanji)\
**Post date:** [August 3, 2018, 3:47pm UTC](https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830/6 "2018-08-03T15:47:16Z")

</div>

Thanks. This works. But the same file if I try using CSV.jl or CSVFiles.jl it fails to load. These files can be downloaded from the Kaggle competition website ([Santander Value Prediction Challenge | Kaggle](https://www.kaggle.com/c/santander-value-prediction-challenge/data)).

---

<div class="post-metadata">

**Author:** ![Altaf\_Sultanji](https://avatars.discourse-cdn.com/v4/letter/a/8797f3/32.png) [@Altaf\_Sultanji](https://discourse.julialang.org/u/Altaf_Sultanji)\
**Post date:** [August 3, 2018, 3:47pm UTC](https://discourse.julialang.org/t/julia-is-unable-to-load-csv-files-from-the-kaggle-competition/12830/7 "2018-08-03T15:47:17Z")

</div>

Normally most of the CSV files are loaded using CSV.jl or CSVFiles.jl. In this case I am refering to specific files from the Kaggle competition site which are not loaded by CSV.jl or CSVFiles.jl. These files can be downloaded from the Kaggle competition website ([Santander Value Prediction Challenge | Kaggle](https://www.kaggle.com/c/santander-value-prediction-challenge/data)).  
This clearly means there is a bug in CSV.jl or CSVFiles.jl.