Is python pandas faster than julia CSV?

General Usage
1

I have a CSV file which is 40.0kb. I tried both in pandas(python3.7) and CSV(julia1.4.2). I found python pandas is much faster than Julia CSV. however, Julia is faster than python in every case. why reading CSV is slower in Julia?

julia code:

using CSV
@time df=CSV.read("C:/Users/hafez/personal/r/tutorial for students/Book2.csv")

6.203511 seconds (8.55 M allocations: 409.590 MiB, 1.29% gc time)

python code:

import pandas as pd
import time
start=time.time()
df=pd.read_csv("C:/Users/hafez/personal/r/tutorial for students/Book2.csv")
print(time.time()-start)

0.029919862747192383

2

it’s not slower
https://juliacomputing.com/blog/2020/06/22/fast-csv.html

1 Like
3

Presumably that time includes compilation time, i.e. the time required to compile the Julia package when it runs for the first time. If you try reading the same file again you should see that it is much faster on the second and subsequent times.

2 Likes
4

yeah , you got it, every first time , it takes so much time , when I loaded and run second time it is much faster than pandas.