Save data frame with special characters in csv

The column names of my dataframe has special character like "€. Before updating to v1.3, I was able to write my df in a csv file and keeping the different symbols. However, after updating the csv file doesn’t recognize these symbols and I got a result like that:

image

I can reproduce this problem easily as follow:

using DataFrames, CSV

df = DataFrame([:TypeProduct => [1,2,3], Symbol("Coût - €/l") => [4,5,6]])

CSV.write("df.csv", df;delim=";")

Anyone knows how I can handle this problem?
Thanks.

Works fine for me in both 1.3 and 1.4:

julia> using DataFrames, CSV

julia> df = DataFrame([:TypeProduct => [1,2,3], Symbol("Coût - €/l") => [4,5,6]])
3×2 DataFrame
│ Row │ TypeProduct │ Coût - €/l │
│     │ Int64       │ Int64      │
├─────┼─────────────┼────────────┤
│ 1   │ 1           │ 4          │
│ 2   │ 2           │ 5          │
│ 3   │ 3           │ 6          │

julia> CSV.write("/tmp/df.csv", df;delim=";")
"/tmp/df.csv"

shell> cat /tmp/df.csv
TypeProduct;Coût - €/l
1;4
2;5
3;6

(@v1) pkg> st CSV DataFrames
Status `~/.julia/environments/v1/Project.toml`
  [336ed68f] CSV v0.5.22
  [a93c6f00] DataFrames v0.20.0

Are you sure that the software you are reading it with handles UTF8?

OP, it looks like you are using excel to view the files. Excel has pretty notorious problems with displaying unicode characters (and saving them wrong!).

Best to use a text editor to view the .csv file if it’s not too big.

The BOM Header is missing in the .csv file.
Try the following:

using DataFrames, CSV
df = DataFrame([:TypeProduct => [1,2,3], Symbol("Coût - €/l") => [4,5,6]])
CSV.write("df.csv", df;delim=";")

inStream = open("df.csv","r")
outStream = open("df_bom.csv","w")
print(outStream,"\xEF\xBB\xBF")  #UTF-8 BOM header
print(outStream, read(inStream,String))
close(outStream)
close(inStream)
1 Like

Opened an issue:
https://github.com/JuliaData/CSV.jl/issues/567

1 Like

Thanks a lot