How to save a data frame to file in Julia 1.8?

New to Julia
1

Hello,
I used to save a dataframe to a tsv file with

using CSV, DataFrames
CSV.write("./myFile.tsv", df;
    header=false, delim='\t', append=false, decimal: '.', quotechar: '"')

where df is the object containing the data frame.
However, I now get the error:

julia> CSV.write("./myFile.tsv", df;
           header=false, delim='\t', append=false, decimal: '.', quotechar: '"')
ERROR: syntax: invalid keyword argument syntax "decimal:Char(0x2e000000)"
Stacktrace:
 [1] top-level scope
   @ none:1

If I shorten the syntax, it is even worse:

julia> CSV.write("./myFile.tsv", df;
           header=false, delim='\t', append=false)
ERROR: UndefVarError: writeshortest not defined
Stacktrace:
 [1] writecell(buf::Vector{UInt8}, pos::Int64, len::Int64, io::IOStream, x::Float64, opts::CSV.Options{UInt8, UInt8, Nothing, Tuple{}, typeof(CSV._identity)})
   @ CSV ~/.julia/packages/CSV/jFiCn/src/write.jl:444
 [2] (::CSV.var"#106#107"{Vector{UInt8}, Base.RefValue{Int64}, Int64, IOStream, Int64, CSV.Options{UInt8, UInt8, Nothing, Tuple{}, typeof(CSV._identity)}, UInt8, UInt8})(val::Float64, col::Int64, nm::Symbol)
   @ CSV ~/.julia/packages/CSV/jFiCn/src/write.jl:371
 [3] eachcolumn
   @ ~/.julia/packages/Tables/T7rHm/src/utils.jl:70 [inlined]
 [4] writerow(buf::Vector{UInt8}, pos::Base.RefValue{Int64}, len::Int64, io::IOStream, sch::Tables.Schema{(:timestamp, :value1, :value2, :value3), NTuple{4, Float64}}, row::DataFrameRow{DataFrame, DataFrames.Index}, cols::Int64, opts::CSV.Options{UInt8, UInt8, Nothing, Tuple{}, typeof(CSV._identity)})
   @ CSV ~/.julia/packages/CSV/jFiCn/src/write.jl:367
 [5] (::CSV.var"#99#100"{Bool, Bool, Tables.Schema{(:timestamp, :value1, :value2, :value3), NTuple{4, Float64}}, DataFrames.DataFrameRows{DataFrame}, CSV.Options{UInt8, UInt8, Nothing, Tuple{}, typeof(CSV._identity)}, Vector{UInt8}, Int64, Int64, NTuple{4, Symbol}})(io::IOStream)
   @ CSV ~/.julia/packages/CSV/jFiCn/src/write.jl:235
 [6] with(f::CSV.var"#99#100"{Bool, Bool, Tables.Schema{(:timestamp, :value1, :value2, :value3), NTuple{4, Float64}}, DataFrames.DataFrameRows{DataFrame}, CSV.Options{UInt8, UInt8, Nothing, Tuple{}, typeof(CSV._identity)}, Vector{UInt8}, Int64, Int64, NTuple{4, Symbol}}, io::Any, append::Bool, compress::Bool)
   @ CSV ~/.julia/packages/CSV/jFiCn/src/write.jl:303
 [7] #write#98
   @ ~/.julia/packages/CSV/jFiCn/src/write.jl:225 [inlined]
 [8] write(file::String, itr::DataFrame; delim::Char, quotechar::Char, openquotechar::Nothing, closequotechar::Nothing, escapechar::Char, newline::Char, decimal::Char, dateformat::Nothing, quotestrings::Bool, missingstring::String, transform::typeof(CSV._identity), bom::Bool, append::Bool, compress::Bool, writeheader::Nothing, partition::Bool, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:header,), Tuple{Bool}}})
   @ CSV ~/.julia/packages/CSV/jFiCn/src/write.jl:210
 [9] top-level scope
   @ none:1

What is the current syntax to save a data frame to file?
Thank you

2

is wrong, you should write

decimal='.'

which is really what the error message is telling you.

2 Likes
3

Ops, thank you!

4

No worries. It was useful that you provided enough information in your post.

1 Like
5

Actually, I am still getting the error, in fact there was the error even when omitting decimal…
I have seen from this post, that the error I am getting might be one of global definition. This is strange, since the dataframe is defined in the main script not inside functions. Even if I set global df the error persists…

6

Can you please indicate the version of CSV.jl and Parsers.jl you have. (also: please update the packages to the newest releases and re-check if things are broken)

7

I got:

julia> Pkg.status("CSV")
Status `~/.julia/environments/v1.8/Project.toml`
  [336ed68f] CSV v0.10.8

julia> Pkg.status("DataFrames")
Status `~/.julia/environments/v1.8/Project.toml`
⌃ [a93c6f00] DataFrames v1.3.6
Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> Pkg.status("Parsers")
No Matches in `~/.julia/environments/v1.8/Project.toml`

I added Parsers, but I could not upgrade DataFrames (I already used Pkg.up but apparently was not enough):

julia> add DataFrames@1.3.6
ERROR: syntax: extra token "DataFrames" after end of expression

julia> Pkg.add DataFrames@1.3.6
ERROR: syntax: extra token "DataFrames" after end of expression

julia> Pkg.update("DataFrames")
    Updating registry at `~/.julia/registries/General.toml`
  No Changes to `~/.julia/environments/v1.8/Project.toml`
  No Changes to `~/.julia/environments/v1.8/Manifest.toml`
8

DataFrames.jl is irrelevant for this issue (however, the fact that you have it in version 1.3.6 shows that you use some non-standard package that is blocking its update, and also might block CSV.jl and Parsers.jl).

To see the version of Parsers.jl use -m flag:

(@v1.8) pkg> status -m Parsers
Status `~\.julia\environments\v1.8\Manifest.toml`
  [69de0a69] Parsers v2.5.2

This is the version you should have (if you have older this will be the reason of your problems).

To see what is the offending package run status --outdated.

1 Like
9

I got:

julia> Pkg.status("Parsers")
Status `~/.julia/environments/v1.8/Project.toml`
  [69de0a69] Parsers v2.5.2

If I load Parsers, it looks like it is working now. Thank you