Issue with RData

General Usage
1

Hello,
I am using julia v0.6.0 on the server at work and having a problem loading RData files using the RData package. I ran the following:

“”"
using RData
ResponseMatrix = load(“Dahaniel-DoOR.data-6436660/data/response.matrix.RData”,convert=true)
“”"

and got the following error message

ERROR: AssertionError: FileIO.detect_rdata(io)

with the following stack trace

Stacktrace:
[1] load(::FileIO.Stream{FileIO.DataFormat{:RData},GZip.GZipStream}, ::Array{Any,1}) at /data1/homes/pee3/.julia/v0.6/RData/src/RData.jl:53
[2] gzopen(::RData.##4#5{Array{Any,1},FileIO.File{FileIO.DataFormat{:RData}}}, ::String) at /data1/homes/pee3/.julia/v0.6/GZip/src/GZip.jl:268
[3] #load#3(::Array{Any,1}, ::Function, ::FileIO.File{FileIO.DataFormat{:RData}}) at /data1/homes/pee3/.julia/v0.6/RData/src/RData.jl:46
[4] (::FileIO.#kw##load)(::Array{Any,1}, ::FileIO.#load, ::FileIO.File{FileIO.DataFormat{:RData}}) at ./:0
[5] #load#13(::Array{Any,1}, ::Function, ::String) at /data1/homes/pee3/.julia/v0.6/FileIO/src/loadsave.jl:52
[6] (::FileIO.#kw##load)(::Array{Any,1}, ::FileIO.#load, ::String) at ./:0

Does anyone know what might be going on? Using other julia packages on the server doesn’t seem to cause problems.
Thanks,
Philip

2

What’s in the data file? Could be objects of a class not supported?

3

Do you know what compression type was used in creation of the RData file? Under Linux I can use

bates@lunchbox:~$ file ~/ml10m.rda
/workspace/bates/ml10m.rda: gzip compressed data, from Unix

to determine that.

R allows you to use different compression types when creating an RData file but we only allow gzip compression when reading with the RData package.

One of the items on my “To Do” list is to create a PR to allow for xz and bzip2 compression.

1 Like
4

By the way, what would the preferred package(s) be to open a file, check for gzip, bzip2 or xz compression, then uncompress using the detected compression scheme?

5

This does not seem to be resolved yet. Today I got similar problems and could not even read a simple .Rdata file.

In R (version 3.5.0) I generate the file with:

> a <- 42
> save(a,file="tmp.Rdata")

and then in Julia (version 0.6.2):

julia> Pkg.status("RData")
 - RData                         0.4.0

julia> using RData
WARNING: Method definition unix2zdt(Real) in module TimeZones at /Users/a/.julia/v0.6/TimeZones/src/conversions.jl:122 overwritten in module RData at /Users/a/.julia/v0.6/RData/src/convert.jl:201.

julia> load("tmp.Rdata")
Error encountered while loading "/Users/a/tmp.Rdata".
Fatal error:
ERROR: UndefVarError: load not defined
Stacktrace:
 [1] #load#27(::Array{Any,1}, ::Function, ::FileIO.File{FileIO.DataFormat{:GZIP}}) at /Users/a/.julia/v0.6/FileIO/src/loadsave.jl:180
 [2] load(::FileIO.File{FileIO.DataFormat{:GZIP}}) at /Users/a/.julia/v0.6/FileIO/src/loadsave.jl:167
 [3] #load#13(::Array{Any,1}, ::Function, ::String) at /Users/a/.julia/v0.6/FileIO/src/loadsave.jl:113
 [4] load(::String) at /Users/a/.julia/v0.6/FileIO/src/loadsave.jl:113

while I can read the file iris.rda, which ships with Rdatasets, without any problem:

julia> load("/Users/a/.julia/v0.6/RDatasets/data/datasets/iris.rda")
Dict{String,Any} with 1 entry:
  "iris" => 150×5 DataFrames.DataFrame…

Any help is appreciated.

6

I don’t know anything better than a long chain of try ... catch blocks, but perhaps you could make a feature request in

7

In my above example both files seem to be properly gzip encoded:

> gzip -t -v /Users/arndt/.julia/v0.6/RDatasets/data/datasets/iris.rda
/Users/a/.julia/v0.6/RDatasets/data/datasets/iris.rda:	  OK
> gzip -t -v tmp.Rdata 
tmp.Rdata:	  OK

And with load I can read one file but not the other.

8

Could you try to use the extension .RData instead of .Rdata?

I use RData.jl a lot, and it works well.

3 Likes
9

Yep - that solved the problem - the two extension which work are: .RData and .rda. E.g.

julia> load("tmp.rda")
Dict{String,Any} with 1 entry:
  "a" => 42.0

Thanks a lot.

3 Likes