Simple way to save several variables

New to Julia
1

Although I have been using Julia for over a year now, I still have problems with basic things, so I consider myself to be a ‘newbie’, so please be gentle.

I would like a simple way for me (and my students) to be able to save a few variables to the same file, and read them in again.

There have been many suggestions on here, and this seems to evolve all the time, but I can’t seem to find a simple answer.

Suppose there are arrays A1 and A2

I would like something like

write(filename, A1, A2)

read(filename)

Or even to be able to read and write a dictionary of variables

dict1=Dict(“A1”=>A1,“A2”=>A2)

write(filename,dict1)

dict2=read(filename)

I find it surprising that after all the time and work which has been devoted
to the Julia project, there is no clear and simple solution to this (or at least
one I could find).
PS. MAT seemed to do what I wanted, but the files are huge (even with compression) and anyway, it doesn’t seem to be supported any more (or at least it doesn’t work for me).

Thanks for any help.

2

If the element types are “simple” (eg Float64, Int), use JSON or HDF5. Both support multiple variables in the same file, hence a Dict can be saved.

Given the complexity of Julia’s type system, I don’t think that there will every be a “simple” solution for the very general case, but the basics are fine.

1 Like
3

Thanks for that. So DataFrames can’t be included, I guess.
What about a collection of DataFrames? I guess that is not possible?

4

No, they can be saved to either of the formats above with a little conversion.

It would be great to have an MWE generating the kind of data you want to save to focus the discussion.

5

What is an MWE? If in my example above, A2 were a DataFrame, for example? That would be an example with as much generality as I would envision. Pickle can handle DataFrames in python. Maybe I am thinking about a Julia equivalent of pickle.
(though I know Julia is not python, some concepts are obviously ‘inspired’ by pythonian ones)

6

A minimal working example that generates your data.

This is still an underspecified problem — a DataFrame can contain arbitrary objects.

7

For specifically DataFrames, you can try also JDF, a DataFrame serialisation format…

8

If the goal is to store information for a short period and you don’t need the file to be human readable, you can use serialize:

julia> using Serialization

julia> d = Dict(3 => 4)
Dict{Int64,Int64} with 1 entry:
  3 => 4

julia> serialize("hello.txt", d)

julia> d2 = deserialize("hello.txt")
Dict{Int64,Int64} with 1 entry:
  3 => 4
3 Likes
9

Hi This looks really cool. Thanks for that!

10

Hi, This looks really cool, too!

Thanks!

11

I concur with the suggestion to use serialize for short-term storage - basically, when you are in principle OK with the file becoming unreadable after updating Julia. To store data in the long-term it’s better to write as a “common” format, not a julia-specific one: csv/JSON/BSON/database/parquet/hdf/… Note that this is also true when using other languages, e.g. python: I myself experienced several times that pickle couldn’t read the data back when some packages got updated.

1 Like