Is there a way to write a function into JLD or other h5 file?

General Usage
1

Wondering if there is a way to write a defined function into file.
A MWE is

using JLD2
f(x) = x^2
@save "f.jld2" f

For this case, if I close the current session and execute the following code

using JLD2
@load "f.jld2" f

I will always get

┌ Warning: type Main.#f does not exist in workspace; reconstructing
└ @ JLD2 ~/.julia/packages/JLD2/k9Gt0/src/data/reconstructing_datatypes.jl:369
1 Like
2

I know BSON works if you make your function an anonymous function:

using BSON
f = x -> x^2
@save "f.bson" f
# in new session
BSON.@load "f.bson" f
f(3) # 9

although it can’t have keyword arguments. I don’t think JLD2 works with any unfortunately, but maybe its in the works.

1 Like
3

You’re right. Currently it is not possible.
The PR you mentioned implemented a working draft at some point. However, internals of functions are complicated, and change between julia versions. e.g. significant changes between 1.6 (current LTS) and 1.7.

Additionally, the julia Serialization stdlib is good at this.
This PR tests whether JLD2 could, in the future, use the julia serializer under the hood to store objects such as functions. This has a few other potential problems and an API would need to be worked out.

1 Like
4

I’ve been struggling with saving and loading anonymous functions recently and haven’t found a nice solution, so thank you for highlighting this ability of BSON.jl.

If I save an anonymous function using BSON in - say Julia 1.2 - and then load it again in Julia 1.7 or 1.8, will that be successful? My understanding was that the internals can change without warning and therefore I might not be able to do this. I have a package-internal storage format that strictly uses only raw JSON (for compatibility concerns) so it might be impossible for me to switch, but I can at least learn from how BSON does it.

What I was really trying to do, and I’d love to get advise on what the right approach is here:

struct AnonF{FF}
    f::FF # the actual function 
    str::String
end

(F::AnonF)(x) = F.f(x)

where str stores a string (or may better expression??) specifying the function. (I will always want to wrap the function into a callable object and give it extra functionality, so that’s a separate issue.) Then I’d like to initialize it via

AnonF("x -> x^2")
# or 
AnonF( :( x -> x^2) )

and now I’m looking for functionality to write to JSON or YAML and load it again. This must work robustly and long-term across Julia versions and versions of those packages, and ideally human-readable.

Can this be achieved? If I ignore the str “meta-data” then I can generate a function from a expression like this

macro analytic(fexpr)
          quote
             $(esc(fexpr))
          end
       end

and I thought it would be possible to return at the same time a string of the expression, but I didn’t manage to do that?

5

Hi @cortner ,
the “right” version to do this is to use the Serialization stdlib. BSON does exactly the same. (Most of the relevant code is copied).
If you want to embed serialized functions in some format, build a small wrapper so that you can use Serialization.serialize to convert it into a binary blob.

I made an experimental PR in JLD2 to test this https://github.com/JuliaIO/JLD2.jl/pull/377

6

I see - so basically don’t bother with the analytic expression, just serialize the anonymous function in binary. And will this be compatible across Julia versions?

7

little is promised but if i remember correctly, serialized functions all the way from 1.3 (or so) should still be readable.

https://github.com/JuliaLang/julia/blob/master/stdlib/Serialization/src/Serialization.jl

8

beautiful - thank you! Just one question: If I load a serialized function from file then it assigns the same number to the resulting anon function as when it was originally serialized. E.g. if I then load it and a #1 was already defined, it creates another one. This seems weird to me, but I have no idea at all whether I should worry about it??

julia> f = r -> exp(r)
#1 (generic function with 1 method)

julia> using Serialization

julia> fname = "temp.dat"
"temp.dat"

julia> g = deserialize(fname)
#1 (generic function with 1 method)
9

It seems here might be an alternative answer to my question: https://github.com/MasonProtter/LegibleLambdas.jl/

… or at least partially. I still don’t understand how I can write the expression to a file as a string, load it, parse it and then create a new “legible lambda”.

EDIT: any concerns with this approach?

function write_dict(t::LegibleLambda) 
   buf = IOBuffer()
   show(buf, t)
   return Dict(
         "ex" => "$(t.ex)",             # f is reconstructed from ex 
         "meta" => String(take!(buf))   # this will be ignored 
      )
end

function LegibleLambda(D::Dict)
   ex = Meta.parse(D["ex"])
   return LegibleLambda(ex, eval(ex))
end