Memoize Pkg? Bad use of Metaprogramming?

anon92994695
Fri, 18 Sep 2020 20:48:05 +0000
General Usage

Been doing some long-running computations. I mean … long. So I am doing the obvious thing, storing results to disk along the way. For reproducibility reasons I’d like to be able to memoize lots of intermediate results along the way. Even better, if I could show the code to generate the results but not have to re-run it or have it isolated from the execution flow. So I came up with a solution, it’s something like I once did with @kolia a while ago, but I’d like some feedback.

using JLD2
macro memoize(cache_var)
    filename = String(cache_var) * ".jld2"
    if !isfile( filename )
        @eval @save $filename $cache_var
    else
        @eval @load $filename $cache_var
    end
end

macro memoize(cache_var, fn)
    filename = String(cache_var) * ".jld2"
    if !isfile( filename )
        @eval $fn
        @eval @save $filename $cache_var
    else
        @eval @load $filename $cache_var
    end
end

macro free_memo(cache_var)
    filename = String(cache_var) * ".jld2"
    if !isfile( filename )
        @warn "Cache is already empty."
    else
        rm(filename)
    end
end
@memoize a
a

@memoize a begin
    a = 123
    a += 456
    a /= 5
    println("Hey we ran this code!")
end

@free_memo a

Main points:

is there already a package that does this? if so link please?

Bad use of metaprogramming?

Too dangerous to bundle up, correct, and share (incorrect use could cost a user a lot of time or electrons)?

Feel free to dartboard anything else bothersome. The word “cache” will likely upset some people here, and I foresee concerns about scope and where files are stored :D.

15 posts - 7 participants