[ANN] Memoization.jl

I created Memoization.jl to address some of the limitations of Memoize.jl. It’s API compatible with Memoize.jl (just @memoize your function definitions):


julia> using Memoization

julia> @memoize f(x) = (println("Computed $x"); x)

julia> f(2)
Computed 2
2

julia> f(2)
2

As of this writing, Memoization.jl has the following advantages:

  • It allows more function definition forms, including with keyword arguments and type parameters.

  • It does not issue any warnings, and allows you to memoize different methods of the same function across different modules.

  • Closures work, meaning different instances of closures with different closed-over variables can be separately and simultaneously memoized.

  • You can empty memoization caches.

  • Lookup is a bit faster

Feel free to take it for a spin and let me know if you run into any issues!

10 Likes

eh… so you guys can’t agree on how to combine the two into one package?

No, I just made no attempt to combine the packages or submit a PR into that one since its just a 100% rewrite.

4 Likes

I think that remarks like this are unwarranted. It is perfectly fine to develop a new package for something that already exists, especially if it is a complete rewrite.

The package ecosystem benefited a lot from people exploring new ways of doing something. Eventually parallel approaches may get merged, or develop a common API, but it is also OK to just have slightly parallel or overlapping functionality.

18 Likes

Marious311, kindly inform your audience on the information page of your package if your Memoization.jl is thread safe for use in multi-threading julia-1.3

Thank you

Good question, the answer is it is not threadsafe with either the default IdDict nor with Dict, I can update the docs.

If we had a ThreadSafeDict data type (to my knowledge this doesn’t currently exist in Julia, although ThreadSafeDataStructures.jl was on the right path), then @memoize ThreadSafeDict f(x) = ... should already be threadsafe for top-level functions with no changes to this package (and it could made threadsafe for closures too with some minor tweaks).

5 Likes

Have you thought about adding an option to save the content to disk instead to some dictionary?

1 Like

The annoying warnings with precompilation are gone! Thanks!

1 Like

It probably wouldn’t be crazy hard to build some sort of MmappedDict which could be pluggable into this package.

Just to say that there are some more memoization packages listed here: [ANN] Caching.jl, yet another memoizer with more features, and Caching.jl mentions disk support.

1 Like

Yea I think that’d be the right approach. It almost already works with the output of e.g. jldopen which returns a Dict-like thing. I will play with it.

Thanks for pointing that out, I wasn’t aware of either Caching.jl or Anamnesis.jl linked there, both of which have some complimentary features like disk caching which are really nice. One thing which appears to be unique to this package is that it doesn’t change your function’s name or return instead a non-function object, which is what lets this package memoize on a per-method basis.

4 Likes

Great package, thanks for sharing!

2 Likes