Understanding Macros – A cache between functions

New to Julia
1

I am trying to get started with Macros. As a second challenge for me (see the first one here) I try to understand an easy way to cache interims values between functions. Let’s say I have two functions

function f(p,q,s)
  x = p*q; # or something much more challenging, time-consuming
  return (x*s)^2 # something easy
end
function g(p,q,t,v)
  x = p*q; #recompute - again time consuming
  return (x/t)^(1/v) # something easy
end

such that both functions have an interims value xdepending on both p,q (in my case vectors, but for this example let’s say integers or floats). I would like to be able to hook into both functions with a macro to cache x, so maybe this is similar to Memoize.jl for which I couldn’t find a documentation.

What I would like to do is that a call like
@cache :f (:p,:q) f(p,q,s)
stores the computed x in a dictionary (together with let’s say f,p,q or that it was generated by f with parameter values p,q (the first two parameters).
(Note that my notation for macros and calling them as well as specifying f,p,q in the line might be completely of, since I am barely new to macros) in such a way that I can specify
'@cache [:f,:g] (:p,:q) g(p,p,t,v)that can look up whether a value (namelyx) was computed by either forg` (I.e. the square brackets are a collections) by the same parameters with respect to p and q (hence the round brackets for a tuple, but it has to refer to the first two parameters) such that that value can be reused.

Note that

  • my parameters are always ordered that way, so p is first q is second
  • x will always be called x by convention (it is for example a determinant or a matrix from an singular value decomposition but always with a fixed meaning and hence fixed name consistent over both functions
  • Of course if the first call already “finds” a value for (f,p,q) or (g,p,q) it would also reuse that one

Is that possible with macros or am I getting too exited here about macros?

2

Always when writing macros you have to think what you want the input syntax to be and the resulting output syntax. So before you can write out in julia code what you want the macro to expand to, there is no point worrying about the macro.

So say you write

@cache :f (:p,:q) f(p,q,s)

What should the resulting julia code be?

3

Thanks for asking. You’re right, that was not specified enough. My idea is that the cache (or Memoize) has an internal Dictionary
with the tuple (::Function, Array{Any,1}) as key and stores x (maybe that then also has to be specified, you’re right, it can not be magically detected). But I don’t know how to specify a “local variable name” (since I said x is always called x).

But the better form of that line is then (where the :f in my mind refers to the compete function signature, since that identifies f
and [:p,:q] refers to the argument types of p and q as well as their values. For example if f is called with two integers both being p=1,q=1 thats a different x than that from f being called with floats p=1.0,q=1.0. Then for g any result from either :f or :g with exactly the same types and values for p,q identifies the result. So maybe the dictionary key is five values, f,p,q and the types of p and q (again writing types as :q because I don’t know better).

`@cache (:f, [p,q,:p,:q]) f(p,q,s)

and that’S also why the other call (and maybe even the one I just wrote) has f in there, because the first entry of the tuple is allowed to be either f or gs signature.

Edit. So maybe it’s easy to summarise in a sentence: I would like to “hook” into f telling it to store x and its type together with the information that “f (identified by its signature) computed this based on the types and values of p and q” (as a key). Later I would like to be able to ask for that value saying “If a function of signature(s) f or g computed a value of type of x based on values and types p and q – please give me the cached value, otherwise store it (then with the information that g computed x,…”.

4

This might already be possible using https://github.com/rdeits/CommonSubexpressions.jl. At least it could be a source of inspiration.

5

Why not rewrite this to:

function f(p,q,s)
  x = memoized_func(p, q); # or something much more challenging, time-consuming
  return (x*s)^2 # something easy
end
function g(p,q,t,v)
  x = memoized_func(p, q); #recompute - again time consuming
  return (x/t)^(1/v) # something easy
end

So you could either have memoized_func defined with memoize or have something simple like:

cacheresult(f, args...) = get!(()-> f(args...), my_cache, (f, args...))

and then do in f/g

x = cacheresult(*, p, q)
1 Like
6

Your approach should work, I think; I just wanted to maybe find a way without changing the definition of f nor g (maybe to be more flexible, maybe it’s also too “dangerous” from macro hygiene point of view, or maybe then my idea is also too general).

7

This might not be quite relevant but I do have a package Anamnesis.jl which basically does what Memoize.jl with a few extra features. One of those is to memoize function calls within blocks so if you did, for example

@anamnesis function f(p,q,s)
    x = h(p,q)
    (x*s)^2
end h

this will automatically make h a memoized function. The idea was to make it possible to insert function memoization without actually changing any code.

4 Likes
8

That looks quite nice, I’ll have to look at that a little closer.
For example, whether the storage “remembers” h,p,q for the value x that’s stored, so that the value is only reused if those three match – though I am also looking for a way to do that without storing p,q explicitly (they might be kind of large).

But your idea is the same as mine – I want to introduce a cache for an internal variable (x or your function call h) without changing any code within f.