I haven’t fully digested this thread (working on it) but on quick glance the problem of caches stored in generated functions dissappearing or not getting updated seems similar to what I had to solve in Memoization.jl here. The idea is not to store all the caches in any global store because that can get out-of-date as stuff is re-@generated and between precompilation and runtime, but rather at runtime go through the actual generated methods and extract the exact cache which is currently there (slower, but this is really only needed for emptying so not “performance critical”) Maybe that’s helpful info.
I see, that is very clever and would indeed solve one half of the problem.
The other issue is that calling Core.compiler.return_type from (the generating part of) a generated functions is apparently not supported. Although I have so far not encountered any issues with that.
Unfortunately return_type is rather slow, so it should really only be called once. My idea to solve that was to use a generated function as a switch for the first call, like so:
@generated first_call(funct, args)
ft = [true]
:($(ft[1]) && !($(ft[1]) = false))
end
This could then be used to setup everything on the first call to the memoized function and just fall through to the call to get! afterwards. In this scenario the return type could even be determined by simply calling the kernel function once.
I’m still not sure, though, how to efficiently access the cache in this scenario, but maybe your solution would work there as well.
Edit: Ok, I think I’ve understood how this works in your package now (I didn’t even know that GFs can return plain values instead of expressions - nice). The problem remains that the return type is needed on the first call to the GF (to correctly parameterize the container) but not afterwards, as it’s expensive to obtain.
That would be fantastic. I would much rather use an established package than roll my own solution.
Wow, these efforts were really nice. Did we eventually get a final “universal” great solution here?
What if we can force the return’s type and then skip type deduction and go with that first just to stay type stable as a universal solution till someone find a correct type deduction method for this problem?
I created this, I know this is very basic and has to be sure that you aware with this variable hygenie… But it is working.
macro typed_memo(out_type,funccall)
funcname = funccall.args[1]
args_tsafe = tuple(funccall.args[2:end]...)
cache_funcname = Symbol("$(funcname)_cached")
esc(quote
!@isdefined($cache_funcname) && ($cache_funcname=Dict{typeof($args_tsafe),$out_type}())
if $args_tsafe in keys($cache_funcname)
$cache_funcname[$args_tsafe]
else
$cache_funcname[$args_tsafe] = $funccall
end
end)
end
I hope later I will be able to work more on this to make it properly for everyone.