# \[RFC- ANN\] CachedFunctions.jl

**URL:** <https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091>\
**Category:** Package Announcements\
**Tags:** package\
**Created:** [February 2, 2020, 8:07pm UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091 "2020-02-02T20:07:56Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![longemen3000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/longemen3000/32/7298_2.png) [@longemen3000](https://discourse.julialang.org/u/longemen3000)\
**Post date:** [February 2, 2020, 8:07pm UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/1 "2020-02-02T20:07:57Z")

</div>

Hi, i just want to announce and request for comments about my package:

> **[GitHub - longemen3000/CachedFunctions.jl: cache all the things!](https://github.com/longemen3000/CachedFunctions.jl)**
>
> cache all the things! Contribute to longemen3000/CachedFunctions.jl development by creating an account on GitHub.

I did this as a possible solution to the calculation of higher order hessians, inspired heavily by [this post](https://discourse.julialang.org/t/nested-forwarddiff-jacobian-calls-with-inplace-function/21232/3). the idea is simple: if you have an inplace function and know the size of the input and the output beforehand, and the function returns the same type as the input, then the creation of caches can be programmatically done, instead of manually.  
The idea is transform `f!(y,x)` to `f(x)` and be allowed to use this function freely (the main reason, passing it to a ForwardDiff routine to reduce allocations). im working on a CachedJacobian using this package to calculate higher order jacobians without allocations (preallocating also the caches neccesary by ForwardDiff and FiniteDiff). Comments about the interface, described in the readme, feature requests and corrections would be greatly appreciated.  
Already registered! waiting to be added to the general registry.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 3, 2020, 3:32am UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/2 "2020-02-03T03:32:20Z")

</div>

From a quick look at the code, it looks like invoking `f(x)` from different threads might corrupt the data. Do you have a plan/idea for thread safety?

---

<div class="post-metadata">

**Author:** ![longemen3000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/longemen3000/32/7298_2.png) [@longemen3000](https://discourse.julialang.org/u/longemen3000)\
**Post date:** [February 3, 2020, 3:45am UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/3 "2020-02-03T03:45:53Z")

</div>

good catch!, i’m gonna look how to make the calls thread-safe, but the only thing that comes to me right now is using a `lock` at the `CachedFunction` call level. Another approach would be creating NThreads caches managed by the `CachedFunction`, with the cache selection also including the `threadid` as a key.  
If you have something in mind, let me know!

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 4, 2020, 1:26am UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/4 "2020-02-04T01:26:32Z")

</div>

Yes, I think I’d use `threadid`, too. Something like `Vector{IdDict{DataType, Union{AbstractArray,Number}}}`. I don’t know if “`threadid` as a key” works because `Dict` can (in principle) mutate underlying data structure in a non- thread-safe manner.

* * *

Also, come to think of it, it mgith be possible to get a puzzling behavior even without threading but just with `@async`? Consider

```julia
@assert f isa CachedFunction
@assert typeof(x1) == typeof(x2)

@sync begin
    task = @async f(x1)
    y2 = f(x2)
    y1 = fetch(task)
end

@assert y1 === y2 # but maybe this is puzzling?

```

It’s a bit silly example but it could introduce a subtle bug for something like

```julia
@assert f isa CachedFunction
@assert typeof(x1) == typeof(x2)

@sync begin
    @async g(f(x1))
    @async h(f(x2))
end

```

If `g` contains some I/O, `f(x2)` can be called before `g` finishes using `y`.

I suppose it seems that it’s weird to worry about I/O with `CachedFunction`, given that it’s presumably geared towards compute-intensive works. But it’s conceivable in Julia ecosystem that someone might want to wrap some computational package in a web API. In that case, worrying I/O started to make sense. Though I’m not sure if it’s possible to “fix” this other than re-implementing GC.

---

<div class="post-metadata">

**Author:** ![longemen3000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/longemen3000/32/7298_2.png) [@longemen3000](https://discourse.julialang.org/u/longemen3000)\
**Post date:** [February 4, 2020, 1:57am UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/5 "2020-02-04T01:57:51Z")

</div>

> [@tkf](#):
>
> I suppose it seems that it’s weird to worry about I/O with `CachedFunction`

No, not at all!, i come to the realization that this is a harder problem than i thought, i was thinking of modifying the `CachedFunction` constructor to add a `threadsafe` keyword, where `threadsafe` = false would give the current behavior, whereas `true` would lock the caching stage and evaluation stage,

Other approach is to create different constructors for different uses. a `SyncCachedFunction`, a `ParallelCachedFunction` (to do calculations on all threads at once, for example) and others if necessary.

At last. it should be clear to the user what to expect of the created `CachedFunction`, and those examples are great to bring the expected usage to light. This is also the right moment to do a lot of breaking changes, if those would bring a better package. If you have any request or proposal, those would be very (emphasis on very) appreciated 😀

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 4, 2020, 2:09am UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/6 "2020-02-04T02:09:11Z")

</div>

Thinking about this a bit more, I guess you can add an interface like `lock(g, f::CachedFunction, x)` that locks the _output_ `y` of `f(x)` while evaluating `g(y)`. I think it’s a nice way to declare the lifetime of the cached output.

```julia
@sync begin
    @async lock(f, x1) do y # y = f(x1)
        g(y)
    end
    @async lock(f, x2) do y # y = f(x2)
        h(y)
    end
end

```

Note that `f` still can implement thread-local cache such that

```julia
@sync begin
    @spawn lock(f, x1) do y
        g(y)
    end
    @spawn lock(f, x2) do y
        h(y)
    end
end

```

can still do the works in parallel if there are enough threads.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 4, 2020, 2:11am UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/7 "2020-02-04T02:11:49Z")

</div>

…although I guess this would deadlock with

```julia
@assert typeof(x1) == typeof(x2)

lock(f, x1) do y1
    lock(f, x2) do y2
        g(y1, y2)
    end
end

```

unless it is explicitly supported.

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [February 11, 2020, 9:32pm UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/8 "2020-02-11T21:32:45Z")

</div>

Note that [LRUCache.jl](https://github.com/JuliaCollections/LRUCache.jl) provides a thread-safe associative structure, which I use to cache temporaries in e.g. TensorOperations.jl. Nonetheless, I still use the `threadid()` as part of the key, but that alone is not sufficient if you store all cached results in a common structure.

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [February 11, 2020, 9:59pm UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/9 "2020-02-11T21:59:14Z")

</div>

although the `threadid()` part of the key might be because of how I use the objects stored in the cache, they are indeed overwritten in place after being retrieved, so I want to make sure to have different objects associated to different threads. If it is just to look up values, just the thread safety of the `LRU` structure should be sufficient.

---

<div class="post-metadata">

**Author:** ![longemen3000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/longemen3000/32/7298_2.png) [@longemen3000](https://discourse.julialang.org/u/longemen3000)\
**Post date:** [February 11, 2020, 10:37pm UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/10 "2020-02-11T22:37:41Z")

</div>

so, basically, what i have to do is to do all the unsafe operations behind a lock stored on the struct, right? what’s the difference between the reentrant lock and the spinlock in julia?

---

<div class="post-metadata">

**Author:** ![weymouth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/weymouth/32/15839_2.png) [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Post date:** [April 12, 2021, 5:10pm UTC](https://discourse.julialang.org/t/rfc-ann-cachedfunctions-jl/34091/11 "2021-04-12T17:10:14Z")

</div>

This is just what I need for avoiding some allocations in a function passed to ForwardDiff. However, when I run the README example, I’m getting a method not found error

[https://github.com/longemen3000/CachedFunctions.jl/issues/5#issue-856177902](https://github.com/longemen3000/CachedFunctions.jl/issues/5#issue-856177902)