# Static vector of functions as a field of a composite type

**URL:** <https://discourse.julialang.org/t/static-vector-of-functions-as-a-field-of-a-composite-type/99083>\
**Category:** New to Julia\
**Tags:** performance, function, staticarrays, symbolics, autodiff\
**Created:** [May 18, 2023, 8:17pm UTC](https://discourse.julialang.org/t/static-vector-of-functions-as-a-field-of-a-composite-type/99083 "2023-05-18T20:17:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![aafsar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aafsar/32/47420_2.png) [@aafsar](https://discourse.julialang.org/u/aafsar)\
**Post date:** [May 18, 2023, 8:17pm UTC](https://discourse.julialang.org/t/static-vector-of-functions-as-a-field-of-a-composite-type/99083/1 "2023-05-18T20:17:19Z")

</div>

I’m trying to comprehend the details of the type system and its effect on performance, and would appreciate if you can help me understand how it applies to my current case, thank you!

**Question:**  
Would it be a bad idea to use a `SVector` of functions in the field of a composite type, e.g:

```julia
struct DifferentiableProblem{N, F <: Function}
    target_function::F
    derivatives::SVector{N, F}
end

```

I have two confusions:

1. Since each function is its own concrete type with associated methods, `derivatives` field must obtain a concrete type as defined above. But I’m guessing a concrete function object doesn’t have a fixed size, so having a `SVector` of them is probably not a good idea?
2. I know that high-order functions are performant in Julia, and it’s usually not a good idea to store functions in the structs. But in my case these functions are not “methods” to be applied to the data, but they are the part of the data. So that’s why I thought of storing them in a struct, but perhaps there are better approaches. Please see below for details.

**Details of the case:**  
I’m developing a package where the user defines a problem by entering a (non-linear) function f \colon \mathbb{R}^{n+k} \to \mathbb{R}^n, where \mathbb{R}^k can be thought of as the parameter space. Then for each point p in (a grid of) a bounded subset of the parameter space, the program makes some computations, which uses the function f\_p \colon \mathbb{R}^n \to \mathbb{R}^n given by f\_p(x) = f(x,p), and the main diagonal of its Jacobian. Since for each problem the form of the function f and the relevant entries of its Jacobian is fixed, I thought it might make sense computationally to generate and store the relevant derivative functions once at the beginning when the user defines the problem.

**Addendum:**  
For constructing the derivatives, I am currently doing symbolic calculations using `build_function` of `Symbolics.jl` for 2 reasons: (1) `JuliaInterval` methods are utilized in the computations and I had hard time to mix them well with AD packages, since I guess both use propagation, (2) I thought it would be computationally cheaper to generate the derivative functions symbolically instead of calculating them via AD each time. But maybe I am also confused a bit here.

---

<div class="post-metadata">

**Author:** ![mikmoore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikmoore/32/31109_2.png) [@mikmoore](https://discourse.julialang.org/u/mikmoore)\
**Post date:** [May 18, 2023, 8:44pm UTC](https://discourse.julialang.org/t/static-vector-of-functions-as-a-field-of-a-composite-type/99083/2 "2023-05-18T20:44:11Z")

</div>

An `SVector` containing different types (every function has its own unique type) will either have an element type that is a union (eg `Union{typeof(sin),typeof(cos)}`) or abstract (eg `Function`). Neither of these is going to be great for performance.

For this purpose, I would instead use a plain `Tuple`, since a `Tuple` can hold a different type in every position. In other words, I would suggest something more like

```julia
struct DifferentiableProblem{F,T<:Tuple}
    target_function::F
    derivatives::T
end

```

If _this particular part of the code_ isn’t super performance sensitive, you might do fine with dynamic dispatch. In that case you can just leave these fields weakly typed or untyped:

```julia
struct DifferentiableProblem
    target_function::Any # non-concrete type
    derivatives::Tuple # non-concrete type
end

```

Depending on what you’re doing, there may or may not be a big performance cost to this. Concrete typing is important in performance-critical code segments, but for higher-level pieces sometimes unstable types can work just fine (and save a lot of compiler effort). [Function barriers](https://docs.julialang.org/en/v1/manual/performance-tips/#kernel-functions) can do great work.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 19, 2023, 9:25am UTC](https://discourse.julialang.org/t/static-vector-of-functions-as-a-field-of-a-composite-type/99083/3 "2023-05-19T09:25:25Z")

</div>

Allow me to add that a tuple is interesting for low-dimensional collections, if you have more than a handful of functions I don’t think it’s gonna save you.

There were several Discourse topics on storing functions in arrays recently, but I can’t seem to find them. It would definitely be worth checking out.

As for the computational cost of symbolic vs algorithmic vs numerical differentiation, it depends very much on the problem at hand. There are simple cases where a symbolic formula for the derivative will have exponentially many terms, whereas algorithmic differentiation will avoid duplicate computation by reinterpreting a simple piece of code (not unlike the recursive version of Fibonacci vs a for loop). Can you tell us more about your use case?

---

<div class="post-metadata">

**Author:** ![aafsar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aafsar/32/47420_2.png) [@aafsar](https://discourse.julialang.org/u/aafsar)\
**Post date:** [May 20, 2023, 11:59am UTC](https://discourse.julialang.org/t/static-vector-of-functions-as-a-field-of-a-composite-type/99083/4 "2023-05-20T11:59:54Z")

</div>

Thank you @mikmoore and @gdalle for your great answers!

**Regarding storing functions:** The collections are indeed low-dimensional, e.g. \<10. The reason I explored using `SVector`s is to be able to access the number derivative functions for compiler to reason about when iterating over, [as discussed in this post](https://discourse.julialang.org/t/array-of-functions-is-there-a-way-to-avoid-allocations-performance-penalty/24471), but I guess one can simply use bare `Tuples` [as suggested in this post](https://discourse.julialang.org/t/get-tuple-length-from-type/32483) directly without the extra `Static Array` structure on top. I guess another way would be to use [FunctionWrappers.jl](https://github.com/yuyichao/FunctionWrappers.jl) for the `target_function`, and use the type of the output to inform compiler about the length of the `derivatives` `Tuple` (since if the codomain of the `target_function` is \mathbb{R}^n, then the number of derivatives, i.e. the length of

**Regarding Symbolic D vs AD** : I think the package would benefit from AD, since the `target_function`s are very nonlinear, and in most cases would lead to “expression swell” as you mentioned. As an example of a `target_function` (for some fixed parameters) h \colon \mathbb{R}^n \to \mathbb{R}^n:

h(y)=(h\_1(y),...,h\_n(y)), such that  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; h\_i(y) = \int\_X f\_i(x,y)g(x) dx, where  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f\_i(x,y)=\frac{e^{1 - \lVert x- y\_i \rVert}}{1+\sum\_j{e^{1 - \lVert x- y\_j \rVert}}} and g(x) is some pdf.  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; and the derivatives of interest are \frac{\partial^k h\_i}{\partial y\_i^k} for k \in {1,2}.

**The use case:** The practice is a multi-objective optimization: finding local Nash equilibria in a game theoretic setting. So the aim is to find the roots of \tilde{h}(y)=(\frac{\partial h\_1}{\partial y\_1},...,\frac{\partial h\_n}{\partial y\_n}) such that \frac{\partial^2 h\_i}{\partial y\_i^2}\<0 \, \forall i (ignoring other cases for now). One approach that I have been trying is to use interval arithmetics (à la [JuliaIntervals](https://github.com/JuliaIntervals) by @dpsanders et al.), especially using their [IntervalConstraintProgramming.jl](https://github.com/JuliaIntervals/IntervalConstraintProgramming.jl)(\*), however the current version seems to be not compatible with `Dual` type propagation needed to couple it with something like [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl), since it internally converts constraints to `ModelingToolkit.Operation` types for which the `Dual` is not defined, and I couldn’t figure out how to make it work yet. (I guess I should open a new topic for this since it’s a different issue)  
Long story short, that’s why I reverted to using symbolic differentiation using [Symbolics.jl](https://github.com/JuliaSymbolics/Symbolics.jl).

(\*) The reason that I try to implement IntervalConstraintProgramming is that each y\_i might be allowed to be vector-valued, e.g. points in 2D space, and thus the function for which the roots to be found becomes \tilde{h} \colon \mathbb{R}^{2n} \to \mathbb{R}^n, disallowing methods like Newton due to non-invertable Jacobian (as far as I understood).

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 20, 2023, 12:09pm UTC](https://discourse.julialang.org/t/static-vector-of-functions-as-a-field-of-a-composite-type/99083/5 "2023-05-20T12:09:32Z")

</div>

Have you tried reverse-mode automatic differentiation?