In our constant battle to reduce time-to-first-X, I’m finding it very, very valuable to use the precompile approach of my bottleneck functions (diagnosed with the amazing SnoopCompile frameworks, thanks a bunch for that!). Since it is only possible to precompile a function for a finite set of input types, a crucial part of my strategy is to put a FunctionWrapper barrier between the user-facing API functions (that can take wildly varying types as input), and the digested, internal functions, that do the heavy lifting. The latter will have a predictible and small set of possible input types, so it can be precompiled to great effect. This is a toy example of the approach
using FunctionWrappers: FunctionWrapper
struct DigestedInput
test::FunctionWrapper{Bool,Tuple{Int}}
end
digest(xs) = DigestedInput(x -> test(x, xs))
test(x::Int, xs) = any(test.(x, xs))
test(x::Int, x´::Int) = x == x´
myfunc(x, input) = myfunc(x, digest(input))
function myfunc(x::Int, d::DigestedInput)
# heavy function using d.test
result = d.test(x)
return result
end
precompile(myfunc, (Int, DigestedInput))
Then, a user call with a complicated input type, such as myfunc(0, (1, (2, 3), ((4, 5), 6), (x for x in [2,0,4]))) will quickly be forwarded to the precompiled method myfunct(::Int, ::DigestedInput), where the heavy lifting is made.
In my work I have found this to be a good way to reduce latency while allowing expressive type flexibility in the user API.
But now comes the problem: the basic pillar of this approach is this very special package by @yuyichao called FunctionWrappers that does some inscrutable magic with pointers and whatnot. Way beyond what I understand in any case. In my usecase it essentially encapsulates operations on arbitrary types, to produce a result of known type.
The package is quite old, but it is not registered and is still labeled as experimental, I believe. I assume it is because there are some lingering limitations, e.g. it cannot wrap functions with kwargs, as far as I understand. This situation is quite uncomfortable as FunctionWrappers provides a crucial functionality that is not available in any other way in Julia. For me at least its role in the battle to reduce latency is more important than ever.
Recently FunctionWrappers broke for me in v1.8 beta1. Here is the Github issue. The problem is that I am at a loss to try and help resolve it, because it requires some deep understanding of Julia internals.
I hope the above has made a case for my plea: would it be possible to incorporate FunctionWrappers as first-class mechanism in Julia, or at least put it into the stdlib? I keep basing my work on it, but its current status makes me quite uneasy about it.
(Of course, if somebody can suggest a better/more official way to do what I explain above, I would be very grateful!)
.
. Didn’t budge the compilation time, still recompiles the full solver package.