How to reduce type-instability performance issues in general problem interfaces?

General Usage Performance
1

I am trying to write a simple interface to define and solve a specific class of problem. The problems are defined by sets of functions and sets of variables. There can be a mix of function types, and a mix of variable types. The solver won’t know what these are at compile time; they are user defined, and, as I mentioned, heterogeneous. This type-instability will incur some dynamic dispatch performance penalties at some point in the program. With my current design, it’s being incurred at each call to the underlying functions, making the program very slow. However, there are large numbers of each type of function, so I believe the type instability can be resolved once for each type of function, rather than once for each function. I’m just not sure how to achieve this. One of the problems is that different function types can take a different number of input arguments. Do you know of any examples of similar sounding interfaces that are efficient? I’d like to take a look at them. Many thanks.

2

  • In my use-cases (biomath/agent-based models) it helped to separate type-instable parts from those which can be type stable. For example: If each cell has a position (type-stable) and some custom things (type-instable). Then you could use two data structures to each of the parts, so that the cell positions are very efficient and the custom things don’t drag the runtime down.

  • You can use parametric structs (or NamedTuples) to allow the compiler to specialise to the custom types you deal with. (I guess you already know Performance Tips · The Julia Language )

  • Finally, the trivial advice: Always use a profiler to assert which type-instabilities are actually relevant.

There are many more tips for sure :wink: I think a small example could help to get not too generic answers.

4

Thanks! Yes, I use parametric structs. I’d love some examples of general interfaces (similar to the one I described) that I can look at the code for. If I provided code rather than a description, I suspect I’d get advice on how to change the code, rather than the examples I’m after.

1 Like
5

GitHub - yuyichao/FunctionWrappers.jl is possibly relevant.

1 Like
6

The key issue is to guarantee that all types are created as concrete types when the user sets them, and use function barriers:

julia> struct A{F,T}
           f::F
           x::T
       end

julia> a = A(sin, 1.0) # user interaction: the result is concrete, but known at run time
A{typeof(sin), Float64}(sin, 1.0)

julia> f(a::A) = a.f(a.x) # this will be specialzed to the specific type of a
f (generic function with 1 method)

julia> @btime f($a)
  5.242 ns (0 allocations: 0 bytes)
0.8414709848078965

Concerning this specific point, commonly one lets the user provide the function as closures:

julia> g(f, x) = f(x)
g (generic function with 1 method)

julia> a = 1; b = 2;

julia> g(x -> sin(a*b*x), 1.0)
0.9092974268256817

Thus, while the interface requires a function that has only one parameter, that can be solved from the user side by providing a closure that closes over the other parameters of the user-defined function.

1 Like
7

Do you have a concrete example we could look at, to give specific design/architectural advice? Since julia programs are compiled Just Ahead Of Time, there is (generally) no one “compile time” and specializing on the exact functions & arguments put in happens either way.

8

As I mentioned earlier, I’m not after advice on my current design. I’m after examples of interfaces that solve this problem, that I can learn from. So I’d rather not provide code, as I don’t think it will lead to the answers I’m after.

9

The key issue is to guarantee that all types are created as concrete types when the user sets them

Useful tip. Thanks. Function barriers I knew about, but also useful.

While the interface requires a function that has only one parameter, that can be solved from the user side by providing a closure that closes over the other parameters of the user-defined function.

The interface requires access to the function inputs (which vary in number), so they can’t be hidden by a closure. If I’ve understood you correctly…

10

This is somewhat too abstract, but you can let the user provide multiple functions if you store than in tuples (each function has its own type, so types become complex):

julia> struct A{F}
           functions::F
       end

julia> f(a::A,x) = sum(f(x) for f in a.functions)
f (generic function with 1 method)

julia> a = A((sin,cos))
A{Tuple{typeof(sin), typeof(cos)}}((sin, cos))

julia> @btime f($a, 1.0)
  0.880 ns (0 allocations: 0 bytes)
1.3817732906760363
11

Thanks. However, I don’t think this helps in my case. I think the variable number of arguments is an issue that perhaps does require some example code. I’ll start another thread for that, and share a link. I’m keen to get suggestions of example interfaces here.