Understanding the limitations of generated functions

General Usage
, ,
17

assuming that every dispatch is dynamic is not very lifelike

You know what the rest of your code looks like far better than we do, so I’ll accept that “lifelike” for you means that you’re not planning to loop over lists of objects with different types or other difficult-to-infer operations. Since you seem to like the generated function approach, you can do what you want with your own code but be aware that the downsides are:

  • your compile-time latency will be poor even if you get good runtime performance
  • you can’t empty!(pools) and start from scratch—getpool will return the old vector forever filled with whatever contents it had last. Likewise you can’t free the memory of each vector without calling empty!(getpool(T)) on every T that has ever been used.
  • direct manipulations of pools may or may not yield the same result as accessing via getpool. For example direct manipulations of pools may interact badly with precompilation due to the need to rehash the dict

Basically, that “prohibition” on mutation/observation is there because it’s the only way to ensure that the generated function will reproducibly do the same thing. Failing to observe it means that you may open yourself up to surprising behavior depending on the history of what you’ve done. For private code, only you can decide whether these concerns are relevant. If you really do want to use the generated function, I’d recommend a pattern

let pools = Dict{Type,Vector}()
    global getpool
    @generated ...
end

to prevent anyone from manipulating pools directly.

In part, you can think of an IdDict as a Dict with @nospecialize wrapped around all usage of keys. It’s specifically designed to handle the situation where the keys may be of diverse types, circumventing runtime dispatch. The other piece of it will only be clear if you know C: you can think of the objectid as a pointer, and that’s immediately available for types so looking up values is fast.

The reason that having a UnionAll type as the value-type makes it slow is presumably this line though I didn’t explicitly check. Type assertions are fast when the type being asserted is concrete, but when it’s non-concrete then it requires subtype analysis. Any gets special-cased.

3 Likes