[ANN] Plugins.jl: Zero cost plugins

Package Announcements
1

Hi,

I am thrilled to announce the result of my first contact with staged programming: Plugins.jl

Motivation

Let’s say you are writing a lib which performs some work in a loop quickly (up to several million times per second). You need to allow your users to monitor this loop, but the requirements vary: One user just wants to count the total number of cycles, another wants to measure the performance and write some metrics to a file regularly, etc. The only requirement that everybody has is maximum performance.

Maybe you also have an item on your wishlist: To allow your less techie users to configure the monitoring without coding. Sometimes you even dream of reconfiguring the monitoring while the app is running…

Plugins can be found everywhere, including much-used Julia packages like Documenter.jl, PkgTemplates.jl, Genie.jl and HTTP.jl (called Layers).

Plugins can help to build flexible and extensible architectures, but their dynamic nature comes with a performance burden that prevents their use in really hot code.

This is the problem that Plugins.jl solves.

There is no standard definition of the word “plugin”, here is the one used by Plugins.jl:

A plugin (aka extension) is a chunk of code that extends the functionality of a system. The plugin has its own lifecycle and it implements so-called hooks to react to events generated by the system. If multiple plugins implement the same hook, they will be called in their order, with any plugin able to halt the processing.

Example

A very simple example with a plugin that counts the calls to the tick() hook:

mutable struct CounterPlugin <: Plugin
    count::UInt
    CounterPlugin() = new(0)
end

Plugins.symbol(::CounterPlugin) = :counter

function tick(me::CounterPlugin, app)
    me.count += 1
    return false # return true to interrupt hook processing
end

And an “application” that calls the tick() hook in a tight loop:

mutable struct App
    plugins::PluginStack
    App(plugins, hookfns) = new(PluginStack(plugins, hookfns))
end

function tickerop(app::App)
    tickhook = hooks(app).tick
    tickerop_kern(app, tickhook) # Using a function barrier to get ~7ns per hook activation
end

function tickerop_kern(app::App, tickhook)
    for i = 1:1e6 
        tickhook()
    end
end

const app = App([CounterPlugin()], [tick])
@btime tickerop(app)
println("Total Tick Count: $(app.plugins[:counter].count)")

  221.766 μs (0 allocations: 0 bytes)
Total Tick Count: 27580000000

1 million hook calls in 0.222 ms (That is ~4.5 GHz, the clock speed of my proc).

A little more lifelike example with two plugins on the same hook can be found in the getting started guide.

Features

  • Merging hook implementations into a single function body
  • Skipping non-implementing plugins with zero cost
  • Hook implementations can prevent later ones from running by returning true (like DOM event handlers)
  • Access of the loaded plugins by their registered symbol (from other plugins or from the outside)
  • Passing a shared state and arbitrary number of arguments to hook implementations
  • Adding or removing plugins dynamically
  • A fast cache of merged hooks that gets rebuilt when the plugin stack changes

Interestingly, neither @generated nor eval() or any compiler magic was needed to make this, only a chain of parametric types. The main idea was borrowed from HTTP.jl’s layers.

I hope that this package will be useful to the community!

14 Likes
2

Looks cool! Though, I confess I’m not totally sure what the package does. I encourage you to expand this snippet from the docs, which seems to provide the basic intuition of “what’s this package for” a bit easier than the github readme:

Let’s say you are writing a lib which performs some work in a loop quickly (up to several million times per second). You need to allow your users to monitor this loop, but the requirements vary: One user just wants to count the total number of cycles, another wants to measure the performance and write some metrics to a file regularly, etc. The only requirement that everybody has is maximum performance.

Maybe you also have an item on your wishlist: To allow your less techie users to configure the monitoring without coding. Sometimes you even dream of reconfiguring the monitoring while the app is running…

1 Like
3

This is very similar to me to the concept of callbacks implemented by solvers like CPLEX, Gurobi, and so on.

4

Thanks for the feedback! (Now I have added this to the Motivation section up there)

The story continues like this:

  1. You:
  • add Plugins.jl as a dependency to your lib.
  • define a function named e.g. somework_callback() without implementation.
  • add a Plugins.PluginStack field to your model struct and pass the plugins your users optionally provide when they set up the model. (The model is called App in the example here)
  • call hooks(model).somework_callback() in every step
  1. Then plugin authors:
  • create a subtype of Plugins.Plugin
  • import somework_callback() from your package
  • and implement it on their plugin
  1. Finally your users:
  • load the plugins they like
  • instantiate them and provide them to your model
1 Like
5

Thanks for pointing to that!

I never used any solver, maybe I miss something, but after checking htps://jump.dev/JuMP.jl/dev/callbacks/ I think “InlineCallbacks” could be a valid name for this package. Plugins.jl allows a nicer API, maximal performance and composability of callbacks.

6

I think if I saw InlineCallbacks.jl I would immediately have a good idea of what your package does (and be a little bedazzled, as this seems to involve a some dark magic). Plugins.jl is kinda of a generic name. I would expect it to be something like Tables.jl but for people wanting to create their own plugin packages for diverse environments (like desktop environments, browsers, etc…).

3 Likes