How to implement a plugin system in Julia

General Usage
1

TL;DR: Does multiple dispatch allow for a better alternative to plugins? If no, then a code generation gimmick is needed.

I am working on a framework and I would like to move most of the code into plugins. A plugin alters the behavior of the framework by implementing “hooks”. It has its own configuration and possibly state, and although it can work independently from other plugins, it may communicate with them. (E.g. several telemetry plugins create metrics and a monitoring plugin collects them).

I have a dynamic implementation that works, but it is not fast enough. There are a few hot points where the hooks need to be inlined. I did what my current understanding of Julia allows and come up with a possible solution, but I have two questions, a philosophical and a technical:

  1. I have the gut feeling that the whole idea of a plugin is somehow not well aligned with multiple dispatch, and that a better solution may exist to the problem of building the framework out of composable chunks of code in a way that allows the user (application developer) to remove, replace or reconfigure any of the chunks easily. (Please check the code to see the functionality I feel essential)

  2. If traditional plugins are the way to go, then: My solution is built on the idea of chained types (borrowed from layers in HTTP.jl) , but to allow plugin configuration and state, I instantiate the plugin types. So a plugin set looks like PerfPlugin(LogPlugin(DefaultPlugin(), "Configured logger:"))) which has the type PerfPlugin{LogPlugin{DefaultPlugin}}. The instances are also chained. This allows inlining, because for every hook the compiler knows which plugins in the chain implement it. For a few plugins this works great.
    Unfortunately, to find the instance of the next plugin to call, intermediate ones must be skipped through, so the compiler generates a movq (%rax), %rax for every skipped plugin (if the config/state of a later one is needed). This is a minor but not negligible issue, as having 20 plugins seems to be normal, and the hottest point I hope to make hookable is executed up to 4 million times per second.
    How can I eliminate traversal of the chain?

You can try out the code at repl.it

# A framework with plugins:

abstract type Plugin end

struct DefaultPlugin <: Plugin end # Default behavior, implemented as a plugin

struct Framework{TPlugin}
    firstplugin::TPlugin
end

next(plugin) = plugin.next

# A hookable operation
@inline step(framework::Framework) = step_hook(framework.firstplugin, framework)

@inline step_hook(plugin::Plugin, framework) = step_hook(next(plugin), framework)
@inline step_hook(::DefaultPlugin, framework) = sleep(rand() * 0.01)

# Plugin implementations:

using Dates
mutable struct PerfPlugin{Next} <: Plugin
    next::Next
    timesum::Millisecond
    stepcount::UInt
    PerfPlugin(next::Next) where Next = new{Next}(next, Millisecond(0.0), 0)
end
@inline step_hook(plugin::PerfPlugin{Next}, framework) where Next = begin
    ts = Dates.now()
    step_hook(next(plugin), framework)
    timetaken = Dates.now() - ts
    plugin.timesum += timetaken
    plugin.stepcount += 1
    println(timetaken)
end

# This can be called by plugins installed earlier in the chain
avgsteptime(plugin::Plugin) = avgsteptime(next(plugin))
avgsteptime(::DefaultPlugin) = NaN
avgsteptime(plugin::PerfPlugin) = plugin.timesum.value / plugin.stepcount

struct LogPlugin{Next} <: Plugin
    next::Next
    prefix::String
    LogPlugin(next::Next, prefix="Default Logger:") where Next = new{Next}(next, prefix)
end
@inline step_hook(plugin::LogPlugin{Next}, framework) where Next = begin
    println("$(plugin.prefix) Stepping...")
    step_hook(next(plugin), framework)
end

# Does nothing, needs to be skipped
struct EmptyPlugin{Next} <: Plugin
    next::Next
    EmptyPlugin(next::Next) where Next = new{Next}(next)
end

# Application

println("-- a1: Perf plugin only:")
a1 = Framework(PerfPlugin(DefaultPlugin()))
step(a1)

println("\n-- a2: Log plugin(configured) only:")
a2 = Framework(LogPlugin(DefaultPlugin(), "Configured logger:"))
step(a2)

println("\n-- a3 Both plugins, two steps:")
a3 = Framework(LogPlugin(PerfPlugin(DefaultPlugin())))
step(a3)
step(a3)
println("Average step time: $(avgsteptime(a3.firstplugin)) milliseconds")

println("\n-- a4: three Empty plugins + 1 Log plugin")
a4 = Framework((EmptyPlugin(EmptyPlugin(EmptyPlugin(LogPlugin(DefaultPlugin()))))))
step(a4)

println("\n-- @code_native step(a4)")
using InteractiveUtils
@code_native step(a4)
1 Like
2

These are interesting notions. How about a plugin “cursor” to point at a selected plugin?

1 Like
3

Thank you for replying!

Yes, in the previous version (that simply stores plugins in a Dict) I already have something like that: for every hook I filter the implementing plugins and store the list in an array, so that at hook time a cycle can go through them directly.

It is not yet clear to me how to build a similar logic into the current scheme. I feel like there is no need to store the filtered list of plugins, because it will be used only once, at code generation. So this could really be something like a cursor. But how to implement it?

4

What is the problem you are trying to solve? Why is plugin the apparent solution?

1 Like
5

Extendibility of an actor framework.

For example, actors can migrate between schedulers, and messages that arrive at the source scheduler during and after migration have to be handled. The default behavior is to buffer messages during migration and send them to the new location when the migration is successful. Messages arriving later will be sent back as “recipient moved”. A latency-critical application may need a different logic, so this is implemented as a plugin that hooks into the message routing. There is also a websocket plugin that hooks into the same point to forward messages to external actors running in a web browser.

To be fair, those examples do not need inlining, as they are handling only the exception case when the recipient actor is not found on the scheduler. The default local routing behavior cannot be changed by a plugin currently because of the performance penalty, and this makes it hard to implement features like message authentication and filtering in a pluggable fashion. Of course it is possible to change the default routing by creating a specific scheduler type and overloading a function, but then composing of such modifications seems to be manual work waiting for the application developer. Something I want to avoid.

1 Like