Public package as plugin

Hi there,

I am developing a program that should have a plugin system. I am thinking of using Julia’s package system for this purpose. The idea is that you install a Julia package with additional functionality and when you run the program this functionality activates. This doesn’t sound too hard, but here is the catch

I don’t want my program to know in advance what extension packages exist. I hope other people can write these plugin packages and just by installing them, my program finds them when it runs.

  • Is this possible?
  • How would you go about it?
  • Is it a good idea?

Cheers,
kalcool

this happens all the time, for example when you load LoopVectorization it enables Tullio, or any of the million packages that have Plotting recipe for their objects, they extends Plots.plot

Maybe I am missing something, but in this case the user must introduce an object of a new type and call the function so the extension is used, right? So to call the function the object would have to be part of the code of my program already.
Therefore I think introducing new function signatures would not solve my problem by itself
and overloading an existing function signature would also not work, because multiple plugins cannot share that overloaded function.

To illustrate: Imagine my program has the routine runPlugins(x) which is normally empty, a new package could add a signature runPlugins(i::Int32), but my program would have to call runPlugins with an integer, so it needs to know in advance the type of arguments to use, to trigger the new function for each plugin. I don’t know exactly what is possible with new packages so if overwriting the runPlugins(x) function of the main program is possible. Plugin 1 could supply a runPlugins function and Plugin2 could supply a run plugins function, but my programs runPlugins() function should run those functions sequentially, and not just one of them.

the new package you loaded can completely change the behavior of an existing function, but that’s really bad practice, and I honestly don’t know what’s good with that.

Do you have a concrete use case example?

Imagine a graphics editor. Let’s say I would like to have Gaussian blur as a plugin and invert colors as a plugin. If both packages are installed the editor should show one button for each to manipulate the picture. If either one, two or none of the plugins are installed, the editor should show the appropriate number of buttons.
But again my program should not know in advance what buttons can exist, so plugin writers don’t need write access to my code repository.

ok, the kind of plugin system you described should NOT be at the language level. It should be part of your software, ecosystem logic.

In other words, the “loading a plugin” should not translate to using some_plugin directly in Julia.

So you say it’s a bad idea. Could you elaborate on why? Or is it impossible? Because a lot of functionality that I would like to have for my plugins, seems to come for free with Julia’s package management.

I’m saying you need to implement your software logic additionally. Because a plugin is fundamentally different than a package – it’s possible for packages to extend and specialize each other’s behavior, but you can’t have a “manager” dynamically populate some menu data structure based on how many and what packages you loaded (it will be extremely hacky); not to mention how to communicate what plugin applies to what, you need your plugin specification for this (which can involve Julia types, for example AbstractPlugin, and then AbstractFilter <: AbstractPlugin)

OK. Thank you. I’ll wait see if there are more suggestions that don’t look super hacky and otherwise I’ll look for another solution.

The init functions of the plugins could populate the menu (by pushing the new features to a shared list). Isn’t that possible?

o.k. so have looked into using __init__, and this can probably work. I can check if a package is installed using Pkg.installed() or Pkg.dependencies(). What I am missing now is a way to do using on a Package specified by a String.

using "Modulename"

doesn’t work and neither does

using Symbol("Modulname")

Do you have any ideas for that?