I have a main program. It will load any one of various drivers (plugins?), depending on a configuration file. Each driver will operate a hardware device or accept files from the outside. The drivers have to be all listed in Project.toml, but only one can be loaded, because they all define and export the same function names. How do I do this?
I tried @less using, to see the source code of using, but the REPL hung and I had to type ^C. I tried @less using(), but got an error because using is not a function. Is there a function I can call to load a module, whose name is not known until runtime, in the same way as using?
I suspect there might be better solutions, but you could use eval for this:
If this is main.jl
module A1
export foo
foo(x) = x
end
module A2
export foo
foo(x) = 2x
end
moduleName = Symbol(ARGS[1])
@eval using .$(moduleName)
println(foo(1))
You would get
$ julia main_test.jl A1
1
$ julia main_test.jl A2
2
Using @eval comes with the usual caveats of execution any code you pass to it from the outside of course (whatever you read from the config file).
Depending on if you need the list of drivers/plugins to be expandable from the outside, you could just check if the runtime value of the plugin name is in a list of allowed names or something like that.
But if I understand this correctly, the @eval using $() expression would at least error if you try to interpolate anything that’s not just a symbol into it. So it should be reasonably restrictive already.
Note the dot . in my example since I’m using modules defined directly in Main. In your case it would probably be @eval using $(...) without the leading ..
PS: If your list of possible plugins is fixed in advance, you can also just write an if statement
if moduleName == :A1
using .A1
elseif ...
...
end
which would avoid @eval altogether. Of course it’s not as concise, but might be enough in your situation and is quite straightforward to understand. For some reason I thought that this doesn’t work, but I can’t recall why… after all, conditional loading of packages is also what is done by some people in the startup.jl file, e.g. something like
I think you should always load all the drivers and solve this with dispatch. Here is how this could look
Have on abstract type for the plugins e.g.
# module structure optional but good style and helps show the dependencies in this example
module DriversBase
# export interface functions
export load, end
abstract type AbstractDriver end
# also define the functions the drivers implement
# so the driver can add their dispatches
# also it's nice to see the whole interface together in one file
"""
load(::AbstractDriver)
Set up the driver
"""
function load end
"""
dostuff(::AbstractDriver, msg::String)
Do stuff with the message.
"""
function dostuff end
end #module
Have each driver/plugin define a type and then add their methods to the aforementioned functions:
module FooDriver
# import DriversBase module from parent
using ..DriversBase
# export this driver's type
export FooDriver
# note we don't need to reexport the functions because they are already exported by Drivers module
# type for dispatch
struct FooDriver <: DriversBase.AbstractDriver end
## implement the functions
function DriversBase.load(::FooDriver)
# ...
end
function DriversBase.dostuff(::FooDriver, msg)
# ...
end
end #module
Pull this together into a package/module
module MyDriverCollection
export load, dostuff
export FooDriver #, BarDriver, ...
include("drivers_base.jl")
using DriversBase
include("foo.jl")
using FooDriver
# and so on
end # module
Then all you need to do to switch drivers is write function that return one of these type:
function decide_driver(path)
# parse file and return appropriate type
return FooDriver()
end
Note: this decision is inherently type unstable so be sure to use a function barrier to contain it properly, i.e. do something like this:
function main()
# ...
driver = decide_driver()
do_everything_else(driver) # 1 runtime dispatch but then type stable again
end
Also disclaimer: I wrote this up on my phone, so minor mistakes are possible. But overall this approach should work. Feel free to ask if it doesn’t work out-of-the-box and you have trouble getting it right
Doesn’t the module name have to be listed in Project.toml for using to succeed?
It isn’t, but for a particular installation, the driver is generally fixed, though the software may need to be upgraded.
Type of what? All the drivers export startDriver(), justStopDriver(), and waitDriverFinish(), and there’s nothing obvious for it to be a type of.
There’s no need or way to switch drivers once the program’s running.
It’ll be at least a few weeks. I have one half-written driver and am going to start another tonight, and it’ll be next month at the earliest before I can buy hardware to write a hardware driver.
In general, yes. But without knowing more about your code, it’s hard to make accurate predictions. My guess is everything will be fine as long as you hit the “top-level” of your main module after dynamically loading the plugins and before your other functions execute. But I’m not even sure if using ... will create the same type of problems you would have when you try to call a function defined in @eval directly (before returning to the top-level).
Sure (or the module is defined in your code somewhere). That’s why I think that @eval using ... will error for anything that doesn’t parse to a valid “using” expression and it’s quite safe to use in this situation.
All that being said, I think @abraemer’s approach probably makes more sense here. It requires a bit of extra code, but has the additional benefit that other people (including you) can define new drivers/plugins later independently of your main package. Even if you don’t need to switch the driver dynamically, it might be convenient for testing or for writing new drivers to have a function that can turn a driver on/off.
You just create a new type representing the driver itself. This will allow these three functions to dispatch on the driver type (by which Julia will automatically choose the desired driver for you without the need of a “dynamic using expression”.
E.g. the driver called Diver1 implements the methods of this function specific to that driver
And similarly for all other drivers. The argument driver will likely not be used inside the method definition, but that’s fine.
You still have to carry around the driver explicitly for this to work and be performant. But you could also make a global variable that holds the “current driver” and define a fallback method like
This is not an artificial limitation, but I can’t tell you why, because the project is so far a trade secret. There is one planned driver which will drive two hardware devices.
I’ve done some plotting. I use CairoMakie or GLMakie, but not both at the same time, because they export functions with the same name. To change it, I have to edit the source code and restart the program. I’d like to do something similar, but choosing the module at the start of runtime by reading a config file.
This is rather paranthetical to your actual problem, but you can load both CairoMakie and GLMakie, and switch between which backend is active by doing:
using CairoMakie
using GLMakie
CairoMakie.activate!() # Sets CairoMakie as the active backend for plotting
# Do some plotting with CairoMakie
GLMakie.activate!() # Set GLMakie as the active backend
# Do some plotting with GLMakie
I have no idea how this is implemented, but at least this should make plotting a bit less of a pain.
I agree that this would be my first recommendation:
But alternatively, this seems like a good use case for Preferences. Maybe something like:
module MyPackage
using Preferences: Preferences, @load_preference
const driver = @load_preference("driver")
# We're at top-level, so no need for @static
if driver == "foo"
include("foo_driver_functions.jl")
elseif driver == "bar"
include("bar_driver_functions.jl")
elseif driver == "baz"
include("baz_driver_functions.jl")
else
error("Unsupported driver name: $driver")
end
end # module MyPackage
Or alternatively, if the drivers are all separate packages:
module MyPackage
using Preferences: Preferences, @load_preference
const driver = @load_preference("driver")
# We're at top-level, so no need for @static
if driver == "foo"
using FooDriver
elseif driver == "bar"
using BarDriver
elseif driver == "baz"
using BazDriver
else
error("Unsupported driver name: $driver")
end
end # module MyPackage
I guess that was what was meant by extensions. They have a similar purpose, but it’s more about dynamically activating some functionality in PackageA that is required for working nicely with PackageB (but only when PackageB is actually loaded with using PackageB). So in your scenario it would again require a using MyDriver statement to activate the corresponding extension, which you want to avoid, if I understand correctly.
Preferences sounds workable, but IIRR its file is in the source tree. I’m already using ConfParser so that the config file can be in /etc (it’s in ~/.config for now). Can I do the same with ConfParser?
