I have a main program. It will load any one of various drivers (plugins?), depending on a configuration file. Each driver will operate a hardware device or accept files from the outside. The drivers have to be all listed in Project.toml, but only one can be loaded, because they all define and export the same function names. How do I do this?
I tried @less using, to see the source code of using, but the REPL hung and I had to type ^C. I tried @less using(), but got an error because using is not a function. Is there a function I can call to load a module, whose name is not known until runtime, in the same way as using?
I suspect there might be better solutions, but you could use eval for this:
If this is main.jl
module A1
export foo
foo(x) = x
end
module A2
export foo
foo(x) = 2x
end
moduleName = Symbol(ARGS[1])
@eval using .$(moduleName)
println(foo(1))
You would get
$ julia main_test.jl A1
1
$ julia main_test.jl A2
2
Using @eval comes with the usual caveats of execution any code you pass to it from the outside of course (whatever you read from the config file).
Depending on if you need the list of drivers/plugins to be expandable from the outside, you could just check if the runtime value of the plugin name is in a list of allowed names or something like that.
But if I understand this correctly, the @eval using $() expression would at least error if you try to interpolate anything that’s not just a symbol into it. So it should be reasonably restrictive already.
Note the dot . in my example since I’m using modules defined directly in Main. In your case it would probably be @eval using $(...) without the leading ..
PS: If your list of possible plugins is fixed in advance, you can also just write an if statement
if moduleName == :A1
using .A1
elseif ...
...
end
which would avoid @eval altogether. Of course it’s not as concise, but might be enough in your situation and is quite straightforward to understand. For some reason I thought that this doesn’t work, but I can’t recall why… after all, conditional loading of packages is also what is done by some people in the startup.jl file, e.g. something like
I think you should always load all the drivers and solve this with dispatch. Here is how this could look
Have on abstract type for the plugins e.g.
# module structure optional but good style and helps show the dependencies in this example
module DriversBase
# export interface functions
export load, end
abstract type AbstractDriver end
# also define the functions the drivers implement
# so the driver can add their dispatches
# also it's nice to see the whole interface together in one file
"""
load(::AbstractDriver)
Set up the driver
"""
function load end
"""
dostuff(::AbstractDriver, msg::String)
Do stuff with the message.
"""
function dostuff end
end #module
Have each driver/plugin define a type and then add their methods to the aforementioned functions:
module FooDriver
# import DriversBase module from parent
using ..DriversBase
# export this driver's type
export FooDriver
# note we don't need to reexport the functions because they are already exported by Drivers module
# type for dispatch
struct FooDriver <: DriversBase.AbstractDriver end
## implement the functions
function DriversBase.load(::FooDriver)
# ...
end
function DriversBase.dostuff(::FooDriver, msg)
# ...
end
end #module
Pull this together into a package/module
module MyDriverCollection
export load, dostuff
export FooDriver #, BarDriver, ...
include("drivers_base.jl")
using DriversBase
include("foo.jl")
using FooDriver
# and so on
end # module
Then all you need to do to switch drivers is write function that return one of these type:
function decide_driver(path)
# parse file and return appropriate type
return FooDriver()
end
Note: this decision is inherently type unstable so be sure to use a function barrier to contain it properly, i.e. do something like this:
function main()
# ...
driver = decide_driver()
do_everything_else(driver) # 1 runtime dispatch but then type stable again
end
Also disclaimer: I wrote this up on my phone, so minor mistakes are possible. But overall this approach should work. Feel free to ask if it doesn’t work out-of-the-box and you have trouble getting it right
Doesn’t the module name have to be listed in Project.toml for using to succeed?
It isn’t, but for a particular installation, the driver is generally fixed, though the software may need to be upgraded.
Type of what? All the drivers export startDriver(), justStopDriver(), and waitDriverFinish(), and there’s nothing obvious for it to be a type of.
There’s no need or way to switch drivers once the program’s running.
It’ll be at least a few weeks. I have one half-written driver and am going to start another tonight, and it’ll be next month at the earliest before I can buy hardware to write a hardware driver.
In general, yes. But without knowing more about your code, it’s hard to make accurate predictions. My guess is everything will be fine as long as you hit the “top-level” of your main module after dynamically loading the plugins and before your other functions execute. But I’m not even sure if using ... will create the same type of problems you would have when you try to call a function defined in @eval directly (before returning to the top-level).
Sure (or the module is defined in your code somewhere). That’s why I think that @eval using ... will error for anything that doesn’t parse to a valid “using” expression and it’s quite safe to use in this situation.
All that being said, I think @abraemer’s approach probably makes more sense here. It requires a bit of extra code, but has the additional benefit that other people (including you) can define new drivers/plugins later independently of your main package. Even if you don’t need to switch the driver dynamically, it might be convenient for testing or for writing new drivers to have a function that can turn a driver on/off.
You just create a new type representing the driver itself. This will allow these three functions to dispatch on the driver type (by which Julia will automatically choose the desired driver for you without the need of a “dynamic using expression”.
E.g. the driver called Diver1 implements the methods of this function specific to that driver
And similarly for all other drivers. The argument driver will likely not be used inside the method definition, but that’s fine.
You still have to carry around the driver explicitly for this to work and be performant. But you could also make a global variable that holds the “current driver” and define a fallback method like
