# How to dynamically import packages and run their functions from a setting file

**URL:** <https://discourse.julialang.org/t/how-to-dynamically-import-packages-and-run-their-functions-from-a-setting-file/119548>\
**Category:** General Usage\
**Created:** [September 18, 2024, 12:36pm UTC](https://discourse.julialang.org/t/how-to-dynamically-import-packages-and-run-their-functions-from-a-setting-file/119548 "2024-09-18T12:36:03Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [September 18, 2024, 12:36pm UTC](https://discourse.julialang.org/t/how-to-dynamically-import-packages-and-run-their-functions-from-a-setting-file/119548/1 "2024-09-18T12:36:03Z")

</div>

I am trying to create a set of packages where I have a main package that know nothing of subpackages but it call a certain function of them, and where the subpackages to use are defined in a configuration file.

So, I have a main package:

\*_MainPackage_

- `run()`

And several “optional” packages:

**OptionalPackage\_regionA**

- `foo!()`

**OptionalPackage\_regionB**

- `foo!()`

The user should be able to just use the main package and call the `run()` function:

```julia
using MainPackage
MainPackage.run()

```

Where the `run` function should do something like that:

- load a setting toml file and get `regions_to_run` (`["regionA","regionB"]`) and `settings` (a dict)
- 

```for
 if OptionalPackage_R in current environment
     import OptionalPackage_R
     run OptionalPackage_R.foo!(settings)
 else
     raise an error
 end

```

I am doing this because the different regions implement things in a different way, but I don’t want the main package to have a priori list of possible regions.

Perhaps with extensions, but there the user has first to type before the `run` `using OptionalPackage_regionA; using OptionalPackage_regionB` and then the main package, at least on its `Project.toml` file, needs to know about these regions.

Suggestions ?

---

<div class="post-metadata">

**Author:** ![mikmoore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikmoore/32/31109_2.png) [@mikmoore](https://discourse.julialang.org/u/mikmoore)\
**Post date:** [September 18, 2024, 4:38pm UTC](https://discourse.julialang.org/t/how-to-dynamically-import-packages-and-run-their-functions-from-a-setting-file/119548/2 "2024-09-18T16:38:50Z")

</div>

I don’t exactly understand what you want. Do you want to load packages conditionally, based on a parameter file? You might need `@eval` to do that (a symptom that there is probably a better way to accomplish your goals). You might find `using Pkg; Pkg.project().dependencies` to be useful for finding what is in your project (although this may not be the best way – this kind of introspection isn’t super-well supported because it’s not usually very useful). If you’re thinking about this, I’ll discourage it and say that there’s probably a better way. If you elaborate a little bit, someone can probably suggest it.

* * *

In any case, your `MainPackage` does not need to import, depend on, or even know about your optional packages. More idiomatically, your `MainPackage` might define an interface (a set of abstract types and/or functions) and your optional packages would extend it with their custom functionality. So your optional packages would depend on your main package (not so onerous when they intend to implement/extend its functionality), but not the other way around.

An example of the pattern I would propose is given in [this post](https://discourse.julialang.org/t/module-parameters-and-run-time-efficiency/119505/4). This is basically the same way any interface in Julia is extended (for example, how one implements a custom `AbstractArray`) except that in this case the dispatch is based on a token (that may hold no data) instead of some data-carrying object (like an `AbstractArray`).

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [September 19, 2024, 9:46am UTC](https://discourse.julialang.org/t/how-to-dynamically-import-packages-and-run-their-functions-from-a-setting-file/119548/3 "2024-09-19T09:46:31Z")

</div>

The difference with your approach and what I am trying to achieve is that in your approach it is the user of MainPackage/optional packages to make the call, while in my needs it is MainPackage itself that needs to make a call to the optional packages, where the name of the package is in some configuration file (as a string).

I think what I am looking for is a sort of session-specific Project/Manifest files.  
Normally Project/Manifest files are persistent, while I need a version that exists only for the time of a session.

---

<div class="post-metadata">

**Author:** ![mikmoore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikmoore/32/31109_2.png) [@mikmoore](https://discourse.julialang.org/u/mikmoore)\
**Post date:** [September 19, 2024, 2:55pm UTC](https://discourse.julialang.org/t/how-to-dynamically-import-packages-and-run-their-functions-from-a-setting-file/119548/4 "2024-09-19T14:55:33Z")

</div>

There isn’t a concept of a session-specific project file (that I’ve ever heard of). It just doesn’t really fit with Julia’s concept of a project. You can use `Pkg` to assemble one programmatically, I guess, but I don’t think you’re doing yourself any favors. Is it such a problem to have them all in one larger project and only utilize a subset in a given session? Why do they need to be loaded only conditionally?

Your desired ultra-simple `using MainPackage; MainPackage.run()` just isn’t practical (especially not without `@eval` but maybe even with it) without `MainPackage` being aware of what it can call.

Still lacking a clear understanding of exactly what you want, I will suggest two possibilities:

- Make `MainPackage` depend on all your subpackages. Use your `regions_to_run` to assemble a `Dict`/`Vector`/etc of functions/tokens that you’ll actually use.
- `MainPackage` does not depend on the subpackages, but the subpackages depend on `MainPackage`. The subpackages register themselves with `MainPackage` in their ` __init__ ()` by calling `MainPackage.register_functionality("regionA", functionality_for_A)` so that when `MainPackage` wants `"regionA"`, it uses `functionality_for_A` (a function, dispatch token, etc) to implement it. Your `regions_to_run` can be used to subselect the registered functionalities into an active list for the actual run.

In the latter version, you would instead call

```julia
using MainPackage
using PackageForRegionA, PackageForRegionB
MainPackage.run()

```

where `run()` will read the config file, assemble the region processing subset, and throw an error if an unrecognized processing mode is requested (e.g., if you request `"regionZ"` but you haven’t registered functionality for it – probably by `using PackageForRegionZ`).