# Error with multiple extensions - related to \_\_init\_\_()?
**URL:** https://discourse.julialang.org/t/error-with-multiple-extensions-related-to-init/108196
**Category:** General Usage
**Tags:** package-extensions, \_\_init\_\_
**Created:** [December 30, 2023, 8:38pm UTC](https://discourse.julialang.org/t/error-with-multiple-extensions-related-to-init/108196 "2023-12-30T20:38:32Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@johnomotani](https://discourse.julialang.org/u/johnomotani)
#### Post date: [December 30, 2023, 8:38pm UTC](https://discourse.julialang.org/t/error-with-multiple-extensions-related-to-init/108196/1 "2023-12-30T20:38:32Z")
Why is ` __init__ ()` for a package called during precompilation of package extensions? I thought ([from the docs](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation)) that ` __init__ ()` would only be called when the module was loaded with `using` or `import?
The reason for the question is that I came across confusing errors when trying to precompile a package I am developing that has two extensions. I am trying to load the extensions whenever the needed dependencies are installed (without the user having to do `using DependencyA`, etc., explicitly) by having something like
```julia
function __init__ ()
try
Base.requires(Main, DependencyA)
catch
end
try
Base.requires(Main, DependencyB)
catch
end
end
```
In the main module of the package. It seems that the ` __init__ ()` function of the main module of the package is being called during precompilation of both of the extensions. My current guess is that the error is something like a race condition in package precompilation - the error only happens when both extensions have been modified and so need to be precompiled at the same time.
I didn’t understand the error message, as ` __precompile__ (false)` is not declared anywhere in my code. [The `Calling moment_kinetics. __init__ ()` output is from a print statement in the ` __init__ ()` function of the main `moment_kinetics` package that I added to see when it was being called].
```julia
Precompiling project...
✓ moment_kinetics
2 dependencies successfully precompiled in 16 seconds. 176 already precompiled.
1 dependency precompiled but a different version is currently loaded. Restart julia to access the new version
1 dependency had output during precompilation:
┌ moment_kinetics → manufactured_solns_ext
│ Calling moment_kinetics. __init__ ()
│ Calling moment_kinetics. __init__ ()
│ ┌ Warning: Module manufactured_solns_ext with build ID ffffffff-ffff-ffff-0000-1447d6c33442 is missing from the cache.
│ │ This may mean manufactured_solns_ext [246d111d-28e3-525f-a3fa-d7908d2bc032] does not support precompilation but is imported by a module that does.
│ └ @ Base loading.jl:1942
│ ┌ Error: Error during loading of extension manufactured_solns_ext of moment_kinetics, use `Base.retry_load_extensions()` to retry.
│ │ exception =
│ │ 1-element ExceptionStack:
│ │ Declaring __precompile__ (false) is not allowed in files that are being precompiled.
│ │ Stacktrace:
│ │ [1] _require(pkg::Base.PkgId, env::Nothing)
│ │ @ Base ./loading.jl:1946
│ │ [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│ │ @ Base ./loading.jl:1806
│ │ [3] #invoke_in_world#3
│ │ @ Base ./essentials.jl:921 [inlined]
│ │ [4] invoke_in_world
│ │ @ Base ./essentials.jl:918 [inlined]
│ │ [5] _require_prelocked
│ │ @ Base ./loading.jl:1797 [inlined]
│ │ [6] _require_prelocked
│ │ @ Base ./loading.jl:1796 [inlined]
│ │ [7] run_extension_callbacks(extid::Base.ExtensionId)
│ │ @ Base ./loading.jl:1289
│ │ [8] run_extension_callbacks(pkgid::Base.PkgId)
│ │ @ Base ./loading.jl:1324
│ │ [9] run_package_callbacks(modkey::Base.PkgId)
│ │ @ Base ./loading.jl:1158
│ │ [10] __require_prelocked(uuidkey::Base.PkgId, env::String)
│ │ @ Base ./loading.jl:1813
│ │ [11] #invoke_in_world#3
│ │ @ Base ./essentials.jl:921 [inlined]
│ │ [12] invoke_in_world
│ │ @ Base ./essentials.jl:918 [inlined]
│ │ [13] _require_prelocked(uuidkey::Base.PkgId, env::String)
│ │ @ Base ./loading.jl:1797
│ │ [14] macro expansion
│ │ @ Base ./loading.jl:1784 [inlined]
│ │ [15] macro expansion
│ │ @ Base ./lock.jl:267 [inlined]
│ │ [16] __require(into::Module, mod::Symbol)
│ │ @ Base ./loading.jl:1747
│ │ [17] #invoke_in_world#3
│ │ @ Base ./essentials.jl:921 [inlined]
│ │ [18] invoke_in_world
│ │ @ Base ./essentials.jl:918 [inlined]
│ │ [19] require(into::Module, mod::Symbol)
│ │ @ Base ./loading.jl:1740
│ │ [20] include
│ │ @ Base ./Base.jl:495 [inlined]
│ │ [21] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│ │ @ Base ./loading.jl:2216
│ │ [22] top-level scope
│ │ @ stdin:3
│ │ [23] eval
│ │ @ Core ./boot.jl:385 [inlined]
│ │ [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│ │ @ Base ./loading.jl:2070
│ │ [25] include_string
│ │ @ Base ./loading.jl:2080 [inlined]
│ │ [26] exec_options(opts::Base.JLOptions)
│ │ @ Base ./client.jl:316
│ │ [27] _start()
│ │ @ Base ./client.jl:552
│ └ @ Base loading.jl:1295
└
```
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [December 31, 2023, 3:31am UTC](https://discourse.julialang.org/t/error-with-multiple-extensions-related-to-init/108196/2 "2023-12-31T03:31:48Z")
> [@johnomotani](#):
>
> Why is ` __init__ ()` for a package called during precompilation of package extensions? I thought ([from the docs](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation)) that ` __init__ ()` would only be called when the module was loaded with `using` or `import?
Do your extensions import the package?
> [@johnomotani](#):
>
> My current guess is that the error is something like a race condition in package precompilation
I’m not certain, but if your extensions load the package which then loads them in ` __init__ ` via `Base.require`, it seems circular enough to cause problems at precompilation.
---
### Author:  [@johnomotani](https://discourse.julialang.org/u/johnomotani)
#### Post date: [December 31, 2023, 10:32am UTC](https://discourse.julialang.org/t/error-with-multiple-extensions-related-to-init/108196/3 "2023-12-31T10:32:32Z")
@Benny the extensions do import some function(s) from the package (which I guess is enough to load the package) - don’t extensions have to do that in order to do anything at all?
Strangely, adding the extension-triggering dependencies in the opposite order (i.e. something like `Pkg.add(DependencyB); Pkg.add(DependencyA)` instead of `Pkg.add(DependencyA); Pkg.add(DependencyB)`) seems to have made the error go away. Still very confused about what was causing it, so worried it might come back…
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [December 31, 2023, 9:13pm UTC](https://discourse.julialang.org/t/error-with-multiple-extensions-related-to-init/108196/4 "2023-12-31T21:13:48Z")
> [@johnomotani](#):
>
> don’t extensions have to do that in order to do anything at all?
It’s their intended purpose, but I had to ask because you seemed surprised `moment_kinetics` was loaded by the extensions and because you’re already working outside the intended purpose. Extensions are intended to be conditionally loaded depending on what packages had been imported, not just installed, because source code imports are much easier to track than whether `add`ing an arbitrary package also installs an extension dependency through some long dependency chain (why we compute Manifest.toml from Project.toml’s). Extensions aside, I’m also uncomfortable with the package loading other packages conditionally on their installation instead of making them dependencies, in which case the extension’s code would just be part of the package. I prefer conditions to be fully explicit or else varied very little like the operating system; project environments are neither.
> [@johnomotani](#):
>
> seems to have made the error go away. Still very confused about what was causing it, so worried it might come back…
Even if the error went away I wouldn’t trust that there isn’t a silent problem, especially with what seems like repeated loading (`Base.require` is documented to force reload), which could explain the “a different version is currently loaded” part. It’s still not clear what is happening, e.g. missing and mismatching names or nicknames (`DependencyA`), an absence of full protocols to replicate or evade the error, which involved modules import each other and in what order exactly, lacking precompilation reports for each add, misleading error message. I wonder if there is a way to log or print precompilation or load order more plainly.
---
### Author:  [@johnomotani](https://discourse.julialang.org/u/johnomotani)
#### Post date: [January 1, 2024, 2:15pm UTC](https://discourse.julialang.org/t/error-with-multiple-extensions-related-to-init/108196/5 "2024-01-01T14:15:53Z")
Thanks @Benny! I think I’m starting to understand better. It makes sense I guess that **init** () is called when `using MainPackage` (or equivalent) is evaluated in the extensions when they are precompiled. I think I was a bit mislead by the [docs for modules](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation) which say about ` __init__ ()` that “Effectively, you can assume it will be run exactly once in the lifetime of the code” so I was confused by the multiple calls - I guess this is one of the cases that comes under ‘Effectively’ that people shouldn’t normally need to worry about.
Maybe it would help to expand a bit on the actual use case for doing what I’m doing. The main package is a simulation code, and I want to do something like: if `NCDatasets` is installed, then the user can set an option to request output in NetCDF files. But if the user doesn’t need NetCDF-format output, I don’t want to require having `NCDatasets` installed. Actually the bigger issue is my other optional dependency, `Symbolics`. I could live with always installing `NCDatasets` even if it’s not needed, but `Symbolics` is a very heavy dependency - it takes a long time to precompile, and adds a few minutes to the time it takes to build a system image (at least, it did on julia-1.9.x, this seems to be a lot better on 1.10.0!). Again though, the features that use `Symbolics` are called if the user sets certain options in an input file. The simulation code is usually run in batch jobs on HPC clusters rather than interactively, so to use extensions in the expected way, a bunch of different scripts would need to know whether NCDatasets and/or Symbolics are installed, load them if they are, and then load the simulation package. It seems simpler to handle that logic in one place.
So in the end what I want is something like ‘if NCDatasets is in Project.toml, load the extension MyNetCDFExt’. I can appreciate that allowing this in a way that’s compatible with the package that wants to do something like this just being a dependency of a dependency of something else, so none of the relevant packages need to be in Project.toml at all, could be challenging. I guess I’m thinking it would be nice to be able to do something like
```julia
pkg> Foo[OptionalFeature1,OptionalFeature2]
```
Or have in a Project.toml
```julia
[deps]
Foo[OptionalFeature1] = "xxxxxxxxxxxxxx"
```
And have the package load some extension-like things that provide extra features (but might have extra dependencies). That’s probably a complicated addition to the package management though!
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [January 1, 2024, 5:40pm UTC](https://discourse.julialang.org/t/error-with-multiple-extensions-related-to-init/108196/6 "2024-01-01T17:40:09Z")
> [@johnomotani](#):
>
> I think I was a bit mislead by the [docs for modules](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation)
Those needed updating, even before v1.9 introduced extensions, and the documentations on extensions aren’t entirely clear about the conditions for installation and precompilation. I still don’t really understand the exact process myself, so I’ve only done what I’ve seen to work.
> [@johnomotani](#):
>
> So in the end what I want is something like ‘if NCDatasets is in Project.toml, load the extension MyNetCDFExt’.
This is not a feature and just doesn’t seem like it helps reproducibility. Imports are supposed to be how you control what is loaded, e.g. `import moment_kinetics, NCDatasets` to load `moment_kinetics`, `NCDatasets`, then `MyNetCDFExt`. The intended way for a package to “automatically” load another package is an import statement in its source code, making the 2nd package its dependency. I haven’t heard of any context where importing a package implicitly imports a separately installed package, even if it’s a specified weak dependency that may be automatically installed (not so in Julia). It seems possible but you need the conditional imports to occur after all installs, which I’m not sure can happen smoothly even in an ideal install order, something that is hard to reproduce and usually avoided by package managers. Maybe you can dodge these problems with a `--compiled-modules=no` process to disable precompilation, forcing package code to evaluate fresh in the process.