Precompilation error when evaling into a module in the __init__ function

General Usage
,
1

Is this allowed?

module PackageProblem

function __init__()
    @eval begin
        struct NewType
            a::Float64
        end

        const b = NewType(1.0)
    end
end

The package loads fine

julia> using PackageProblem

julia> PackageProblem.b
PackageProblem.NewType(1.0)

but Aqua.jl gives this error message:

ERROR: LoadError: InitError: Evaluation into the closed module `PackageProblem` breaks incremental compilation because the side effects will not be permanent. This is likely due to some other module mutating `PackageProblem` with `eval` during precompilation - don't do this.

You get a similar error if you create another package, pkg2, and use the package manager to add PackageProblem as a dependency to pkg2.

(pkg2) pkg> add ..\PackageProblem\\
 ...
Precompiling project...
  ✗ pkg2
  1 dependency successfully precompiled in 2 seconds
  1 dependency errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

julia> err
...
PkgPrecompileError: The following 1 direct dependency failed to precompile:...
ERROR: LoadError: InitError: Evaluation into the closed module `PackageProblem` breaks incremental compilation because the side effects will not be permanent. This is likely due to some other module mutating `PackageProblem` with `eval` during precompilation - don't do this.

If you want your package usable as a dependency is it forbidden to add new types to it in the __init__ function? Or is there a way to do the eval that doesn’t run afoul of precompilation rules?

If it is forbidden then would it be okay to have another function that adds types to the package like this:

function user_init()
    @eval PackageProblem begin ...(type defs here)... end
end

which the user calls after the package has been loaded? Or does this also mess up precompilation, perhaps in a way that Aqua.jl doesn’t detect?

In this MWE it obviously is more sensible to define the type and const outside of the __init__ function. But, for my application the types and consts can only be determined when the package is loaded so they either have to be defined in the __init__ function or afterward in the user_init function.

2

I really don’t know. The docs warn about “Calling eval to cause a side-effect in another module”, but people have frequently stumbled into precompilation issues with eval calls from and to the same module in the last several years. As far as I’ve gathered, the actual restriction is eval into a module that is not the module precompiling currently, and since __init__ is intended to execute strictly after its own module has completed its precompilation, I can’t imagine and haven’t ran into a case where eval in __init__ can work. It’s logical in a broader sense; the significant changes to a module should occur during its own precompilation phase, even if a subsequent change would only occur immediate after and once (which isn’t known to the language, other modules can manually call __init__ many times with possibly diverging results due to side effects).

In interactive usage, there aren’t any precompilation restrictions if no package is precompiling. RuntimeGeneratedFunctions.jl currently makes users call an init (not __init__) in each module that imports it to eval a const, a struct, a @generated function for functionality, and it works just fine interactively.

But if they’re going to call it in their own precompiling package, then you have the same problem as __init__ again. RuntimeGeneratedFunctions demands manual specification of modules to work with precompilation.

What are you trying to do and why? If names in general (not just const) only depend on a package being loaded, then it sounds like it could just be part of the package, as you admit for the MWE. It also sounds like you intend this to be a dependency for end-users, and it goes against the concept of dependencies to encourage them to tweak things that they’re not developing directly.

3

This is for the package AGFFileReader, which downloads AGF glass files for optical design. The files are publicly available for download at manufacturer’s websites. But, the files may be copyrighted and the manufacturers may object to them being bundled with my package. It’s a legal hassle I want to stay clear of.

Consequently, the end user has to download the files and get them compiled into Julia modules and types.

4

I think that’s an XY problem. The actual issue is that you’re trying to make a package’s evaluated source code depend on external state, in this case files manually downloaded from manufacturer websites. Changes to the websites, the available files, or what the user provides would theoretically change what such a package does, and that goes against a package’s purpose as a reproducible, distributable, and precompiled module. The early move away from build.jl was for this broader reason, too.

I don’t know a solution to pull off what you intend, so I encourage you to make a new topic for this, and with an example closer to what you’re doing (are you generating code from those files? That’s important to mention). I do have a couple parting thoughts: 1) I really doubt __init__ is what you need, and it is possible to generate and precompile code in a package module’s body, 2) if those downloaded files form a fixed and fairly stable set, then you could generate the source code in advance and include it in a version of your package. The project that generates the code should be stored in the repository and document how and when it was used, but it wouldn’t need to be ran by any of your users.