How do libraries compiled by PackageCompiler.jl interact with Preferences.jl?

General Usage
5

Well, originally I didn’t think it would be worth to create another example, especially after @staticfloat having already provided a great test repo. However, I then decided to create a proper MWE myself, and lo!, I was unable to reproduce my earlier findings. Since this basically means that most of my original complaints are gone, I am not sure if I should :tada: or :cry:.

Anyhow, here’s the MWE repo I created for testing using Preferences.jl with PackageCompiler.jl-generated libraries:

With this, I was able to make the following work:

  • Setting compile-time preferences by storing them in the LocalPreferences.toml located in the package_dir argument to PackageCompiler.create_library
  • Setting compile-time preferences by storing them in the Project.toml located in the package_dir argument to PackageCompiler.create_library
  • Setting run-time preferences by creating an appropriate LocalPreferences.toml in the library’s LOAD_PATH/DEPOT_PATH (i.e., in PREFIX/share/julia).
  • Setting run-time preferences by creating an appropriate Project.toml in the library’s LOAD_PATH/DEPOT_PATH (i.e., in PREFIX/share/julia).

What still does not work:

  1. Setting compile-time preferences in the library by setting them in the environment that invokes PackageCompiler.create_library (such that an existing package does not have to be modified in-source but can be loaded, e.g., from the default depot)
  2. Setting run-time preferences in the library without having to add files to the install prefix of the library (such that it can really be done at runtime)

Re 1: I feel like this should be doable, to respect the preferences set in the PackageCompiler.jl environment. Alternatively, maybe it would be possible to add a local_preferences argument that accepts a user-prepared LocalPreferences.jl file with settings to be respected in the compiled library? I am thinking about use cases such as MPI.jl, HDF5.jl etc. where you need to switch out the JLL-provided binaries for system-local binaries.

Re 2: Right now, because of these two lines, there is no way to modify the DEPOT_PATH or LOAD_PATH from outside. Maybe it would be an option to not force setting these environment variables if they have already been set by the user? Or, alternatively, to have an additional set of environment variables that, if set, are used for the depot and load path?

However, with respect to the ability to set compile-time preferences, most of my original troubles have been solved by providing a LocalPreferences.toml in the package dir. It is now less a matter of necessity but more of convenience.