April 21, 2023, 6:35pm
While exploring a bit how packages and artifacts are stored locally, I came across a couple of
Apparently, this is not returned by
So the question(s) is quite simple, where does it come from? why it differs from the usual
Artifacts.toml? And shouldn’t it be returned by
NB: For the context, I was trying to implement a function that returns package sizes, source files and artifacts included.
Examples of packages with it:
I’ve explored Julia’s source code, but just found that:
const artifact_names = ("JuliaArtifacts.toml", "Artifacts.toml")
I don’t really understand how it all fits together, but this comment here is probably a good breadcrumb to follow - JLLs depended on by the stdlib are “special” and have unique handling as such.
@staticfloat would be the clear expert.
> Now it isn't?
> If there was a way to get the "real" GMP_jll (and
… similar for other JLLs in this PR), just for accessing the headers in it, that'd solve this final issue for me (in particular, it would not be necessary for me to load the "real" GMP_jll, just a way to get paths into it would suffice. Is something like that a possibility?
Yes! This is precisely why we bundle a `StdlibArtifacts.toml` file; so that "special" packages that are in the know can do these kinds of workarounds. In your case, you'll just need to do some artifacts shenanigans if you're on v1.6, where you find the `StdlibArtifacts.toml` file, select the appropriate `GMP` artifact within it, and then you can use your headers and whatnot as usual.
> Perhaps it would be a good idea to use something like JLLWrappers here, too, to avoid some of the repetition and ensure consistency. Indeed, perhaps one could adapt JLLWrapper itself?
I would have loved to, and indeed, the first cut at this did exactly that, but there are some issues:
* First, the JLLWrappers API has evolved a lot over the past few months. You've seen this (and helped a bit) so I knew that wholesale importing JLLWrappers into v1.6 wasn't quite a possibility yet. IMO, better to ship no API at all then to ship a broken one that we replace two months later in v1.7.
* Second, we don't have actual artifacts getting installed on v1.6 yet. That breaks a lot of assumptions in the JLLWrapper world. I would like for JLLWrappers to be smart enough to deal with `USE_SYSTEM_XXX=1` configurations where you just `dlopen("libfoo.so")`, alternate artifact location configurations (e.g. a `LibFoo_jll_debug=true` is set in your `LocalPreferences.toml`) or complete path overrides (e.g. the preference `LibFoo_jll_libfoo_path=/opt/special_julia_stuff/lib/libfoo.so.1.2` is set). But that API isn't here yet, and that's _really_ what we need in order for JLLs to be useful across all Julia configurations.
Ideally, it would all come together in one piece, but unfortunately that kind of thinking ends up with PRs that have thousands of lines changed at once, and are completely un-reviewable. So we have to do them in small batches, and when release time comes, I can only hold it back for so long. ;)
Unfortunately, this forces a bit of complexity onto you, but hopefully it's not _too_ too inconvenient to have some `v1.6`-specific workarounds.
April 22, 2023, 3:49pm
Thanks for the digging, this clarifies things a bit.
I understand part of the reason is explained later in the PR:
This seems to be to allows to
Pkg to include stdlib’s jll’s during the resolve process, while locking them to the version that ships with julia.
Confirmation would be appreciated, though
It’s actually not used by Pkg; it can be used by downstream tools such as BinaryBuilder. Pkg knows when things are stdlibs and excludes them from package resolution. There are very very few packages that use this file, but there are a few, which you can see here: