Problem using Overrides.toml

Hi all,

I’m trying to follow the instructions at 8. Artifacts · Pkg.jl to override the libcxxwrap-julia binaries (so I can use my local development version), using the following Overrides.toml file:

[3eaa8342-bff7-56a5-9981-c04077f7cee7]
libcxxwrap_julia = "/Users/user/src/build/libcxxwrap-julia/lib/libcxxwrap_julia.dylib"
libcxxwrap_julia_stl = "/Users/user/src/build/libcxxwrap-julia/lib/liblibcxxwrap_julia_stl.dylib"
libjlcxx_containers = "/Users/user/src/build/libcxxwrap-julia/lib/libjlcxx_containers.dylib"
libexcept = "/Users/user/src/build/libcxxwrap-julia/lib/libexcept.dylib"
libextended = "/Users/user/src/build/libcxxwrap-julia/lib/libextended.dylib"
libfunctions = "/Users/user/src/build/libcxxwrap-julia/lib/libfunctions.dylib"
libhello = "/Users/user/src/build/libcxxwrap-julia/lib/libhello.dylib"
libbasic_types = "/Users/user/src/build/libcxxwrap-julia/lib/libbasic_types.dylib"
libinheritance = "/Users/user/src/build/libcxxwrap-julia/lib/libinheritance.dylib"
libparametric = "/Users/user/src/build/libcxxwrap-julia/lib/libparametric.dylib"
libpointer_modification = "/Users/user/src/build/libcxxwrap-julia/lib/libpointer_modification.dylib"
libtypes = "/Users/user/src/build/libcxxwrap-julia/lib/libtypes.dylib"

However, when trying to add or load the libcxxwrap_julia_jll package, that gives this error:

(CxxWrap) pkg> add libcxxwrap_julia_jll
  Updating registry at `~/.julia/registries/General`
┌ Warning: Some registries failed to update:
│     — `~/.julia/registries/General` — registry dirty
└ @ Pkg.Types /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1199
 Resolving package versions...
ERROR: Unable to automatically install 'libcxxwrap_julia' from '/Users/user/.julia/packages/libcxxwrap_julia_jll/LDRaF/Artifacts.toml'
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] #ensure_artifact_installed#42(::Pkg.BinaryPlatforms.Platform, ::Bool, ::typeof(Pkg.Artifacts.ensure_artifact_installed), ::String, ::Dict{String,Any}, ::String) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Artifacts.jl:873
 [3] (::Pkg.Artifacts.var"#kw##ensure_artifact_installed")(::NamedTuple{(:platform, :verbose),Tuple{Pkg.BinaryPlatforms.MacOS,Bool}}, ::typeof(Pkg.Artifacts.ensure_artifact_installed), ::String, ::Dict{String,Any}, ::String) at ./none:0
 [4] #ensure_all_artifacts_installed#43(::Pkg.BinaryPlatforms.Platform, ::Nothing, ::Bool, ::Bool, ::typeof(Pkg.Artifacts.ensure_all_artifacts_installed), ::String) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Artifacts.jl:919
 [5] #ensure_all_artifacts_installed at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:0 [inlined]
 [6] #download_artifacts#78(::Pkg.BinaryPlatforms.MacOS, ::Bool, ::typeof(Pkg.Operations.download_artifacts), ::Array{String,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:580
 [7] #download_artifacts at ./none:0 [inlined]
 [8] #download_artifacts#73(::Pkg.BinaryPlatforms.MacOS, ::Bool, ::typeof(Pkg.Operations.download_artifacts), ::Array{Pkg.Types.PackageSpec,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:570
 [9] (::Pkg.Operations.var"#kw##download_artifacts")(::NamedTuple{(:platform,),Tuple{Pkg.BinaryPlatforms.MacOS}}, ::typeof(Pkg.Operations.download_artifacts), ::Array{Pkg.Types.PackageSpec,1}) at ./none:0
 [10] #add#112(::Bool, ::Pkg.BinaryPlatforms.MacOS, ::typeof(Pkg.Operations.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Array{Base.UUID,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1017
 [11] #add at ./none:0 [inlined]
 [12] #add#25(::Bool, ::Pkg.BinaryPlatforms.MacOS, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:102
 [13] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:72
 [14] do_add!(::Dict{Symbol,Any}, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Symbol,Any}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/REPLMode.jl:505
 [15] #invokelatest#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Base.invokelatest), ::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:709
 [16] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:708
 [17] do_cmd!(::Pkg.REPLMode.Command, ::REPL.LineEditREPL) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/REPLMode.jl:412
 [18] #do_cmd#23(::Bool, ::typeof(Pkg.REPLMode.do_cmd), ::REPL.LineEditREPL, ::String) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/REPLMode.jl:391
 [19] do_cmd at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/REPLMode.jl:387 [inlined]
 [20] (::Pkg.REPLMode.var"#28#31"{REPL.LineEditREPL,REPL.LineEdit.Prompt})(::REPL.LineEdit.MIState, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Bool) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/REPLMode.jl:619
 [21] #invokelatest#1 at ./essentials.jl:709 [inlined]
 [22] invokelatest at ./essentials.jl:708 [inlined]
 [23] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/REPL/src/LineEdit.jl:2306
 [24] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/REPL/src/REPL.jl:1045
 [25] run_repl(::REPL.AbstractREPL, ::Any) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/REPL/src/REPL.jl:201
 [26] (::Base.var"#770#772"{Bool,Bool,Bool,Bool})(::Module) at ./client.jl:382
 [27] #invokelatest#1 at ./essentials.jl:709 [inlined]
 [28] invokelatest at ./essentials.jl:708 [inlined]
 [29] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:366
 [30] exec_options(::Base.JLOptions) at ./client.jl:304
 [31] _start() at ./client.jl:460

If I remove the Overrides.toml, it works but of course downloads the released version. Any ideas on how to fix this?

What does the Artifact file look like and how do you use it in the code?

AFAIU (but I haven’t used this much) is that you give a replacement path for a given artifact name. So if your artifact is called libcxxwrap you would have

[3eaa8342-bff7-56a5-9981-c04077f7cee7]
libcxxwrap = "/Users/user/src/build/libcxxwrap-julia/"

and then artifact"libcxxwrap" would give that path.

The “substructure” of the redirected artifact must thus correspond to the unredirected artifact (i.e you don’t override on a file level).

1 Like

I never played directly with override files, but my understanding is the same as Kristoffer. For JLL packages this means that you can only override the “prefix”

1 Like

Awesome, that worked, thanks. The artifact in question is the auto-generated libcxxwrap_julia jll package. The only minor remaining issue is that the library version is included in the file name, so I still have to dev the jll package to override that, but I think that is due to the way BinaryBuilder generates the package.