Jll package is pointing to an older artifacts file?

I am experimenting with BinaryBuilder.jl and jll packages. My Julia version is 1.8.4.

I built ghostbasil_jll which is now on version 0.0.11+0. I then install this package locally via ]add https://github.com/biona001/ghostbasil_jll.jl, and in the REPL I can verify that I am indeed on the latest version

(@v1.8) pkg> st ghostbasil_jll
Status `/home/groups/sabatti/.julia/environments/v1.8/Project.toml`
  [caeec347] ghostbasil_jll v0.0.11+0 `https://github.com/biona001/ghostbasil_jll.jl#main`

However, it seems the actual artifact file it is pointing to is an older version (v0.0.8):

julia> using ghostbasil_jll
julia> ghostbasil_jll.artifact_dir
"/home/groups/sabatti/.julia/artifacts/725b2904491b4c4433907e4d59539908e5937988"

since 725b2904491b4c4433907e4d59539908e5937988 is pointing to v0.0.8 according to my artifact file.

Why is ghostbasil_jll.artifact_dir not pointing to v0.0.11? More importantly, I need ghostbasil_jll.get_libghostbasil_wrap_path to point to the latest version since ghostbasil_jll is wrapping a C++ package and v0.0.8 has an issue. Would it be advised if I delete all artifact entries that are not the latest version?

I see the same tree hash in the artifacts of your 0.11 version:

So I don’t understand what’s the issue.

Ah, wait, do you append new builds to the Artifacts.toml file, instead of deleting and recreating it every time?

Thanks for the reply. Yesterday I tried releasing v0.0.12 before going to sleep, and the original issue I was having (i.e. git-tree-sha1 pointing to an older version) is gone, although it did not solve the problem I was having. I think I’ll open another thread for that.

In this case, I suspect the issue has to do with my build_tarballs.jl. Here is the one I used for v0.0.12:

# Note that this script can accept some limited command-line arguments, run
# `julia build_tarballs.jl --help` to see a usage message.
using BinaryBuilder, Pkg

name = "ghostbasil"
version = v"0.0.12"

# Collection of sources required to complete build
sources = [
    GitSource("https://github.com/biona001/ghostbasil.git", "50aef6a0f7810b2a8a41e6ee36079950fcf54313")
]

# Bash recipe for building across all platforms
script = raw"""
cd $WORKSPACE/srcdir
mkdir -p ghostbasil/julia/build
cd ghostbasil/julia/build

cmake \
    -DJulia_PREFIX=$prefix \
    -DCMAKE_INSTALL_PREFIX=$prefix \
    -DCMAKE_FIND_ROOT_PATH=$prefix \
    -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
    -DEigen3_DIR=$prefix/share/eigen3/cmake \
    -DCMAKE_BUILD_TYPE=Release \
    ../

make
make install

# install license
install_license $WORKSPACE/srcdir/ghostbasil/R/LICENSE.md
"""

# These are the platforms we will build for by default, unless further
# platforms are passed in on the command line
julia_versions = [
    v"1.8.0", v"1.8.1", v"1.8.2", v"1.8.3", v"1.8.4", v"1.8.5",
    v"1.9.0", v"1.9.1", v"1.9.2", v"1.9.3", v"1.9.4", 
]
working_platforms = [ # ghostbasil won't work on windows, and currently fails on mac
    Platform("x86_64", "linux"; libc = "glibc"),
    Platform("aarch64", "linux"; libc = "glibc"),
    Platform("powerpc64le", "linux"; libc = "glibc"),
    Platform("x86_64", "linux"; libc = "musl"),
    Platform("aarch64", "linux"; libc = "musl"),
]

# expand platforms to specify julia version explicitly
# see https://github.com/JuliaInterop/CxxWrap.jl/issues/395
platforms = Platform[]
for p in working_platforms, julia_version in julia_versions
    p["julia_version"] = string(julia_version)
    push!(platforms, deepcopy(p))
end
platforms = expand_cxxstring_abis(platforms)

# The products that we will ensure are always built
products = [
    LibraryProduct("libghostbasil_wrap", :libghostbasil_wrap)
]

# Dependencies that must be installed before this package can be built
dependencies = [
    Dependency("libcxxwrap_julia_jll"),
    BuildDependency("Eigen_jll"),
    BuildDependency("libjulia_jll")
]

# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, script, platforms, products, 
    dependencies; julia_compat="1.9", preferred_gcc_version = v"7.1.0")

To be clear, the problem is that you’re accumulating the list of artifacts in Artifacts.toml instead of re-creating the file from scratch for new builds, which means that Pkg will stop at the first artifact matching your platform. I don’t know what you’re doing exactly.

I’m not sure how to re-recreate the file from scratch for new builds? I have a build_tarballs.jl at /Users/biona001/.julia/dev/Yggdrasil/G/ghostbasil and I deploy to my local github via

julia build_tarballs.jl --debug --verbose --deploy="biona001/ghostbasil_jll.jl"

What should I have done instead?

Not sure, I’ve never seen that behaviour, try to clean up the products directory before doing a new build.

Unless I’m missing something you aren’t stepping over these lines (from_scratch=false)

but as far as I can tell the only way for from_scratch to be false is to step over

which means skip_build=true, which means you’re passing --skip-build

1 Like

I’m certain I never passed in --skip-build. Also, at some point, I did start deleting everything inside products before rebuilding. Do you recommend editing the Artifacts.toml file manually to remove all entries older than the latest version?