I added the following to my bisect script to resolve this
Pkg.add("PrecompileTools", io=devnull)
Pkg.add("Preferences", io=devnull)
using PrecompileTools, Preferences
set_preferences!(PrecompileTools, "precompile_workloads" => false; force=true)
The error is due to PrecompileTools not being supported until Fix package precompilation (PrecompileTools) (#57828) · JuliaLang/julia@c89b1ff · GitHub on 1.12.