Sounds like Memory leak with Julia 1.11's GC (discovered in SymbolicRegression.jl) · Issue #56759 · JuliaLang/julia · GitHub which was fixed by gc: improve mallocarrays locality by vtjnash · Pull Request #56801 · JuliaLang/julia · GitHub which is marked for backport.
Can you try nightly?