XGBoost Broken for Mac Mojave and earlier - Initialization Error

General Usage
1

I haven’t used XGBoost in a while. Now when I try to ‘use’ it, I get all sorts of errors, even when I update, and build, etc., even after fresh install of Julia.

The difficulty seems to lie with c++ modules… Does anyone know of a version of xgboost that is written in pure Julia or doesn’t have this dependency? [I came across JuML but that seems to be dead]

┌  Info: Precompiling XGBoost [009559a3-9522-5dbb-924b-0b6ed2b22bb9]
└ @ Base loading.jl:1423
ERROR: LoadError: InitError: could not load library "/Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib"
dlopen(/Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib, 1): Symbol not found: ___cxa_deleted_virtual
  Referenced from: /Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libc++.1.dylib
 in /Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:117
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ XGBoost_jll ~/.julia/packages/XGBoost_jll/Ytq8S/src/wrappers/x86_64-apple-darwin.jl:10
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:768
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:854
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1097
  [8] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [9] require(into::Module, mod::Symbol)
...
 [15] top-level scope
    @ none:1
during initialization of module XGBoost_jll
in expression starting at /Users/davide/.julia/packages/XGBoost/D30Xd/src/XGBoost.jl:1
Failed to precompile XGBoost [009559a3-9522-5dbb-924b-0b6ed2b22bb9] to /Users/davide/.julia/compiled/v1.7/XGBoost/jl_YoEynb.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1466
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1410
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1120
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997
 [7] eval
   @ ./boot.jl:373 [inlined]
 [8] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
   @ Base ./loading.jl:1196

Does anyone have any ideas?. I wanted to use XGBoost for the class I teach, but I can’t if this sort of thing keeps happening.
Thanks for any help!
*** Update!! same error persists after fresh upgrade/install to Julia 1.8

2

You could try EvoTrees (https://github.com/Evovest/EvoTrees.jl) - that’s probably the closest approximation to XGBoost in pure Julia. I’ve also had intermittent issues with XGBoost - you might want to try a complete clean install of julia + XGBoost. Maybe you need to update something like your xcode developer tools or something. Seems like some library is missing.

1 Like
3

Thanks for that, I will try EvoTrees.
[About XGBoost] Thanks for your suggestion - I did a completely fresh install and no dice. I may be able to spend a day and fix my problem, but part of the issue is that I teach 300+ Undergrads and Postgrads, and I have to constantly defend setting Julia over R or Python to my superiors, and if the most commonly used Machine Learning algorithm (xgboost) fails with a fresh (standard) installation of Julia in a class of 200, I’m ‘dead meat’ !

Anyway, thanks.

4

I’m running Julia 1.9 (dev) on a Mac M1 and just installed XGBoost to a new package environment without difficulty. Perhaps try one of the nightly versions of 1.9, which has been quite stable for me in daily use.

Step-by-step 
$ cd Desktop

$ julia -e 'using Pkg; Pkg.generate("xg")'

$ cd xg

$ julia -e 'using Pkg; Pkg.activate("."); Pkg.instantiate()'

$ julia

julia> versioninfo()

Julia Version 1.9.0-DEV.1274
Commit a9d3f7bb35 (2022-09-03 06:24 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin21.6.0)
  CPU: 10 × Apple M1 Max
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.5 (ORCJIT, apple-m1)
  Threads: 1 on 8 virtual cores
Environment:
  JULIA_EDITOR = Emacs

]    ## switch to Pkg prompt

(xg) pkg> add XGBoost
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
   Installed XGBoost_jll ──── v1.6.1+0
   Installed LLVMOpenMP_jll ─ v14.0.4+0
   Installed XGBoost ──────── v1.5.2
  Downloaded artifact: XGBoost
  Downloaded artifact: LLVMOpenMP
    Updating `~/Desktop/xg/Project.toml`
  [009559a3] + XGBoost v1.5.2
    Updating `~/Desktop/xg/Manifest.toml`
  [692b3bcd] + JLLWrappers v1.4.1
  [21216c6a] + Preferences v1.3.0
  [009559a3] + XGBoost v1.5.2
  [1d63c593] + LLVMOpenMP_jll v14.0.4+0
  [a5c6f535] + XGBoost_jll v1.6.1+0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.2+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.0+0
  [14a3606d] + MozillaCACerts_jll v2022.2.1
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [bea87d4a] + SuiteSparse_jll v5.10.1+0
  [83775a58] + Zlib_jll v1.2.12+3
  [8e850b90] + libblastrampoline_jll v5.1.1+0
  [8e850ede] + nghttp2_jll v1.48.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
Precompiling project...
  4 dependencies successfully precompiled in 1 seconds. 7 already precompiled.
5

What version of macOS do you have? As written in the error message, it requires 10.15. There’s nothing we can possibly do about, that’s a requirement from the upstream XGBoost library.

6

Thanks - I hadn’t noticed that - well-spotted.

I have 10.14.6 - but only a few years old. Is there a way I can downgrade XGB to a compatible version? It used to work just fine with this OS. [Also, my students have laptops which are even older than mine]
Thanks again

7

You know, actually reading the error messages sometimes helps :slightly_smiling_face:

You can use XGBoost_jll@1.5.2.

If you have further complaints about XGBoost requiring macOS 10.15 you should direct them to XGBoost developers.

8

Thank you so much. I have already raised an issue on their repo - I will update that with the new info.

Regards

9

If you mean

that’s the wrong repository. As I’ve been trying to tell all this time it’s the upstream project GitHub - dmlc/xgboost: Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow which requires macOS 10.15. But I wouldn’t hold my breath about them doing anything about that.

1 Like
10

I tried that, restarted, and got the same error.

Thanks for highlighting the issue, though.

11

Sorry if I’m skeptical, but it looks to me rather unlikely you’ve got the same error since XGBoost_jll 1.5.2 was built for macOS 10.12.

12

And in

there is nothing specific to Julia, it’s the C++ library itself which requires macOS 10.15! There is no “Julia implementation” of XGBoost, only a Julia package calling into the C++ library.

13

Thanks - I would be grateful if you could please tell me where I went wrong, then:

First

using Pkg
Pkg.rm("XGBoost")
Pkg.add(Pkg.PackageSpec(; name="XGBoost", version="1.5.2"))

Seemed to go without a hitch. Then restart. Then:

using XGBoost

┌ Info: Precompiling XGBoost [009559a3-9522-5dbb-924b-0b6ed2b22bb9]
└ @ Base loading.jl:1423
ERROR: LoadError: InitError: could not load library "/Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib"
dlopen(/Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib, 1): Symbol not found: ___cxa_deleted_virtual
  Referenced from: /Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libc++.1.dylib
 in /Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:117
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ XGBoost_jll ~/.julia/packages/XGBoost_jll/Ytq8S/src/wrappers/x86_64-apple-darwin.jl:10
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:768
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:854
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1097
  [8] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [9] require(into::Module, mod::Symbol)
...
 [15] top-level scope
    @ none:1
during initialization of module XGBoost_jll
in expression starting at /Users/davide/.julia/packages/XGBoost/D30Xd/src/XGBoost.jl:1
Failed to precompile XGBoost [009559a3-9522-5dbb-924b-0b6ed2b22bb9] to /Users/davide/.julia/compiled/v1.7/XGBoost/jl_VyAmrv.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1466
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1410
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1120
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997
 [7] eval
   @ ./boot.jl:373 [inlined]
 [8] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
   @ Base ./loading.jl:1196

Thanks for your help so far (and any additional).

14

You get again the error on file

18efbf9f697f68d4d6b74e269856f743e6ce7d10 is the artifact of XGBoost_jll 1.6.1 for x86_64 macOS:

Hence you are not using XGBoost_jll 1.5.2.

15

You’re pinning the wrong package. I’ve been talking about XGBoost_jll, you pinned XGBoost.

16

Thanks, that seems to have worked.

17

An XGBoost project maintainer here.

There’s nothing upstream XGBoost that requires MacOS 10.15 specifically. In the Python package, we target MacOS 10.13+:

Which libxgboost.dylib the Julia package using? Is it using Homebrew’s package (xgboost — Homebrew Formulae) ?

18

Ok, it turned out XGBoost does require 10.15, but the good news is that it was unintentional and the fix was very simple.

New build of XGBoost_jll (v1.6.1+1) includes the fix, and it should work on all macOS systems supported by Julia (>= 10.10).

Thanks a lot @hcho3 for the quick help!

19

Thanks all! Does this mean my students may add XGBoost or must they specify XGBoost_jll ?
[not sure I understand the distinction here]

Thanks again

20

Installing XGBoost should just work now (the library provided by XGBoost_jll is installed automatically and you don’t need to pin it to a specific version anymore). BTW, I also just updated it to the latest version v1.6.2: New version: XGBoost_jll v1.6.2+0 by jlbuild · Pull Request #67690 · JuliaRegistries/General · GitHub.