# Issue with XGBoost.jl and LIBSVM.jl when Julia 1.8.4

**URL:** <https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396>\
**Category:** General Usage\
**Tags:** packages\
**Created:** [January 1, 2023, 9:43pm UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396 "2023-01-01T21:43:35Z")\
**Posts on this page:** 13\
**Page:** 3

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [April 29, 2023, 1:32am UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/42 "2023-04-29T01:32:17Z")

</div>

@vchuravy , I think you might be the only one who can solve this.

`std::call_once` is invoked to implement `llvm::call_once`:

> <https://github.com/JuliaLang/llvm-project/blob/046f90173567535397c97b2d00406c93ee4bc890/llvm/include/llvm/Support/Threading.h#L86-L91>

However, it seems it was compiled against a `std::mutex` without `_GLIBCXX_HAVE_TLS` defined on Windows.

> <https://github.com/gcc-mirror/gcc/blob/502051957444a2fc392dd973e978664e2cd0be2b/libstdc%2B%2B-v3/include/std/mutex#L850-L853>

Why does our libLLVM not have `_GLIBCXX_HAVE_TLS` defined on Windows?

Are these related?

- [[clang] [MinGW] Don't mark emutls variables as DSO local · llvm/llvm-project@0e4cf80 · GitHub](https://github.com/llvm/llvm-project/commit/0e4cf807aeaf54a10e02176498a7df13ac722b37)
- [c++ - `std::call_once` always segfaults on Clang 12 on Windows (when using libstdc++) - Stack Overflow](https://stackoverflow.com/questions/67262415/stdcall-once-always-segfaults-on-clang-12-on-windows-when-using-libstdc)

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [April 29, 2023, 10:53pm UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/43 "2023-04-29T22:53:26Z")

</div>

I am confused? What does libLLVM have to do with this problem? LibLLVM is not a dependency of libgomp?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [April 29, 2023, 11:14pm UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/44 "2023-04-29T23:14:15Z")

</div>

libLLVM was compiled against a GCC configured without thread local storage on Windows. Upon updating GCC to GCC 12 we found missing symbols (e.g. `_ZSt14__once_functor`) needed by libLLVM as documented in the following pull request.

> <https://github.com/JuliaLang/julia/pull/45582#issuecomment-1295810344>
>
> Ok, it looks like that configuring GCC for MinGW with \`--disable-tls\` brings \`st…d::\_\_once\_functor\` and the other missing symbols back:
> \`\`\`console
> \# nm libstdc++-6.dll 2\> /dev/null | grep -E '(D|T) .\*(\_|et)\_once\_(functor|mutexv)'
> 00000003bea83a80 D \_ZSt14\_\_once\_functor
> 00000003bea683d0 T \_ZSt16\_\_get\_once\_mutexv
> 00000003bea6b430 T \_ZSt23\_\_get\_once\_functor\_lockv
> 00000003bea6b5e0 T \_ZSt27\_\_set\_once\_functor\_lock\_ptrPSt11unique\_lockISt5mutexE
> \`\`\`
> 
> This was an early attempt, I need to polish the build and then try to push it here.

To restore those symbols, we had to explicitly disable thread local storage, in part due to an upstream change in GCC 12. However, by explicitly disabling thread local storage in GCC 12 we may have broken libgomp or perhaps how many other libraries (e.g. XGBoost) interact with libgomp. So far it appears that substituing a libgomp built with thread local storage appears to resolve the issue for those packages.

One potential solution, still unproven but complicated to test, is perhaps we should just enable GCC’s thread local storage on Windows, which actually does have native support for thread local storage. This in part would involve having to rebuild libLLVM against GCC with thread local storage enabled.

The specific questions to you, @vchuravy , are

1. Is there a reason we have disabled thread local storage for LLVM on Windows? (My guess is no. That’s just how CompilerSupportLibraries\_jll was built in the past)
2. Could we enable thread local storage support on Windows and still have libLLVM function correctly?

To move that in this direction, some additional work is needed to prove that thread local storage support is actually the issue, but that is the current lead suspect here.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [April 30, 2023, 12:31am UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/45 "2023-04-30T00:31:47Z")

</div>

> [@mkitti](#):
>
> The specific questions to you, @vchuravy , are
> 
> 1. Is there a reason we have disabled thread local storage for LLVM on Windows? (My guess is no. That’s just how CompilerSupportLibraries\_jll was built in the past)
> 2. Could we enable thread local storage support on Windows and still have libLLVM function correctly?

I don’t have answers for those question, my best guess is probably?  
It sounds like we compiled libLLVM against a libstdc++ from GCC and that baked in the fact that TLS was disabled. So yes a “simple” recompilation might work?

---

<div class="post-metadata">

**Author:** ![laborg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laborg/32/5474_2.png) [@laborg](https://discourse.julialang.org/u/laborg)\
**Post date:** [May 9, 2023, 7:52am UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/46 "2023-05-09T07:52:31Z")

</div>

Any news on this issue? 1.9.0 is imminent but I don’t think its fixed, right?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [May 9, 2023, 9:14am UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/47 "2023-05-09T09:14:49Z")

</div>

> [@laborg](#):
>
> 1.9.0 is imminent but I don’t think its fixed, right?

Still waiting for someone to understand _ **what** _ the issue is in order to be able to come up with a fix.

---

<div class="post-metadata">

**Author:** ![jd-foster](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jd-foster/32/35824_2.png) [@jd-foster](https://discourse.julialang.org/u/jd-foster)\
**Post date:** [May 10, 2023, 4:14am UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/48 "2023-05-10T04:14:20Z")

</div>

In the absence of understanding the precise issue, what do you think about the utility of bumping the version of the MinGW-w64 [release](https://www.mingw-w64.org/changelog/) from v7 (2019-11-10) to v11 (2023-04-29)?

The current line in [gcc\_sources.jl](https://github.com/JuliaPackaging/Yggdrasil/blob/7b666555d6a8ecfca77381e8e5c98f87c64ab0e0/0_RootFS/gcc_sources.jl#L258) seems to have been there since 2020 ([Final cleanups to GCCBootstrap shards and RootFS by staticfloat · Pull Request #1607 · JuliaPackaging/Yggdrasil · GitHub](https://github.com/JuliaPackaging/Yggdrasil/pull/1607)).

(Side note: seems like mingw-w64 v11 fixed the parallel build race condition: [[0\_RootFS] Add GCC 12 by giordano · Pull Request #4980 · JuliaPackaging/Yggdrasil · GitHub](https://github.com/JuliaPackaging/Yggdrasil/pull/4980/commits/71ce5a6a1d803f578b95e2d9878bd142d4c29134)

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [May 11, 2023, 4:14pm UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/49 "2023-05-11T16:14:05Z")

</div>

Send the pull request. We’ll probably want to do that eventually, I think.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [May 11, 2023, 4:24pm UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/50 "2023-05-11T16:24:57Z")

</div>

We generally use old toolchains for a reason (compatibility).

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [June 9, 2023, 12:02am UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/51 "2023-06-09T00:02:46Z")

</div>

I’m using [IDA](https://hex-rays.com/ida-pro/) to do debugging, and it’s assigning the critical error to different location:  
`_ZNSt12 __shared_ptrIN7xgboost10SparsePageELN9__ gnu_cxx12_Lock_policyE2EEC2ISaIS1_EJEEESt19_Sp_make_shared_tagRKT_DpOT0_.isra.751+0xE6`

Interestingly, this path goes through `00007FF92124323F	libgcc_s_seh-1.dll	libgcc_s_seh-1___emutls_get_address+1EF` again suggesting possible involvement of TLS.

```julia
Address	Module	Function
00007FF9670AF3D3	ntdll.dll	ntdll_RtlIsZeroMemory+A3
00007FF9670B812F	ntdll.dll	ntdll_RtlpNtSetValueKey+44F
00007FF9670BAEF5	ntdll.dll	ntdll_RtlpNtSetValueKey+3215
00007FF9670B8192	ntdll.dll	ntdll_RtlpNtSetValueKey+4B2
00007FF9670B847A	ntdll.dll	ntdll_RtlpNtSetValueKey+79A
00007FF966FD47B1	ntdll.dll	ntdll_RtlFreeHeap+51
00007FF9670BE101	ntdll.dll	ntdll_RtlpNtSetValueKey+6421
00007FF966FD5BF0	ntdll.dll	ntdll_RtlGetCurrentServiceSessionId+13A0
00007FF936086F29	libwinpthread-1.dll	libwinpthread-1_sem_destroy+79
00007FF92124323F	libgcc_s_seh-1.dll	libgcc_s_seh-1___emutls_get_address+1EF
00007FF8C966E21C	libstdc++-6.dll	libstdc++-6__Znwy+1C
00000000067C6036	xgboost.dll	_ZNSt12 __shared_ptrIN7xgboost10SparsePageELN9__ gnu_cxx12_Lock_policyE2EEC2ISaIS1_EJEEESt19_Sp_make_shared_tagRKT_DpOT0_.isra.751+0xE6
000000000698DE8A	xgboost.dll	xgboost::data::SimpleDMatrix::SimpleDMatrix<xgboost::data::DenseAdapter>(xgboost::data::DenseAdapter *,float,int)+0x27A
0000000006A41633	xgboost.dll	xgboost::DMatrix* xgboost::DMatrix::Create<xgboost::data::DenseAdapter>(xgboost::data::DenseAdapter *,float,int,std::__cxx11::basic_string<char,std::char_traits<char>,std::allocator<char>> const&)+0x33
0000000006744A9D	xgboost.dll	XGDMatrixCreateFromMat+0x8D

```

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/4/c41fba083ed4df9c2fd4b592fb71b7a74a340a5b.png)

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [June 11, 2023, 12:50am UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/52 "2023-06-11T00:50:26Z")

</div>

We need someone to show if the bug can recreated in C++ using the libraries shipprd with Julia. If we can show this is not specifically a Julia issue, we stand a better chance of getting help from upstreamm

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [June 11, 2023, 6:22pm UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/53 "2023-06-11T18:22:55Z")

</div>

Should be fixed by [[CompilerSupportLibraries\_jll] Upgrade to v1.0.5 by giordano · Pull Request #50135 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/50135)

---

<div class="post-metadata">

**Author:** ![matnbo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/matnbo/32/220166_2.png) [@matnbo](https://discourse.julialang.org/u/matnbo)\
**Post date:** [June 12, 2023, 6:37pm UTC](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396/54 "2023-06-12T18:37:21Z")

</div>

Many thanks to everybody that contributed to fix this difficult problem, amazing job greatly appreciated. 👍

[Previous page](https://discourse.julialang.org/t/issue-with-xgboost-jl-and-libsvm-jl-when-julia-1-8-4/92396.md?page=2)
