Let me try to describe what I’m seeing above in plain English.
XGBoost is a C++ project that uses OpenMP for parallelization. When XGDMatrixCreateFromMat is invoked, the method unsigned long long xgboost::SparsePage::Push<xgboost::data::DenseAdapterBatch>(xgboost::data::DenseAdapterBatch const&, float, int) is eventually called. This contains a potentially parallel for loop via OpenMP.
The creation of this parallel for loop is created by interacting with libstdc++ and libgomp. Julia provides both of these libraries as part of its CompilerSupportLibraries_jll:
Some update to this these libraries seems to causing XGBoost to free some memory that should not be freed. Recent updates bumped this the underlying gcc to version 12.
Another change is the disabling of thread local storage on Windows builds:
The issue is thus not directly with Julia itself but rather with libraries that it provides.