# MLJ for XGBoost - extracting feature gain

**URL:** <https://discourse.julialang.org/t/mlj-for-xgboost-extracting-feature-gain/93518>\
**Category:** General Usage\
**Tags:** mlj\
**Created:** [January 25, 2023, 5:34pm UTC](https://discourse.julialang.org/t/mlj-for-xgboost-extracting-feature-gain/93518 "2023-01-25T17:34:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ivan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ivan/32/208442_2.png) [@Ivan](https://discourse.julialang.org/u/Ivan)\
**Post date:** [January 25, 2023, 5:34pm UTC](https://discourse.julialang.org/t/mlj-for-xgboost-extracting-feature-gain/93518/1 "2023-01-25T17:34:26Z")

</div>

I have used MLJ for XGBoost in the past, and was able to explore the feature importances doing the following:

```julia
using MLJ, DataFrames
@load XGBoostRegressor pkg=XGBoost
X = DataFrame(rand((0, 1), (200, 5)), :auto)
y = rand([true, false], 200)

model = MLJXGBoostInterface.XGBoostRegressor()
xgb = machine(model, X, y) |> fit!
y_hat = MLJ.predict(xgb, X)

f = fitted_params(xgb)
r = report(xgb)

# the report had all the following fields I could access:
begin
    gains = [i.gain for i in r[1]]
    covers = [i.cover for i in r[1]]
    freqs = [i.freq for i in r[1]]
    feats = [i.fname for i in r[1]]
end

```

However, this no longer works because it’s no longer part of `report(mach)`:

```julia
fieldnames(typeof(r))
> (:features,)

```

How can I now access the gain, cover and frequency of the different features?

PS: This is straightforward using the original XGBoost package, but I like using MLJ:

```julia
using XGBoost
b = xgboost((X, y))
c = importancereport(b)

```

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [January 25, 2023, 9:22pm UTC](https://discourse.julialang.org/t/mlj-for-xgboost-extracting-feature-gain/93518/2 "2023-01-25T21:22:03Z")

</div>

@Ivan Thanks for reporting this.

In the last breaking release of MLJXGBoostInterface those particular access points were indeed removed. However, MLJ now has a generic `feature_importance` accessor function you can call on machines wrapping supported models, and the MLJXGBoostInterface models are now supported.

Unfortunately, I just discovered a minor [bug](https://github.com/JuliaAI/MLJXGBoostInterface.jl/issues/33), so that only the classifier is currently working. Here’s the workflow in that case:

```julia
using MLJ
XGBoostClassifier = @load XGBoostClassifier pkg=XGBoost
X, y = @load_iris

model = XGBoostClassifier()
mach = machine(model, X, y) |> fit!

julia> feature_importances(mach)
4-element Vector{Pair{Symbol, Float32}}:
 :petal_length => 2.991818
  :petal_width => 1.3149351
  :sepal_width => 0.072732545
 :sepal_length => 0.042442977

```

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [January 26, 2023, 9:57pm UTC](https://discourse.julialang.org/t/mlj-for-xgboost-extracting-feature-gain/93518/3 "2023-01-26T21:57:36Z")

</div>

@Ivan When [this](https://github.com/JuliaRegistries/General/pull/76457) is merged (next hour) then you can update MLJXGBoostInterface to correct the bug.

---

<div class="post-metadata">

**Author:** ![Ivan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ivan/32/208442_2.png) [@Ivan](https://discourse.julialang.org/u/Ivan)\
**Post date:** [September 22, 2024, 2:07am UTC](https://discourse.julialang.org/t/mlj-for-xgboost-extracting-feature-gain/93518/4 "2024-09-22T02:07:16Z")

</div>

Sorry for reopening this… It seems like the model XGBoostClassifier is not supported anymore by `feature_importances` (calling `feature_importances(mach)` on the MWE of the solution of this post returns `Nothing`). What is the current recommended way of accessing this?

Also, how do I keep myself updated on these changes? I feel like I’m missing something obvious here!

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [September 22, 2024, 5:13am UTC](https://discourse.julialang.org/t/mlj-for-xgboost-extracting-feature-gain/93518/5 "2024-09-22T05:13:23Z")

</div>

Mmm. That’s strange. The code in my [previous comment](https://discourse.julialang.org/t/mlj-for-xgboost-extracting-feature-gain/93518/2) is working fine for me with latest versions:

```julia
(jl_62DF8o) pkg> st
Status `/private/var/folders/4n/gvbmlhdc8xj973001s6vdyw00000gq/T/jl_62DF8o/Project.toml`
  [a7f614a8] MLJBase v1.7.0
  [54119dfa] MLJXGBoostInterface v0.3.11

```

Can I suggest you open an issue at MLJXGBoostInterface.jl, with a minimum working example and output of `using Pkg; Pkg.status()`. Thanks.