# Univariate feature selection

**URL:** <https://discourse.julialang.org/t/univariate-feature-selection/87414>\
**Category:** Machine Learning\
**Tags:** mlj\
**Created:** [September 17, 2022, 6:22pm UTC](https://discourse.julialang.org/t/univariate-feature-selection/87414 "2022-09-17T18:22:20Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [September 18, 2022, 9:07pm UTC](https://discourse.julialang.org/t/univariate-feature-selection/87414/2 "2022-09-18T21:07:37Z")

</div>

Thanks @lucasmsoares96 for giving MLJ a spin.

MLJ does not currently have much in the way of canned feature selection tools. However, you can provide the MLJ wrapper `TunedModel` with any list of models, and training that will pick the best model based on specified resampling strategy (eg, CV), and then train on all data.

As you probably realize, `FeatureSelector` just cuts your table down using user-specified features.

So here’s a demo of what I think you are looking for:

```julia
using MLJ
using Combinatorics
using Tables

X, y = @load_iris # table, vector
KNN = @iload KNNClassifier
knn = KNN()

features = Tables.columnnames(X)
selections = combinations(features, 2)

models = map(selections) do s
    FeatureSelector(features=s) |> knn
end

tmodel = TunedModel(models=models, resampling=CV(nfolds=4, rng=123), measure=log_loss)

# Training `tmodel` means choosing the model in `models` with the best cv score, and then
# retraining best model on all data

mach = machine(tmodel, X, y)
fit!(mach)
predict(mach, X)[1:2] # this prediction based on best model trained on all data

# You can also inspect the best model:
r = report(mach).best_model

julia> r.feature_selector.features
2-element Vector{Symbol}:
 :petal_length
 :petal_width

```

P.S. Maybe you want to change the title of your post. I had to scratch my head a bit. Maybe `CV-based feature selection` would be more informative??

---

_[View the full topic](https://discourse.julialang.org/t/univariate-feature-selection/87414)._