# MLJ TunedModel with imbalanced classes

**URL:** <https://discourse.julialang.org/t/mlj-tunedmodel-with-imbalanced-classes/63667>\
**Category:** General Usage\
**Tags:** question, package, mlj\
**Created:** [June 27, 2021, 10:22pm UTC](https://discourse.julialang.org/t/mlj-tunedmodel-with-imbalanced-classes/63667 "2021-06-27T22:22:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vishal\_Sharma1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vishal_sharma1/32/26553_2.png) [@Vishal\_Sharma1](https://discourse.julialang.org/u/Vishal_Sharma1)\
**Post date:** [June 27, 2021, 10:22pm UTC](https://discourse.julialang.org/t/mlj-tunedmodel-with-imbalanced-classes/63667/1 "2021-06-27T22:22:59Z")

</div>

The TunedModel interface supports parameter - weights but not class\_weights.  
For classification problems with highly imbalanced classes, we need tuned model measures working with class weights.  
Is it currently possible to use TunedModel with highly imbalanced data for classification

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [June 28, 2021, 3:51am UTC](https://discourse.julialang.org/t/mlj-tunedmodel-with-imbalanced-classes/63667/2 "2021-06-28T03:51:26Z")

</div>

Most of the measures in MLJ do not directly support class weights. The only measures that currently support class weights are the multiclass-classification measures, as we can see by running the following code snippet:

```julia
julia> measures() do m
           m.supports_class_weights
       end
8-element Vector{NamedTuple{(:name, :instances, :human_name, :target_scitype, :supports_weights, :supports_class_weights, :prediction_type, :orientation, :reports_each_observation, :aggregation, :is_feature_dependent, :docstring, :distribution_type), T} where T<:Tuple}:
 (name = MulticlassFScore, instances = [macro_f1score, micro_f1score, multiclass_f1score], ...)
 (name = MulticlassFalseDiscoveryRate, instances = [multiclass_falsediscovery_rate, multiclass_fdr], ...)
 (name = MulticlassFalseNegativeRate, instances = [multiclass_false_negative_rate, multiclass_fnr, multiclass_miss_rate, multiclass_falsenegative_rate], ...)
 (name = MulticlassFalsePositiveRate, instances = [multiclass_false_positive_rate, multiclass_fpr, multiclass_fallout, multiclass_falsepositive_rate], ...)
 (name = MulticlassNegativePredictiveValue, instances = [multiclass_negative_predictive_value, multiclass_negativepredictive_value, multiclass_npv], ...)
 (name = MulticlassPrecision, instances = [multiclass_positive_predictive_value, multiclass_ppv, multiclass_positivepredictive_value, multiclass_recall], ...)
 (name = MulticlassTrueNegativeRate, instances = [multiclass_true_negative_rate, multiclass_tnr, multiclass_specificity, multiclass_selectivity, multiclass_truenegative_rate], ...)
 (name = MulticlassTruePositiveRate, instances = [multiclass_true_positive_rate, multiclass_tpr, multiclass_sensitivity, multiclass_recall, multiclass_hit_rate, multiclass_truepositive_rate], ...)

```

However, many of the measures support sample weights:

```julia
julia> length(measures(m -> m.supports_weights))
26

```

So what you can do is manually create a weight vector `w` for the sample weights where the weight of each sample is determined by the true class of the observation and then pass `w` to the `weights` keyword argument of `TunedModel`. Of course you have to pick one of the measures that supports sample weights. Use `measures(m -> m.supports_weights)` to list all the measures that support sample weights.

By the way, you might have figured this out already, but don’t forget to set `resampling=StratifiedCV()` if you’re dealing with imbalanced classes. Actually, it’s a good idea to use [`StratifiedCV`](https://alan-turing-institute.github.io/MLJ.jl/stable/evaluating_model_performance/#MLJBase.StratifiedCV) for any classification problem.

Another option is to not use weights and instead use a measure that is less influenced by class imbalance, like `balanced_accuracy` or `auc`.