How do I compute a multiclass F-score using MLJ?

Specific Domains Machine Learning
5

If you want to access the per class weighted FScore in the example above by @ablaom,

julia> m_no_avg = MulticlassFScore(average=no_avg)
MulticlassFScore(β = 1.0,average = MLJBase.NoAvg(),return_type = LittleDict)

julia> class_w = LittleDict('a' => 0.1, 'b' => 0.4, 'c' => 0.5)
LittleDict{Char, Float64, Vector{Char}, Vector{Float64}} with 3 entries:
  'a' => 0.1
  'b' => 0.4
  'c' => 0.5

julia> f1_no_avg = m_no_avg(y1, y2, class_w)
LittleDict{String, Float64, Vector{String}, Vector{Float64}} with 3 entries:
  "a" => 0.06
  "b" => 0.0
  "c" => 0.0

julia> m(y1, y2, class_w)  # By default, macro_avg is used and notice that its same as mean(f1_no_avg)
0.02

MLJBase considers this family of multiclass scores to be of:

  • micro_avg → M(ulticlass)TP, MTN… are computed across classes and then the MRecall, MFScore… are computed
  • macro_avg → MTP, MTN… are computed per class and MRecall, MFScore… are also computed per class and averaged value is returned
  • no_avg → MTP, MTN… are computed per class and per class MRecall, MFScore are returned

Further the weighted scores are supported for both average=macro_avg and average=no_avg to suit our broader design of package and stay flexible (most scores return vector, as it’s convenient to check per-class values and apply aggregation if necessary).

As class info is not available with micro_avg, it’s promoted to macro_avg whenever weights are passed.

2 Likes