Using measure in MLJ to evaluate binary classifier

New to Julia
,
2

You can make your categorical arrays ordered:

using MLJ

predStr = categorical(
    ["fast", "fast", "slow"];
    ordered = true,
    levels = ["slow", "fast"]
)

gtStr = categorical(
    ["slow", "fast", "slow"];
    ordered = true,
    levels = ["slow", "fast"]
)

julia> confusion_matrix(predStr, gtStr)
              ┌───────────────────────────┐
              │       Ground Truth        │
┌─────────────┼─────────────┬─────────────┤
│  Predicted  │    slow     │    fast     │
├─────────────┼─────────────┼─────────────┤
│    slow     │      1      │      0      │
├─────────────┼─────────────┼─────────────┤
│    fast     │      1      │      1      │
└─────────────┴─────────────┴─────────────┘

Note that according to the docstring for ConfusionMatrix, the first argument should be the predicted values and the second argument should be the true values.

Let’s compare the scientific types of an ordered and an unordered categorical array:

julia> scitype(predStr)
AbstractVector{OrderedFactor{2}}

julia> scitype(categorical(["a", "b"]))
AbstractVector{Multiclass{2}}

Also note that you can make categorical arrays with integers:

pred = categorical([1, 1, 0]; ordered=true)
gt = categorical([0, 1, 0]; ordered=true)

julia> confusion_matrix(pred, gt)
              ┌───────────────────────────┐
              │       Ground Truth        │
┌─────────────┼─────────────┬─────────────┤
│  Predicted  │      0      │      1      │
├─────────────┼─────────────┼─────────────┤
│      0      │      1      │      0      │
├─────────────┼─────────────┼─────────────┤
│      1      │      1      │      1      │
└─────────────┴─────────────┴─────────────┘