# MLJ confusion\_matrix() - MethodError

**URL:** <https://discourse.julialang.org/t/mlj-confusion-matrix-methoderror/46856>\
**Category:** Machine Learning\
**Tags:** question, package\
**Created:** [September 18, 2020, 4:15pm UTC](https://discourse.julialang.org/t/mlj-confusion-matrix-methoderror/46856 "2020-09-18T16:15:04Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [September 18, 2020, 4:15pm UTC](https://discourse.julialang.org/t/mlj-confusion-matrix-methoderror/46856/1 "2020-09-18T16:15:04Z")

</div>

I am trying to make a RF predictor of a categorical variable (3 levels), and then look at how well the model works. I mostly followed the steps in the [Getting Started](https://alan-turing-institute.github.io/MLJ.jl/stable/getting_started/) section of the docs - here’s a MWE:

```julia
using DataFrames
using MLJ
using DecisionTree
using Random

df = DataFrame(a = categorical(rand(['a', 'b', 'c'], 100)), t = repeat(["test", "train"], inner=50), x1=rand(100), x2=rand(100))
train = findall(x-> x == "train", df.t)
test = findall(x-> x == "test", df.t)

y, X = unpack(df, ==(:a), x-> x in [:x1, :x2])

tree_model = MLJ.@load DecisionTreeClassifier verbosity=1
tree = machine(tree_model, X, y)

MLJ.fit!(tree, rows=train)
yhat = MLJ.predict(tree, X[test,:])

cross_entropy(yhat, y[test]) |> mean # this works

MLJ.confusion_matrix(yhat, y[test]) # this doesn't

```

That last gives me

```julia
ERROR: MethodError: no method matching confusion_matrix(::MLJBase.UnivariateFiniteVector{Multiclass{3}, Char, UInt32, Float64}, ::CategoricalVector{Char, UInt32, Char, CategoricalValue{Char, UInt32}, Union{}})
Closest candidates are:
  confusion_matrix(::AbstractVector{var"#s887"} where var"#s887"<:CategoricalValue, ::AbstractVector{var"#s886"} where var"#s886"<:CategoricalValue; rev, perm, warn) at /home/kevin/.julia/packages/MLJBase/uKzAz/src/measures/confusion_matrix.jl:64

```

I assumed that this would work given the [function signature](https://alan-turing-institute.github.io/MLJ.jl/stable/performance_measures/#MLJBase.confusion_matrix), though the text

> Computes the confusion matrix given a predicted `ŷ` with categorical elements and the actual `y`

and the error lead me to think I may have to convert the `UnivariateFinite{Multiclass{3}, Char, UInt32, Float64}` of `yhat` to an actual categorical array, but it’s not imediately clear how to do that. The probabilities in `yhat` all seem to be `1.0` or `0.0`, so it seems like this should be straightforward.

I naively tried `categorical(yhat)`, and that actually runs, but the categorical array of `UnivariateFinite` has a different order, so I’m assuming that’s not actually correct:

```julia
julia> levels(categorical(yhat))
3-element Vector{UnivariateFinite{Multiclass{3}, Char, UInt32, Float64}}:
 UnivariateFinite{Multiclass{3}}(a=>0.0, b=>1.0, c=>0.0)
 UnivariateFinite{Multiclass{3}}(a=>1.0, b=>0.0, c=>0.0)
 UnivariateFinite{Multiclass{3}}(a=>0.0, b=>0.0, c=>1.0)

julia> levels(y[test])
3-element Vector{Char}:
 'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)
 'b': ASCII/Unicode U+0062 (category Ll: Letter, lowercase)
 'c': ASCII/Unicode U+0063 (category Ll: Letter, lowercase)

```

So - is there a straightforward way to convert the array of `UnivariateFinite` to a categorical array?

---

<div class="post-metadata">

**Author:** ![samuel\_okon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuel_okon/32/10285_2.png) [@samuel\_okon](https://discourse.julialang.org/u/samuel_okon)\
**Post date:** [September 18, 2020, 5:52pm UTC](https://discourse.julialang.org/t/mlj-confusion-matrix-methoderror/46856/2 "2020-09-18T17:52:54Z")

</div>

> [@kevbonham](#):
>
> ```julia
> MLJ.confusion_matrix(yhat, y[test]) 
> 
> ```

`MLJ.confusion_matrix(mode.(yhat), y[test])` should do the trick. `UnivariateFiniteArray` object is an `AbstractVector` of `UnivariateFinite` (which is basically a distribution) so we have to take `mode`

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [September 18, 2020, 6:56pm UTC](https://discourse.julialang.org/t/mlj-confusion-matrix-methoderror/46856/3 "2020-09-18T18:56:40Z")

</div>

Ah, great!

FYI - formatting is a bite weird in your answer - guessing you have a stray ``` somewhere 🙂

---

<div class="post-metadata">

**Author:** ![samuel\_okon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuel_okon/32/10285_2.png) [@samuel\_okon](https://discourse.julialang.org/u/samuel_okon)\
**Post date:** [September 18, 2020, 7:07pm UTC](https://discourse.julialang.org/t/mlj-confusion-matrix-methoderror/46856/4 "2020-09-18T19:07:58Z")

</div>

Yeah. Fixed that

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [September 18, 2020, 7:31pm UTC](https://discourse.julialang.org/t/mlj-confusion-matrix-methoderror/46856/5 "2020-09-18T19:31:36Z")

</div>

The backtick placement still seems to be slightly off… 😂

---

<div class="post-metadata">

**Author:** ![samuel\_okon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuel_okon/32/10285_2.png) [@samuel\_okon](https://discourse.julialang.org/u/samuel_okon)\
**Post date:** [September 18, 2020, 7:39pm UTC](https://discourse.julialang.org/t/mlj-confusion-matrix-methoderror/46856/6 "2020-09-18T19:39:33Z")

</div>

Don’t mind my clumsiness