# How to visualise the structure of the decision tree built by MLJ?

**URL:** <https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946>\
**Category:** Machine Learning\
**Created:** [November 11, 2019, 9:53am UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946 "2019-11-11T09:53:14Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [November 11, 2019, 9:53am UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/1 "2019-11-11T09:53:14Z")

</div>

```julia
using RDatasets, DataFrames
iris = dataset("datasets", "iris")

using MLJ # using the MLJ framework
using MLJModels # loads the modesl MLJ can use e.g. linear regression, decision tree
tree_model = @load DecisionTreeClassifier verbosity=1
y, X = unpack(iris, ==(:Species), !=(:Species))
tree_machine = machine(tree_model, X, y)
fit!(tree_machine)

using DecisionTree
print_tree(tree_machine.model)

```

Normally with a DecisionTree model I can print it using `print_tree` but no so one fitted with MLJ.

The error is odd because `typeof(tree\_machine.model) ==

```julia
┌ Info: A model type "DecisionTreeClassifier" is already loaded. 
│ No new code loaded. 
└ @ MLJModels C:\Users\RTX2080\.julia\packages\MLJModels\5Qzge\src\loading.jl:43
┌ Info: Training Machine{DecisionTreeClassifier} @ 1…95.
└ @ MLJ C:\Users\RTX2080\.julia\packages\MLJ\BEVGY\src\machines.jl:141
┌ Info: Not retraining Machine{DecisionTreeClassifier} @ 1…95.
│ It appears up-to-date. Use `force=true` to force retraining.
└ @ MLJ C:\Users\RTX2080\.julia\packages\MLJ\BEVGY\src\machines.jl:148
MethodError: no method matching print_tree(::Machine{DecisionTreeClassifier})
Closest candidates are:
  print_tree(!Matched::Nothing) at C:\Users\RTX2080\.julia\packages\DecisionTree\y42n2\src\scikitlearnAPI.jl:390
  print_tree(!Matched::DecisionTreeRegressor) at C:\Users\RTX2080\.julia\packages\DecisionTree\y42n2\src\scikitlearnAPI.jl:389
  print_tree(!Matched::DecisionTree.DecisionTreeClassifier) at C:\Users\RTX2080\.julia\packages\DecisionTree\y42n2\src\scikitlearnAPI.jl:388
  ...

Stacktrace:
 [1] top-level scope at In[74]:14

```

---

<div class="post-metadata">

**Author:** ![tlienart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tlienart/32/7640_2.png) [@tlienart](https://discourse.julialang.org/u/tlienart)\
**Post date:** [November 16, 2019, 4:23pm UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/2 "2019-11-16T16:23:39Z")

</div>

In the doc I believe it’s explained that the machine is just a wrapper for the model (a container with just the hyperparameters) and essentially the “fitresults” (whatever is learned during the fitting process that needs to be passed on to the predict or transform method).

So in this case if you want to recuperate the actual _trained tree_, please have a look at `tree_machine.fitresult`; IIRC the first element of the tuple is the tree that you want. This is also returned maybe more transparently by `fitted_params(tree_machine)` as a namedtuple.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [November 17, 2019, 1:33am UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/3 "2019-11-17T01:33:55Z")

</div>

> [@tlienart](#):
>
> So in this case if you want to recuperate the actual _trained tree_ , please have a look at `tree_machine.fitresult` ; IIRC the first element of the tuple is the tree that you want

That’s correct. Thanks.

```julia
print_tree(tree_machine.fitresult[1])

print_tree(fitted_params(tree_machine)[1])

```

I must admit I only tried to skim over the docs, I found [this passage](https://alan-turing-institute.github.io/MLJ.jl/stable/adding_models_for_general_use/#The-fitted_params-method-1)

> The fitted\_params method  
> A fitted\_params method may be optionally overloaded. It’s purpose is to provide MLJ access to a user-friendly representation of the learned parameters of the model (as opposed to the hyperparameters). They must be extractable from fitresult.

After reading it, I wouldn’t have figured out that that’s where I can get the tree model object. So having a post here that people can more easily find the answer is valuable.

---

<div class="post-metadata">

**Author:** ![tlienart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tlienart/32/7640_2.png) [@tlienart](https://discourse.julialang.org/u/tlienart)\
**Post date:** [November 17, 2019, 8:16am UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/4 "2019-11-17T08:16:32Z")

</div>

Great, we still have a lot to do in order to clarify how to do the many things that users would want to do with an ML toolbox and having people trying things out and telling us where they find things un-intuitive is very helpful!

Generally the [tutorials](https://alan-turing-institute.github.io/MLJTutorials/) is the good first place to get started; though there’s still need for a fair bit of work there as well of course.

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [November 17, 2019, 8:10pm UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/5 "2019-11-17T20:10:08Z")

</div>

Thanks for commenting on your experience. For the record, you can also get the tree printed by increasing the verbosity level of the fit:

```julia
julia> X, y = @load_iris
((sepal_length = [5.1, 4.9, 4.7, 4.6, 5.0, 5.4, 4.6, 5.0, 4.4, 4.9 … 6.7, 6.9, 5.8, 6.8, 6.7, 6.7, 6.3, 6.5, 6.2, 5.9], sepal_width = [3.5, 3.0, 3.2, 3.1, 3.6, 3.9, 3.4, 3.4, 2.9, 3.1 … 3.1, 3.1, 2.7, 3.2, 3.3, 3.0, 2.5, 3.0, 3.4, 3.0], petal_length = [1.4, 1.4, 1.3, 1.5, 1.4, 1.7, 1.4, 1.5, 1.4, 1.5 … 5.6, 5.1, 5.1, 5.9, 5.7, 5.2, 5.0, 5.2, 5.4, 5.1], petal_width = [0.2, 0.2, 0.2, 0.2, 0.2, 0.4, 0.3, 0.2, 0.2, 0.1 … 2.4, 2.3, 1.9, 2.3, 2.5, 2.3, 1.9, 2.0, 2.3, 1.8]), CategoricalArrays.CategoricalString{UInt8}["setosa", "setosa", "setosa", "setosa", "setosa", "setosa", "setosa", "setosa", "setosa", "setosa" … "virginica", "virginica", "virginica", "virginica", "virginica", "virginica", "virginica", "virginica", "virginica", "virginica"])

julia> mach = machine(@load(DecisionTreeClassifier), X, y)

julia> fit!(mach, verbosity=2)
[ Info: Training Machine{DecisionTreeClassifier} @ 1…47.
Feature 3, Threshold 2.45
L-> 1 : 50/50
R-> Feature 4, Threshold 1.75
    L-> Feature 3, Threshold 4.95
        L-> Feature 4, Threshold 1.65
            L-> 2 : 47/47
            R-> 3 : 1/1
        R-> Feature 4, Threshold 1.55
            L-> 3 : 3/3
            R-> Feature 3, Threshold 5.449999999999999
                L-> 2 : 2/2
                R-> 3 : 1/1
    R-> Feature 3, Threshold 4.85
        L-> Feature 2, Threshold 3.1
            L-> 3 : 2/2
            R-> 2 : 1/1
        R-> 3 : 43/43
Machine{DecisionTreeClassifier} @ 1…47

julia> fitted_params(mach)
(tree_or_leaf = Decision Tree
Leaves: 9
Depth: 5,
 encoding = Dict("virginica"=>0x03,"setosa"=>0x01,"versicolor"=>0x02),)

```

The correct way for the user to access the learning parameters is to use `fp=fitted_params(mach)`. In this case `fp.encoding` tells you how to decode the printed tree levels. If you wanted to print without refitting, you could do `DecisionTree.print_tree(fp.tree_or_leaf)`.

In general there are two ways to inspect outcomes of training: `fitted_params(mach)` returns a named-tuple representing (in as user-friendly form as possible) the actual learned parameters, while `report(mach)` returns everything else. In the case of DecisionTree, one could expand the report to include a method that prints the tree. (In the `fit` method in “MLJModels/src/DecisionTree.jl” change `report = (classes_seen=classes_seen,)` to

```julia
report = (classes_seen=classes_seen, 
               print_tree= () -> DecistionTree.print_tree(tree, model.display_depth))

```

Or something along those lines. Pull request welcome!

---

<div class="post-metadata">

**Author:** ![BVPs](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bvps/32/2190_2.png) [@BVPs](https://discourse.julialang.org/u/BVPs)\
**Post date:** [October 23, 2020, 7:38pm UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/6 "2020-10-23T19:38:29Z")

</div>

I am wondering whether we can plot the learned decision tree model within MLJ like the following cite suggests [Visualize a Decision Tree in 4 Ways with Scikit-Learn and Python | MLJAR](https://mljar.com/blog/visualize-decision-tree/) (via Python version of scikitlearn) instead of simply displaying it as ASCII characters as `print_tree` does.

---

<div class="post-metadata">

**Author:** ![jeremiedb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeremiedb/32/29150_2.png) [@jeremiedb](https://discourse.julialang.org/u/jeremiedb)\
**Post date:** [November 16, 2020, 8:05pm UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/7 "2020-11-16T20:05:41Z")

</div>

Latest [EvoTrees.jl](https://github.com/Evovest/EvoTrees.jl) version (0.5.2), which is integrated with MLJ, introduces a basic tree visualization:

```julia
plot(model, 3) # second argument refers to the ith tree of the model

```

 ![image](https://global.discourse-cdn.com/julialang/original/3X/6/c/6cd09e44cfcc01b62fef6c17a5e69dd150005796.png)

---

<div class="post-metadata">

**Author:** ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)\
**Post date:** [March 24, 2022, 2:32pm UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/8 "2022-03-24T14:32:38Z")

</div>

Cool! Can the same visualization be applied to trees from DecisionTree.jl?

---

<div class="post-metadata">

**Author:** ![jeremiedb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeremiedb/32/29150_2.png) [@jeremiedb](https://discourse.julialang.org/u/jeremiedb)\
**Post date:** [March 26, 2022, 4:01pm UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/9 "2022-03-26T16:01:45Z")

</div>

The plot recipe is unfortunately specific to EvoTrees structure. It would require some adaptations for other tree base models.

---

<div class="post-metadata">

**Author:** ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)\
**Post date:** [March 27, 2022, 6:42pm UTC](https://discourse.julialang.org/t/how-to-visualise-the-structure-of-the-decision-tree-built-by-mlj/30946/10 "2022-03-27T18:42:08Z")

</div>

I suppose decision trees will be around for a while, possibly longer than EvoTrees.jl and DecisionTree.jl. Some AbstractDescisionTree interface, which would define visualizations, would be pretty awesome.