# \[ANN\] NeuroTreeModels.jl - Differentiable tree-based models for tabular data

**URL:** <https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707>\
**Category:** Package Announcements\
**Tags:** announcement, machine-learning\
**Created:** [February 24, 2024, 7:56pm UTC](https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707 "2024-02-24T19:56:21Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![jeremiedb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeremiedb/32/29150_2.png) [@jeremiedb](https://discourse.julialang.org/u/jeremiedb)\
**Post date:** [February 24, 2024, 7:56pm UTC](https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707/1 "2024-02-24T19:56:21Z")

</div>

Announcement for the initial release of [NeuroTreeModels.jl](https://evovest.github.io/NeuroTreeModels.jl/dev/)

`NeuroTree` based models comprise a collection of differentiable trees, in an attempt get both the performance benefits of boosted tree methods and flexibility of gradient based learning. A more comprehensive description can be found in the doc’s design section: [NeuroTree - A differentiable tree operator for tabular data | NeuroTreeModels](https://evovest.github.io/NeuroTreeModels.jl/dev/design)

Comprehensive bencharmks have been run against XGBoost, LightGBM, CatBoost and EvoTrees on 6 datasets commonly used in publications of ML methods on tabular data. Results and code to reproduce are found at [MLBenchmarks.jl](https://github.com/Evovest/MLBenchmarks.jl).

NeuroTree share similarities with Yandex’s [Neural Oblivious Decision Ensemble](https://github.com/Qwicen/node). Key differences include:

- Full binary trees (rather than oblibious ones).
- Rely on a simple `NeuroTree` operator that behaves similarly to a `Dense` operator for tabular, 2D input data. Such operator can be composed like a Dense operator in Flux chains to compose more complex models, like stack of trees, or combination with any other operators.

---

<div class="post-metadata">

**Author:** ![liuyxpp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/liuyxpp/32/9870_2.png) [@liuyxpp](https://discourse.julialang.org/u/liuyxpp)\
**Post date:** [February 25, 2024, 3:03am UTC](https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707/2 "2024-02-25T03:03:29Z")

</div>

Thank you for the package and the benchmark results. I don’t know CatBoost before, whose performance looks even better and promising.

---

<div class="post-metadata">

**Author:** ![jeremiedb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeremiedb/32/29150_2.png) [@jeremiedb](https://discourse.julialang.org/u/jeremiedb)\
**Post date:** [April 21, 2024, 11:00pm UTC](https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707/3 "2024-04-21T23:00:20Z")

</div>

Some meaningful updates with the v1.3.0 relase:

- The kwarg `device` (`:cpu` / `:gpu`) is moved from `NeuroTreeRegressor` to `fit`. Same for `gpuID`.
- Removal of `outsize` argument.
- Introduction of `NeuroTreeClassififier`, respecting the MLJ interface
- It’s no longer need to specify the number of classes (formerly through the deprecated `outsize` kwarg). It’s automatically detected throuh the target variable number of levels.
- Classification tasks (using `NeuroTreeClassififier`) now require the target variable to be `Categorical`

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [October 12, 2024, 12:05am UTC](https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707/4 "2024-10-12T00:05:34Z")

</div>

Just saw the talk. Fantastic. I have been doing research into differentable tree modesl

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [October 12, 2024, 3:15am UTC](https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707/5 "2024-10-12T03:15:58Z")

</div>

> [@liuyxpp](#):
>
> CatBoost

I finally have some time and might restart effort on JLBoost.jl which in contrast to XGBoost and Cat Boost allows for experimentation better because the structure is more hackable and allows the user to tweak many aspects of the boosting process.

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [October 12, 2024, 6:36am UTC](https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707/6 "2024-10-12T06:36:34Z")

</div>

> [@xiaodai](#):
>
> Just saw the talk. Fantastic.

Which talk? Is there a link to video?

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [October 12, 2024, 7:05am UTC](https://discourse.julialang.org/t/ann-neurotreemodels-jl-differentiable-tree-based-models-for-tabular-data/110707/7 "2024-10-12T07:05:48Z")

</div>

[![](https://global.discourse-cdn.com/julialang/original/3X/8/0/80c6d2f03c99b0430ba56f70fe8bdccf0547becf.jpeg "NeuroTree - A differentiable tree operator for tabular data | Desgagne-Bouchard | JuliaCon 2024") ](https://www.youtube.com/watch?v=1kNbZJOs8H0)