# \[ANN\] BetaML.jl.. yet an other (simple) Machine Learning Package

**URL:** <https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057>\
**Category:** Package Announcements\
**Tags:** package, announcement, machine-learning\
**Created:** [June 9, 2020, 10:14am UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057 "2020-06-09T10:14:25Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [June 9, 2020, 10:14am UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/1 "2020-06-09T10:14:25Z")

</div>

![BetaML_logo](https://sea2.discourse-cdn.com/julialang/images/transparent.png)

 ![microExample_white](https://global.discourse-cdn.com/julialang/original/3X/7/8/783168adf14f2466209a91bf5f0616809667864c.png)

Dear all,

I would like to announce the availability of “[BetaML](https://github.com/sylvaticus/BetaML.jl)” , the **Beta Machine Learning** toolkit, a package for Machine Learning algorithms and related utilities.

The toolkit is currently made of 4 modules. **Perceptron** includes the classical perceptron linear classifier, but also the non-linear kernel perceptron and the gradient-based Pegasus classifier. **Nn** implements easy-to-model Artificial Neural Networks (simple feed-forward only for the moment, but we plan to add support for convolutional layers, Recurrent Neural Network and LSTM ones). Note that automatic differentiation with `Zygote` is optional, you can pass your own derivative of the activation function if you wish (common ones are provided). **Clustering** has algorithms such as kmeans, Kmedoids and Expectation-Maximisation based on Gaussian Mixture Models (GMM). As the EM algorithm supports partially missing observations (observations with missing data only on some dimensions), it is used as backbone algorithm for collaborative filtering (recommendation systems). Finally **Utils** is a module implementing common functions as scaling, one-hot encoding, various kernels and distance metrics.

BetaML most likely has value only didactically, as the approaches are the “vanilla” ones, i.e. the simplest possible ones, and GPU is not supported. For “serious” machine learning work in Julia I would suggest to use either [Flux](https://fluxml.ai/) or [Knet](https://github.com/denizyuret/Knet.jl).

As the focus is mainly didactic, functions have pretty longer but more explicit names than usual.. for example the `Dense` layer is a " `DenseLayer` `"` , the `RBF` kernel is " `radialKernel` `"` , etc.

That said, Julia is a relatively fast language and most hard job is done in multithreaded functions or using matrix operations whose underlying libraries may be multithreaded, so it is reasonably fast for small exploratory tasks. Also it is already very flexible. For example, one can implement its own layer as a subtype of the abstract type `Layer` or its own optimisation algorithm as a subtype of `OptimisationAlgorithm` or even specify its own distance metric in the Kmedoids algorithm..

This repository started from implementing in the Julia language the concepts taught in the [MITX 6.86x - Machine Learning with Python: from Linear Models to Deep Learning](https://www.edx.org/course/machine-learning-with-python-from-linear-models-to) course, and theoretical notes describing most of these algorithms are available at the companion repository [GitHub - sylvaticus/MITx\_6.86x: Notes of MITx 6.86x - Machine Learning with Python: from Linear Models to Deep Learning](https://github.com/sylvaticus/MITx_6.86x).

Cheers,

Antonello Lobianco, [Bureau d’Economie Théorique et Appliquée](http://www.beta-umr7522.fr/) of Nancy & [AgroParisTech](http://agroparistech.fr)

References:

- main repository: [GitHub - sylvaticus/BetaML.jl: Beta Machine Learning Toolkit](https://github.com/sylvaticus/BetaML.jl)

- documentation: [Index · BetaML.jl Documentation](https://sylvaticus.github.io/BetaML.jl/dev)

- online runnable notebooks: [https://sylvaticus.github.io/BetaML.jl/dev/Notebooks.html](https://sylvaticus.github.io/BetaML.jl/dev/Notebooks.html)

(yep, the logo is inspired by a popular superhero…. the wish is that whenever we have a numerical problem, the Beta Machine Learning toolkit could come to the rescue with its superpowers! 🙂 🙂 🙂 )

This is a full example of multi-class classification of the Sepal dataset:

```julia
# Load Modules
using BetaML.Nn, DelimitedFiles, Random, StatsPlots # Load the main module and ausiliary modules
Random.seed!(123); # Fix the random seed (to obtain reproducible results)

# Load the data
iris = readdlm(joinpath(dirname(Base.find_package("BetaML")),"..","test","data","iris.csv"),',',skipstart=1)
iris = iris[shuffle(axes(iris, 1)), :] # Shuffle the records, as they aren't by default
x = convert(Array{Float64,2}, iris[:,1:4])
y = map(x->Dict("setosa" => 1, "versicolor" => 2, "virginica" =>3)[x],iris[:, 5]) # Convert the target column to numbers
y_oh = oneHotEncoder(y) # Convert to One-hot representation (e.g. 2 => [0 1 0], 3 => [0 0 1])

# Split the data in training/testing sets
ntrain = Int64(round(size(x,1)*0.8))
xtrain = x[1:ntrain,:]
ytrain = y[1:ntrain]
ytrain_oh = y_oh[1:ntrain,:]
xtest = x[ntrain+1:end,:]
ytest = y[ntrain+1:end]

# Define the Artificial Neural Network model
l1 = DenseLayer(4,10,f=relu) # Activation function is ReLU
l2 = DenseLayer(10,3) # Activation function is identity by default
l3 = VectorFunctionLayer(3,3,f=softMax) # Add a (parameterless) layer whose activation function (softMax in this case) is defined to all its nodes at once
mynn = buildNetwork([l1,l2,l3],squaredCost,name="Multinomial logistic regression Model Sepal") # Build the NN and use the squared cost (aka MSE) as error function

# Training it (default to SGD)
res = train!(mynn,scale(xtrain),ytrain_oh,epochs=100,batchSize=6) # Use optAlg=SGD (Stochastic Gradient Descent) by default

# Test it
ŷtrain = predict(mynn,scale(xtrain)) # Note the scaling function
ŷtest = predict(mynn,scale(xtest))
trainAccuracy = accuracy(ŷtrain,ytrain,tol=1) # 0.983
testAccuracy = accuracy(ŷtest,ytest,tol=1) # 1.0

# Visualise results
testSize = size(ŷtest,1)
ŷtestChosen = [argmax(ŷtest[i,:]) for i in 1:testSize]
groupedbar([ytest ŷtestChosen], label=["ytest" "ŷtest (est)"], title="True vs estimated categories") # All records correctly labelled !
plot(0:res.epochs,res.ϵ_epochs, ylabel="epochs",xlabel="error",legend=nothing,title="Avg. error per epoch on the Sepal dataset")

```

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/3/c3da6b5b38334f8b19c35f99ffc7fdb5afa044ac.png) ![image](https://global.discourse-cdn.com/julialang/original/3X/e/f/ef18d4b9888dc4a1d0a72c2adfc35c84458dc141.png)

PS: thanks to @kevbonham on topic [37198](https://discourse.julialang.org/t/my-own-feedforward-neural-network-library/37198):

> [@My own Feedforward Neural Network library slight\_smile](https://discourse.julialang.org/t/my-own-feedforward-neural-network-library/37198/2):
>
> Cool! Now toss it into a package, write some tests, and get it registered

It ended up that writing tests, doc, getting CI and registration has been almost as time consuming that writing the library itself, but a very rewarding experience !

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [June 9, 2020, 11:34am UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/2 "2020-06-09T11:34:52Z")

</div>

That sentence of motivation is now my largest contribution to Julia machine learning 😂.

Great work!

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [June 17, 2020, 2:39pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/3 "2020-06-17T14:39:32Z")

</div>

I see you use ReLU (and you have some other usual suspects). But it’s outdated, and the closest also fast seems to be PLU (feel free to copy my implementation there, and for others):

[https://github.com/onnx/onnx/pull/2575#issuecomment-645007473](https://github.com/onnx/onnx/pull/2575#issuecomment-645007473)

Mish seems to me the best activation function (and CELU and more I link to there also interesting).

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [June 17, 2020, 4:29pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/4 "2020-06-17T16:29:58Z")

</div>

Thank you, I added the celu function to master… I don’t feel the need to add too many activation functions as the user has the ability to choose whatever function she/he wants by just providing the `f` parameter in the layer constructor…

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [June 24, 2020, 7:17am UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/5 "2020-06-24T07:17:49Z")

</div>

**v0.2 is out.**

What’s new:

**Clustering: generic mixture support**

Added generic, user-specified Mixture support to the EM algorithm, with {Spherical,Diagonal,Full} Gaussian mixtures already implemented.

The support for missing data allows the EM algorithm to be used for [missing imputation](https://doi.org/10.1016/j.csda.2006.10.002) or collaborative filtering/reccomendation system (using the function `predictMissing`).

**Neural Networks: More default activation functions**

Although the user can provide its own activation function (and optionally its derivative to avoid using AD), we included the most recent activation functions (and their derivatives), namely `relu`, `elu`, `celu`, `plu`, `sigmoid`, `softmax`, `softplus`, `mish` (thanks to user @Palli).

**Utils: Various addition/improvements**

We added reverse scaling (in order to scale back the labels/output values), BIC and AIC criteria, `meanRelError` and the parameter `ignoreLabels` to the `accuracy` function in order to account for classification tasks where the label itself doesn’t matter, just its distribution (e.g. in unsupervised learning/clustering).  
In master you’ll find also PCA.

The documentation for v0.2 [is here](https://sylvaticus.github.io/BetaML.jl/v0.2/).

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [July 7, 2020, 6:56pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/6 "2020-07-07T18:56:42Z")

</div>

V0.2.2 [is out](https://github.com/sylvaticus/BetaML.jl)

What’s new (compared to v0.2.0):

**PCA Analysys**

You can now transform your data using PCA specifying either the number of dimensions you want to keep or the maximum error (variance) you are wiling to accept

**kmeans init strategy for em clustering**

The expectation-maximisation algorithm for fitting a Generative Mixture Models and cluster data/impute missing data can now be automatically initialised with the output of a kmeans clustering (just pass the parameter `initStrategy="kmeans"`.

**ADAM optimisation algorithm for neural networks**

In addition to the classical Stochastic Gradient Descent, we added the efficient ADAM, moment based optimiser. The implementation is the same as [in the paper where it is introduced](https://arxiv.org/pdf/1412.6980.pdf), with the difference that the learning rate can be expressed as a (user-provied) function of the epoch rather than being a constant (but we kept as default t → 0.001 as in the paper).  
The solution we chosen proved to be very flexible: adding a optimiser is just a matter of creating a struct that subclass `OptimisationAlgorithm` and implementing `singleUpdate!(θ,▽,optAlg::OptimisationAlgorithm;nEpoch,nBatch,nBatches,xbatch,ybatch)` and eventually `initOptAlg!(optAlg::OptimisationAlgorithm;θ,batchSize,x,y)`.

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [August 18, 2020, 3:45pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/7 "2020-08-18T15:45:05Z")

</div>

v0.3 [is out](https://github.com/sylvaticus/BetaML.jl)

What’s new in v0.3 (compared to 0.2.2):

- **Decision Trees / Random Forest** (BetaML.Trees)

- **Neural Networks** (BetaML.NN)

- **Utilitis** (BetaML.Utils)

- **Documentation**

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [August 18, 2020, 5:28pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/8 "2020-08-18T17:28:17Z")

</div>

Bdw, if someone would like to review the corresponding Open Source Software paper… The review is stock in the “pre-review” status, as the editors can’t find a reviewer…

[https://github.com/openjournals/joss-reviews/issues/2512](https://github.com/openjournals/joss-reviews/issues/2512)

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [August 19, 2020, 12:28am UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/9 "2020-08-19T00:28:16Z")

</div>

[https://github.com/xiaodaigh/awesome-ml-frameworks](https://github.com/xiaodaigh/awesome-ml-frameworks)

I have added BetaML.jl to the above list.

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [March 4, 2021, 6:34pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/10 "2021-03-04T18:34:23Z")

</div>

## BetaML v0.4.0 [is out](https://github.com/sylvaticus/BetaML.jl/releases/tag/v0.4.0)

What’s new in v0.4 (compared to 0.3):

- **Decision Trees / Random Forests** _(BetaML.Trees)_

- **Perceptron-like models** _(BetaML.Perceptron)_

- **Utilitis** _(BetaML.Utils)_

- **Clustering** _(Beta.Clustering)_

- **MLJ API**

- **Other**

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [April 19, 2021, 9:03pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/11 "2021-04-19T21:03:09Z")

</div>

## BetaML v0.5.0 [is out](https://github.com/sylvaticus/BetaML.jl/releases/tag/v0.5.0)

What’s new in v0.5 (compared to 0.4.1):

- **Documentation**

- **MLJ API**

- **Package reorganisation**

- **Stochasticity management**

- **Utilities** _(BetaML.Utils)_

- **Other**

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [July 7, 2023, 8:25pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/12 "2023-07-07T20:25:20Z")

</div>

BetaML v0.10.2 is out

New stuff include the GroupedLayer and ReplicatorLayer that allow to model multi-branches deep neural networks, like in the following architecture that is discussed [in this tutorial](https://sylvaticus.github.io/BetaML.jl/dev/tutorials/Multi-branch%20neural%20network/betaml_tutorial_multibranch_nn.html):

 ![multibranch_nn](https://global.discourse-cdn.com/julialang/original/3X/6/c/6c215b918a7db3f73603fcac47a15742dbc486c6.png)

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [December 29, 2023, 3:17pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/13 "2023-12-29T15:17:10Z")

</div>

v0.10.4 is out.

Main stuff compared to 0.10.2:

- (v0.10.3) general [`UniversalImputer`](https://sylvaticus.github.io/BetaML.jl/stable/Imputation.html#BetaML.Imputation.UniversalImputer) to impute (with repetitions) missing values using any supervised model (not necessarily from BetaML) that can be wrapped in a `m=Model(hp); fit!(m,x,y); yest = predict(m,x)` interface (specific imputers, like [`RFImputer`](https://sylvaticus.github.io/BetaML.jl/stable/Imputation.html#BetaML.Imputation.RFImputer), where already available in the [`Imputation`](https://sylvaticus.github.io/BetaML.jl/stable/Imputation.html) module)
- (v0.10.4) simple to use [`AutoEncoder`](https://sylvaticus.github.io/BetaML.jl/dev/Utils.html#BetaML.Utils.AutoEncoder) (and [`AutoEncoderMLJ`](https://sylvaticus.github.io/BetaML.jl/dev/Utils.html#BetaML.Utils.AutoEncoderMLJ)) model that follows the API `m=AutoEncoder(hp); fit!(m,x); x_latent = predict(m,x); x̂ = inverse_predict(m,x_latent)` . Users can optionally specify the number of dimensions to shrink the data (`outdims`), the number of neurons of the inner layers (`innerdims`) or the full details of the encoding and decoding layers and all the underlying NN options, but this remains optional.

I had looked a lot in the net, and I believe this is the easiest way to apply a AutoEncoder to reduce the dimensionality of some data, as the user doesn’t really need to deal with the underlying neural network

(note: the release will need a few minutes to go to the Julia public register. The MLJ wrapper model will need I believe manual approval from the MLJ team)

Examples

- Universalmputer

```julia
julia> using BetaML
julia> import DecisionTree
julia> X = [1.4 2.5 "a"; missing 20.5 "b"; 0.6 18 missing; 0.7 22.8 "b"; 0.4 missing "b"; 1.6 3.7 "a"]
6×3 Matrix{Any}:
 1.4 2.5 "a"
  missing 20.5 "b"
 0.6 18 missing
 0.7 22.8 "b"
 0.4 missing "b"
 1.6 3.7 "a"
julia> mod = UniversalImputer(estimator=[DecisionTree.DecisionTreeRegressor(),DecisionTree.DecisionTreeRegressor(),DecisionTree.DecisionTreeClassifier()], fit_function = DecisionTree.fit!, predict_function=DecisionTree.predict, recursive_passages=2)
UniversalImputer - A imputer based on an arbitrary regressor/classifier(unfitted)
julia> X_full = fit!(mod,X)
** Processing imputation 1
6×3 Matrix{Any}:
 1.4 2.5 "a"
 0.94 20.5 "b"
 0.6 18 "b"
 0.7 22.8 "b"
 0.4 13.5 "b"
 1.6 3.7 "a"

```

- AutoEncoder:

```julia
julia> using BetaML

julia> x = [0.12 0.31 0.29 3.21 0.21;
            0.22 0.61 0.58 6.43 0.42;
            0.51 1.47 1.46 16.12 0.99;
            0.35 0.93 0.91 10.04 0.71;
            0.44 1.21 1.18 13.54 0.85];

julia> m = AutoEncoder(outdims=1,epochs=400)
A AutoEncoder BetaMLModel (unfitted)

julia> x_reduced = fit!(m,x)
***
*** Training for 400 epochs with algorithm ADAM.
Training.. avg loss on epoch 1 (1): 60.27802763757111
Training.. avg loss on epoch 200 (200): 0.08970099870421573
Training.. avg loss on epoch 400 (400): 0.013138484118673664
Training of 400 epoch completed. Final epoch error: 0.013138484118673664.
5×1 Matrix{Float64}:
  -3.5483740608901186
  -6.90396890458868
 -17.06296512222304
 -10.688936344498398
 -14.35734756603212

julia> x̂ = inverse_predict(m,x_reduced)
5×5 Matrix{Float64}:
 0.0982406 0.110294 0.264047 3.35501 0.327228
 0.205628 0.470884 0.558655 6.51042 0.487416
 0.529785 1.56431 1.45762 16.067 0.971123
 0.3264 0.878264 0.893584 10.0709 0.667632
 0.443453 1.2731 1.2182 13.5218 0.842298

julia> info(m)["rme"]
0.020858783340281222

julia> hcat(x,x̂)
5×10 Matrix{Float64}:
 0.12 0.31 0.29 3.21 0.21 0.0982406 0.110294 0.264047 3.35501 0.327228
 0.22 0.61 0.58 6.43 0.42 0.205628 0.470884 0.558655 6.51042 0.487416
 0.51 1.47 1.46 16.12 0.99 0.529785 1.56431 1.45762 16.067 0.971123
 0.35 0.93 0.91 10.04 0.71 0.3264 0.878264 0.893584 10.0709 0.667632
 0.44 1.21 1.18 13.54 0.85 0.443453 1.2731 1.2182 13.5218 0.842298

```

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [January 25, 2024, 2:34pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/14 "2024-01-25T14:34:08Z")

</div>

[BetaML](https://github.com/sylvaticus/BetaML.jl) v0.11 is out

Release notes:

**Attention: many breaking changes in this version !!**

- **experimental** new `ConvLayer` and `PoolLayer` for convolutional networks. BetaML neural networks work only on CPU and even on CPU the convolutional layers (but not the dense ones) are 2-3 times slower than Flux. Still they have some quite unique characteristics, like working with any dimensions or not requiring AD in most cases, so they may still be useful in some corner situations. Then, if you want to help in porting to GPU… 😉 (I just got my first machine with usable GPU)
- Isolated MLJ interface models into their own `Bmlj` submodule
- Renamed many model in a congruent way
- Shortened the hyper-parameters and learnable parameters struct names
- Corrected many doc bugs
- Several bugfixes

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [January 25, 2024, 2:37pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/15 "2024-01-25T14:37:32Z")

</div>

Just curious, why don’t you use basic functionality such as conv layers or activations from [NNlib.jl](https://github.com/FluxML/NNlib.jl)?

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [January 25, 2024, 3:06pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/16 "2024-01-25T15:06:58Z")

</div>

I think the first answer is because it is fun 😉

I can also easily add what I am interested on, relying on what I have already wrote. For example, once the EM algorithm is there, I could write a regressor, not only a clusterer, with the RF I can implement an Imputer based on it, with NN I can implement an AutoEncoder…

While I agree that having many independent but interconnected specialised packages  
is often preferable, I think exploring the other route of having everything in once maneageable package could also be productive. Funny, it is 🙂

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [May 15, 2024, 8:47pm UTC](https://discourse.julialang.org/t/ann-betaml-jl-yet-an-other-simple-machine-learning-package/41057/17 "2024-05-15T20:47:37Z")

</div>

[BetaML](https://github.com/sylvaticus/BetaML.jl) v0.12 is out

Release notes:

- Added `FeatureRanker`, a flexible feature ranking estimator using multiple feature importance metrics
- new functions `kl_divergence` and `sobol_index`
- added option to RF/DT models to ignore specific variables in prediction, by following _both_ the splits on nodes occurring on that dimensions, as the keyword `ignore_dims` to the `predict` function
- added option `sampling_share` to `RandomForestEstimator` model
- DOC: added Benchmarks (but then temporarily removed due to the issue of SystemBenchmark not installable, see [this issue](https://github.com/IanButterworth/SystemBenchmark.jl/issues/64) )
- DOC: added `FeatureRanker` tutorial
- bugfix on `l2loss_by_cv` for unsupervised models

A specific announcement for [`FeatureRanker`](https://sylvaticus.github.io/BetaML.jl/dev/Utils.html#BetaML.Utils.FeatureRanker) is posted [here](https://discourse.julialang.org/t/ann-featureranking-learn-which-variables-contribute-the-most-to-the-estimation-of-black-box-models/114321).