Anyone has an implementation of generic boosting?

Domains Machine Learning
#1

Dear All,

I would like to ask, as the title suggest, if anyone has a general implementation of boosting algorithm? I did some search and found that it is usually tightly coupled with a base learners being decision trees. But Boosting is a general meta-algorithm which assumes that the underlying base learner can fit possibly weighted samples and perform prediction on them.

I have recently started to wonder, why general people believes that NNs sucks on tabular datasets and Boosted decision trees shines. I came to conclusion that boosting might be the culprit, since single tree sucks as well. Since I would like to know, if I am right or wrong, I would like to test (and also would like to be right).

Due to my lack of time (you can also read it laziness), I would like to ideally hook already existing implementation (my six years old implementation is in matlab).

Well, thanks for answers and opinions on the matter of learning tabular data.

Tomas

#2

There’s two unmaintained libs that might provide you with a decent starting point:

I don’t have experience with either so ymmv

PS: I think your assertion that “NNs suck on tabular datasets” is not up to date. Packages like e.g. AutoGluonTabular seem to suggest otherwise (though it blends NNs with other things).

1 Like
#3

Thanks for links and correction of my knowlege. I was hoping that someone will point me to updated state of the art.

#4

So I read the AutoGluonTabular, and it is not a model based purely on Neural Networks, but they use whatever model scikit learn offers, and an ensembling strategy seems to be a very important part of the solution.

Thanks a lot @tlienart for pointing me to this direction (more pointers are welcomed).

#5

I have fixed the GradientBoost, such that tests (almost) pass on 1.6. The only trouble is clashing of fit! and predict which I do not know, where they are defined.
The fixed library is here

I will try to contact the owner.