# Can MLJ \`Stack\` already trained models?

**URL:** <https://discourse.julialang.org/t/can-mlj-stack-already-trained-models/124506>\
**Category:** General Usage\
**Tags:** mlj\
**Created:** [January 7, 2025, 6:10pm UTC](https://discourse.julialang.org/t/can-mlj-stack-already-trained-models/124506 "2025-01-07T18:10:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [January 7, 2025, 6:10pm UTC](https://discourse.julialang.org/t/can-mlj-stack-already-trained-models/124506/1 "2025-01-07T18:10:09Z")

</div>

The doc at [Model Stacking · MLJ](https://juliaai.github.io/MLJ.jl/stable/model_stacking/) shows how to stack models, but the base models are trained at the same time.

Is there a way to use already-trained models and only train the `metalearner`?

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [January 7, 2025, 7:03pm UTC](https://discourse.julialang.org/t/can-mlj-stack-already-trained-models/124506/2 "2025-01-07T19:03:16Z")

</div>

It doesn’t look like that’s an option, but in general you want to be very careful about that kind of thing. The metalearner should be trained on out-of-sample predictions.

What you could do is create a wrapper model type that wraps an already fitted model. The `fit` method for the wrapper type would do nothing, and the `predict` method would forward to the `predict` method of the model that was wrapped.

---

<div class="post-metadata">

**Author:** ![zgornel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zgornel/32/217487_2.png) [@zgornel](https://discourse.julialang.org/u/zgornel)\
**Post date:** [January 8, 2025, 4:08pm UTC](https://discourse.julialang.org/t/can-mlj-stack-already-trained-models/124506/3 "2025-01-08T16:08:45Z")

</div>

I believe that if you create a transform (that only handles inference) rather than model, it should work.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [January 8, 2025, 7:27pm UTC](https://discourse.julialang.org/t/can-mlj-stack-already-trained-models/124506/4 "2025-01-08T19:27:46Z")

</div>

If in doubt, check with @ablaom

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [January 9, 2025, 8:06pm UTC](https://discourse.julialang.org/t/can-mlj-stack-already-trained-models/124506/5 "2025-01-09T20:06:53Z")

</div>

No, there is no support for using pre-trained base models in stacking. As @CameronBieganek points out, it is not enough in a Stack to pre-train a base model on your full train set. In a stack, each base model is trained multiple times, once on each fold complement, as in cross-validation, to splice together the so-called out-of-sample prediction used to train train the metalearer - and then trained again on all the data for making inferences on new data.

Perhaps you can say a bit more about your use case. It may be you can roll your own solution using a [learning network model](https://juliaai.github.io/MLJ.jl/dev/learning_networks/).

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [January 10, 2025, 4:48am UTC](https://discourse.julialang.org/t/can-mlj-stack-already-trained-models/124506/6 "2025-01-10T04:48:48Z")

</div>

> [@ablaom](#):
>
> and then trained again on all the data for making inferences on new data.

my use case is precisely that I don’t have new data – I use cross-train and evaluation so that all my data are used in training but also evaluated in the end (on a model that has never seen this subset of data during training)