# What does MLJ.save really save?

**URL:** <https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606>\
**Category:** Machine Learning\
**Tags:** mlj\
**Created:** [January 14, 2022, 11:21am UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606 "2022-01-14T11:21:03Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![wc4wc4wc4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wc4wc4wc4/32/23038_2.png) [@wc4wc4wc4](https://discourse.julialang.org/u/wc4wc4wc4)\
**Post date:** [January 14, 2022, 11:21am UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/1 "2022-01-14T11:21:03Z")

</div>

I am running a MLJ pipeline on a quite large dataset and I want to be able to save the model once, and then use it in later sessions without having to retrain it. I have tried following the [documentation](https://alan-turing-institute.github.io/MLJ.jl/dev/machines/#Saving-machines-1) regarding saving models and that works fine. However, I am quite puzzled at the fact that the size of the saved model changes depending on the size of the dataset.

A MWE below:

```julia
using DataFrames
using MLJ

function get_data(N_rows)
    df = DataFrame(rand(N_rows, 3), :auto)
    df.x4 = rand(["A", "B", "C"], N_rows)
    df.y = rand([0, 1], N_rows)
    df = coerce(df, :y => Binary, :x4 => Multiclass)
    X = select(df, Not(:y))
    y = df.y
    return X, y
end

N_rows = 1000
X, y = get_data(N_rows);

LogReg = @load LogisticClassifier pkg = MLJLinearModels
pipe_logreg = @pipeline(OneHotEncoder, LogReg(penalty = :none, fit_intercept = false),)
mach = machine(pipe_logreg, X, y)
fit!(mach)
MLJ.save("pipeline_test_$N_rows.jlso", mach)

```

The size of the saved model is 227 KB when `N_rows = 1_000` and 1.7 MB when `N_rows = 10_000`, yet they both save the same number of parameters (3 numerical coefficients and 3 coefficients for the categories). Or, that’s at least what I expect, but why does the size really change so drastically with more data? I do not expect MLJ to save the input data as well as that would be quite unintuitive.

This is run with Julia v1.7.0, MLJ v0.16.11, MLJBase v0.18.26, and MLJLinearModels v0.5.7.

Cheers!

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [January 14, 2022, 1:26pm UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/2 "2022-01-14T13:26:51Z")

</div>

It looks like it does save the data. If you do `dump(mach)` or `propertynames(mach)` you can see that `data` and `resampled_data` are both part of `mach`. I am not a user of MLJ so I do not know if there is a way to tell `MLJ.save` to empty those fields, but perhaps there is or they might be willing to accept a PR to do something like that?

Also you need `using MLJLinearModels` in your MWE.

---

<div class="post-metadata">

**Author:** ![wc4wc4wc4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wc4wc4wc4/32/23038_2.png) [@wc4wc4wc4](https://discourse.julialang.org/u/wc4wc4wc4)\
**Post date:** [January 16, 2022, 9:45pm UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/3 "2022-01-16T21:45:07Z")

</div>

I’ll try to make an issue on their Github to see if I can find someone who’s deeper into the mechanics. Thanks for your answer!

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [January 17, 2022, 8:39pm UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/4 "2022-01-17T20:39:53Z")

</div>

@wc4wc4wc4 Thanks for your query.

An MLJ machine caches training data by default to prioritise speed over memory. Now this cached data is not part of the learned parameters and is not saved when serialising. (By the way, you can turn the caching off at construction with `machine(model, data...; cache=false)`).

However, if your model is a _composite model_ - by which a I mean a model that is implemented by “exporting” a [learning network](https://alan-turing-institute.github.io/MLJ.jl/dev/composing_models/#Learning-Networks), then the learned parameters of that model may include training data, because the learned parameters is essentially the whole learning network, which includes machines that may include cached data. Roughly speaking, we can say that training data is being cached at internal nodes of the network (although _not_ at the source nodes which are always emptied after each `fit!`). This is not good for serialization, and there is [work underway](https://github.com/JuliaAI/MLJSerialization.jl/pull/17) to fix this.

However, for piplelines you don’t have to wait for a fix, if you use the newer (non-macro) version of pipelines ([docs](https://alan-turing-institute.github.io/MLJ.jl/dev/linear_pipelines/#Linear-Pipelines)) where there is an option to turn caching off for all machines in the pipeline. In your example above, do

```julia
pipe_logreg = Pipeline(OneHotEncoder,
                       LogReg(penalty = :none, fit_intercept = false),
                       cache=false)
mach = machine(pipe_logreg, X, y)
fit!(mach)
MLJ.save("pipeline_test_$N_rows.jlso", mach)

```

Hope that helps.

---

<div class="post-metadata">

**Author:** ![wc4wc4wc4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wc4wc4wc4/32/23038_2.png) [@wc4wc4wc4](https://discourse.julialang.org/u/wc4wc4wc4)\
**Post date:** [January 17, 2022, 9:14pm UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/5 "2022-01-17T21:14:07Z")

</div>

Thank you so much, @ablaom! That seems like the solution to my problem.

Is the macro version of `pipeline` an old deprecated form? I was pretty sure I based it on officiel MLJ documentation, but I might remember wrongly.

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [January 18, 2022, 5:35am UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/6 "2022-01-18T05:35:47Z")

</div>

@wc4wc4wc4 Your welcome.

`@pipeline` is now deprecated and the docs should be up-to-date, as of the MLJ 0.17 release (about 29th December). If you find documentation suggesting the use of `@pipeline` please let me know and I will fix.

---

<div class="post-metadata">

**Author:** ![wc4wc4wc4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wc4wc4wc4/32/23038_2.png) [@wc4wc4wc4](https://discourse.julialang.org/u/wc4wc4wc4)\
**Post date:** [January 18, 2022, 8:21am UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/7 "2022-01-18T08:21:15Z")

</div>

I see now that it was because I started my project in December that I was using the old docs. The new docs seem to work fine and are updated!

However, and this is only a minor problem, since it’s not my main model, but it doesn’t seem to be working with the `LinearBinaryClassifier` from `MLJGLMInterface`:

```julia
using MLJGLMInterface

pipe_GLM = Pipeline(
    OneHotEncoder(drop_last = true),
    x -> table(Matrix(x)),
    LinearBinaryClassifier(fit_intercept = true),
    cache = false,
)

mach = machine(pipe_GLM, X, y)
fit!(mach)
MLJ.save("pipeline_test_GLM_$N_rows.jlso", mach)

```

In this case, the saved machine take up 0.5 MB of space when `N_rows = 1_000` and 5MB when `N_rows = 10_000`. Am I missing something here or is this a bug in `MLJGLMInterface`?

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [January 18, 2022, 9:38pm UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/8 "2022-01-18T21:38:08Z")

</div>

Actually, machines cache more than data. They also cache _views_ of the data, which are encoded as vectors of integers representing which rows (observations) were used in the last training event. I expect this is the explanation.

I will make a note to clear this extra information in the new serialization PR.

---

<div class="post-metadata">

**Author:** ![wc4wc4wc4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wc4wc4wc4/32/23038_2.png) [@wc4wc4wc4](https://discourse.julialang.org/u/wc4wc4wc4)\
**Post date:** [January 19, 2022, 11:33am UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/9 "2022-01-19T11:33:49Z")

</div>

Is it possible to get around this? Because this means that the size of the GLM pipeline machines take up several gigabytes on my real data.

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [January 20, 2022, 8:54pm UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/10 "2022-01-20T20:54:46Z")

</div>

Thats way too big. Okay, I little investigation shows there is something else going on that is specific to GLM. See [this issue](https://github.com/JuliaStats/GLM.jl/issues/465).

---

<div class="post-metadata">

**Author:** ![wc4wc4wc4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wc4wc4wc4/32/23038_2.png) [@wc4wc4wc4](https://discourse.julialang.org/u/wc4wc4wc4)\
**Post date:** [January 20, 2022, 8:57pm UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/11 "2022-01-20T20:57:40Z")

</div>

Great, I’ll follow the Github issue then!

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [February 23, 2022, 4:05am UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/12 "2022-02-23T04:05:25Z")

</div>

@wc4wc4wc4 Thanks to @samuel_okon, the recently released version of MLJGLMInterface 0.3.0 should now resolve your GLM memory issue.

---

<div class="post-metadata">

**Author:** ![wc4wc4wc4](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wc4wc4wc4/32/23038_2.png) [@wc4wc4wc4](https://discourse.julialang.org/u/wc4wc4wc4)\
**Post date:** [February 23, 2022, 8:31am UTC](https://discourse.julialang.org/t/what-does-mlj-save-really-save/74606/13 "2022-02-23T08:31:17Z")

</div>

Thanks a lot, @ablaom!