# MLJ.save() and restoring with machine() don't work

**URL:** <https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535>\
**Category:** Machine Learning\
**Tags:** question, mlj\
**Created:** [February 21, 2024, 6:13pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535 "2024-02-21T18:13:20Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paulo\_Refosco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulo_refosco/32/204900_2.png) [@Paulo\_Refosco](https://discourse.julialang.org/u/Paulo_Refosco)\
**Post date:** [February 21, 2024, 6:13pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/1 "2024-02-21T18:13:20Z")

</div>

Hello!

I made a simple XGBoost model with the pipeline below, fitted it with data and tried to save it and then restore it, but when restoring it (even if in the same notebook) I get error on trying to apply predict. Any idea on how to solve it?

```julia
XGBC = @load XGBoostClassifier
xgb = XGBC()
ohe = OneHotEncoder()

# Pipeline OneHotEncoder > XGBoost
xgb_pipe = ohe |> xgb

# Setting Target and Features tables:
y, X = unpack(df, ==(:y_label), col->true)

train, test = partition(1:length(y), 0.1, shuffle=true)

xgbm = machine(xgb_pipe, X, y, cache=false)
fit!(xgbm, rows=train, verbosity=0)

MLJ.save("mach_xgb_pipe.jls", xgbm)

# Restoring the model and using for predictions:
mach_restored = machine("mach_xgb_pipe.jls")

yhat = predict_mode(mach_restored , X[test,:])

```

Error message:

```julia
Error: Failed to apply the operation `predict` to the machine machine(:xg_boost_classifier, …), which receives it's data arguments from one or more nodes in a learning network. Possibly, one of these nodes is delivering data that is incompatible with the machine's model.
│ Model (xg_boost_classifier):
│ input_scitype = Unknown
│ target_scitype =Unknown
│ output_scitype =Unknown
│ 
│ Incoming data:
│ arg of predict	scitype
│ -------------------------------------------
│ Node @818 → :one_hot_encoder	Table{AbstractVector{Continuous}}
│ 
│ Learning network sources:
│ source	scitype
│ -------------------------------------------
│ Source @791	Table{Union{AbstractVector{Continuous}, AbstractVector{Multiclass{10}}, AbstractVector{Multiclass{2}}, AbstractVector{Multiclass{89}}, AbstractVector{Multiclass{6}}}}
│ Source @496	AbstractVector{OrderedFactor{2}}
└ @ MLJBase C:\Users\User\.julia\packages\MLJBase\mIaqI\src\composition\learning_networks\nodes.jl:153
XGBoostError: (caller: XGBoosterPredictFromDMatrix)
[14:39:55] /workspace/srcdir/xgboost/src/c_api/c_api.cc:1059: Booster has not been initialized or has already been disposed.

Stacktrace:
  [1] _apply(y_plus::Tuple{Node{Machine{Symbol, true}}, Machine{Symbol, true}}, input::DataFrame; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ MLJBase C:\Users\User\.julia\packages\MLJBase\mIaqI\src\composition\learning_networks\nodes.jl:159
  [2] _apply
    @ C:\Users\User\.julia\packages\MLJBase\mIaqI\src\composition\learning_networks\nodes.jl:144 [inlined]
  [3] (::Node{Machine{Symbol, true}})(Xnew::DataFrame)
    @ MLJBase C:\Users\User\.julia\packages\MLJBase\mIaqI\src\composition\learning_networks\nodes.jl:140
  [4] output_and_report(signature::MLJBase.Signature{NamedTuple{(:predict, :transform), Tuple{Node{Machine{Symbol, true}}, Node{Machine{Symbol, true}}}}}, operation::Symbol, Xnew::DataFrame)
    @ MLJBase C:\Users\User\.julia\packages\MLJBase\mIaqI\src\composition\learning_networks\signatures.jl:374
  [5] predict(model::MLJBase.ProbabilisticPipeline{NamedTuple{(:one_hot_encoder, :xg_boost_classifier), Tuple{Unsupervised, Probabilistic}}, MLJModelInterface.predict}, fitresult::MLJBase.Signature{NamedTuple{(:predict, :transform), Tuple{Node{Machine{Symbol, true}}, Node{Machine{Symbol, true}}}}}, Xnew::DataFrame)
    @ MLJBase C:\Users\User\.julia\packages\MLJBase\mIaqI\src\operations.jl:191
  [6] predict(mach::Machine{MLJBase.ProbabilisticPipeline{NamedTuple{(:one_hot_encoder, :xg_boost_classifier), Tuple{Unsupervised, Probabilistic}}, MLJModelInterface.predict}, false}, Xraw::DataFrame)
    @ MLJBase C:\Users\User\.julia\packages\MLJBase\mIaqI\src\operations.jl:133
  [7] predict
    @ C:\Users\User\.julia\packages\MLJTuning\drqMP\src\tuned_models.jl:795 [inlined]
  [8] predict_mode(m::MLJTuning.ProbabilisticTunedModel{Grid, MLJBase.ProbabilisticPipeline{NamedTuple{(:one_hot_encoder, :xg_boost_classifier), Tuple{Unsupervised, Probabilistic}}, MLJModelInterface.predict}}, fitresult::Machine{MLJBase.ProbabilisticPipeline{NamedTuple{(:one_hot_encoder, :xg_boost_classifier), Tuple{Unsupervised, Probabilistic}}, MLJModelInterface.predict}, false}, Xnew::DataFrame)
    @ MLJBase C:\Users\User\.julia\packages\MLJBase\mIaqI\src\interface\model_api.jl:11
  [9] predict_mode(mach::Machine{MLJTuning.ProbabilisticTunedModel{Grid, MLJBase.ProbabilisticPipeline{NamedTuple{(:one_hot_encoder, :xg_boost_classifier), Tuple{Unsupervised, Probabilistic}}, MLJModelInterface.predict}}, false}, Xraw::DataFrame)
    @ MLJBase C:\Users\User\.julia\packages\MLJBase\mIaqI\src\operations.jl:133
 [10] top-level scope
    @ In[113]:5

```

Side note: the model does work when applied directly to “predict”/“predict\_mode”.  
I also tried, without success:

```julia
using JLSO

smach = serializable(mach_tuned_xgb_pipe)
JLSO.save("machine_serialized.jlso", :machine => smach)

loaded_mach = JLSO.load("machine_serialized.jlso")[:machine]
restore!(loaded_mach)

yhat = predict_mode(loaded_mach, X[test,:])

```

Thanks for helping!

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [February 21, 2024, 10:58pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/2 "2024-02-21T22:58:11Z")

</div>

Thanks for reporting. This is issue has been reported elsewhere, and its good to have your example because I was previously unable to reproduce it. Here is the issue link: [Serialized Composite Model Fails with XGBoost · Issue #927 · JuliaAI/MLJBase.jl · GitHub](https://github.com/JuliaAI/MLJBase.jl/issues/927)

---

<div class="post-metadata">

**Author:** ![Paulo\_Refosco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulo_refosco/32/204900_2.png) [@Paulo\_Refosco](https://discourse.julialang.org/u/Paulo_Refosco)\
**Post date:** [February 22, 2024, 10:21am UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/3 "2024-02-22T10:21:32Z")

</div>

Thanks a lot for the quick answer @ablaom !

I just provided a not-working code with an example csv file attached in the git issue ([Serialized Composite Model Fails with XGBoost · Issue #927 · JuliaAI/MLJBase.jl · GitHub](https://github.com/JuliaAI/MLJBase.jl/issues/927)). Let me know if there is anything else that I can do (and also when/if it gets solved!)

Thanks!

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [February 23, 2024, 3:04am UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/4 "2024-02-23T03:04:41Z")

</div>

Note a possible workaround is to use the gradient boosted trees provided by the pure Julia implementation [EvoTrees.jl](https://github.com/Evovest/EvoTrees.jl) . These should serialise fine in pipelines.

---

<div class="post-metadata">

**Author:** ![Paulo\_Refosco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulo_refosco/32/204900_2.png) [@Paulo\_Refosco](https://discourse.julialang.org/u/Paulo_Refosco)\
**Post date:** [February 23, 2024, 2:23pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/5 "2024-02-23T14:23:33Z")

</div>

@ablaom , do you think it is something you will be able to solve in the near future (both because of direction of efforts and/or priorities)? I’m pursuing an organizational shift towards Julia, and since xgboost is widely adopted here, I’ll need either a workaround or maybe even switching to using directly XGBoost.jl (even though I find MLJ usage more user-friendly, although I also can’t get the xgb model to show feature importances when using MLJ (but this probably to my lack of coding knowledge)).

Anyway, it was great to have such prompt answers from you! (thanks a lot for that!).

---

<div class="post-metadata">

**Author:** ![Paulo\_Refosco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulo_refosco/32/204900_2.png) [@Paulo\_Refosco](https://discourse.julialang.org/u/Paulo_Refosco)\
**Post date:** [February 23, 2024, 3:19pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/6 "2024-02-23T15:19:06Z")

</div>

By the way, the easiest workaround I figured is to keep using MLJ, but break the model away from the pipeline (so not using xgb\_pipe = ohe |\> xgb , but instead treating the raw data with “ohe” in a machine, then another machine to input this treated data and running xgb. Then it’s a matter of saving both machines (ohe and xgb) and restoring them etc). This way is working. The problem seems to be on pipelines…

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [February 26, 2024, 2:01am UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/7 "2024-02-26T02:01:36Z")

</div>

> do you think it is something you will be able to solve in the near future

I’ll try to look at it this week.

> although I also can’t get the xgb model to show feature importances when using MLJ

Definitely raise an issue if you are having trouble. It’s likely that feature importances of a supervised model within a pipeline are not directly accessible, and so adding an issue to request this makes sense.

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [February 29, 2024, 10:37pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/8 "2024-02-29T22:37:38Z")

</div>

Fix coming soon: [Fix problem with serialization of nested models when component model overload `save`/`restore` by ablaom · Pull Request #960 · JuliaAI/MLJBase.jl · GitHub](https://github.com/JuliaAI/MLJBase.jl/pull/960)

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [March 1, 2024, 2:11am UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/9 "2024-03-01T02:11:43Z")

</div>

This should be resolved by updating to latest versions (you need MLJBase 1.1.2). Feel free to re-open [Serialized Composite Model Fails with XGBoost · Issue #927 · JuliaAI/MLJBase.jl · GitHub](https://github.com/JuliaAI/MLJBase.jl/issues/927) if some issue persists.

@Paulo_Refosco Would you mind editing the post title to make it more specific? The issue was limited to XGBoost.jl models.

---

<div class="post-metadata">

**Author:** ![Paulo\_Refosco](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulo_refosco/32/204900_2.png) [@Paulo\_Refosco](https://discourse.julialang.org/u/Paulo_Refosco)\
**Post date:** [March 1, 2024, 2:48pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/10 "2024-03-01T14:48:45Z")

</div>

Thanks @ablaom !

As I mentioned in [Serialized Composite Model Fails with XGBoost · Issue #927 · JuliaAI/MLJBase.jl · GitHub](https://github.com/JuliaAI/MLJBase.jl/issues/927), saving and restoring a Pipeline including XGBoost using MLJ.save() now is working for me!

For the title, I definitely don’t mind changing it, but, maybe due to my profile here (or maybe because I don’t know how to use it), it seems I am unable to change the titles of my topics… I’ll look further into it though.

I will consider raising the issue on features importances and also let me point out that I tried saving the tuned xgb pipe model and restoring it ended up in the same error as we were getting for the “regular” xgb pipe. But I don’t know if it would really be expected to work on the tuned machine, so just pointing out.

And again, thanks very much for all the efforts. It helped me a lot!

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [March 3, 2024, 8:55pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/11 "2024-03-03T20:55:56Z")

</div>

> <https://github.com/alan-turing-institute/MLJ.jl/issues/1099>
>
> XGBoost.jl models have non-persistent \`fitresult\`s which means that they cannot …be directly serialised. That's not a problem, because such models can overload MLJModelInterface's \`save\` and \`restore\`. However, it \[has been reported\](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/10?u=ablaom) that XGBoost models wrapped in \`TunedModel\` don't deserialise properly. Here is a MWE:
> 
> First, we defined a supervised model, \`EphemeralRegressor\` with an ephemeral fitresult. For this model we overload \`save/restore\` to ensure deserialization works, provided you use the \[correct API\](https://alan-turing-institute.github.io/MLJ.jl/dev/machines/#Saving-machines). 
> 
> \<details\>
> 
> \`\`\`julia
> using Statistics, MLJBase, Test, MLJTuning, Serialization, StatisticalMeasures
> import MLJModelInterface
> 
> \# define a model with non-persistent fitresult:
> thing = \[\]
> struct EphemeralRegressor \<: Deterministic end
> function MLJModelInterface.fit(::EphemeralRegressor, verbosity, X, y)
> # if I serialize/deserialized \`thing\` then \`view\` below changes:
> view = objectid(thing)
> fitresult = (thing, view, mean(y))
> return fitresult, nothing, NamedTuple()
> end
> function MLJModelInterface.predict(::EphemeralRegressor, fitresult, X)
> thing, view, μ = fitresult
> return view == objectid(thing) ? fill(μ, nrows(X)) :
> throw(ErrorException("dead fitresult"))
> end
> function MLJModelInterface.save(::EphemeralRegressor, fitresult)
> thing, \_, μ = fitresult
> return (thing, μ)
> end
> function MLJModelInterface.restore(::EphemeralRegressor, serialized\_fitresult)
> thing, μ = serialized\_fitresult
> view = objectid(thing)
> return (thing, view, μ)
> end
> 
> \# EphemeralRegressor cannot be directly serialized:
> X, y = (; x = rand(3)), fill(42.0, 3)
> model = EphemeralRegressor()
> mach = machine(model, X, y) |\> fit!
> io = IOBuffer()
> serialize(io, mach)
> seekstart(io)
> mach2 = deserialize(io)
> @test\_throws ErrorException("dead fitresult") predict(mach2, 42)
> 
> \# But it can be serialized/deserialized using correct API:
> io = IOBuffer()
> serialize(io, serializable(mach))
> seekstart(io)
> mach2 = restore!(deserialize(io))
> @test MLJBase.predict(mach2, (; x = rand(2))) ≈ fill(42.0, 2)
> \`\`\`
> \</details\>
> 
> But wrapping this model using \`TunedModel\` leads to deserialization failure:
> 
> \`\`\`julia
> tmodel = TunedModel(
> models=fill(EphemeralRegressor(), 2),
> measure = l2,
> )
> mach = machine(tmodel, X, y) |\> fit!
> io = IOBuffer()
> serialize(io, serializable(mach))
> seekstart(io)
> mach2 = restore!(deserialize(io))
> MLJBase.predict(mach2, (; x = rand(2)))
> \# ERROR: dead fitresult
> \# Stacktrace:
> \# \[1\] predict(::EphemeralRegressor, fitresult::Tuple{Vector{Any}, UInt64, Float64}, X::@NamedTuple{x::Vector{Float64}})                                                            
> \# @ Main ./REPL\[7\]:3
> \#
> \# \< truncated trace \>
> \`\`\`
> 
> The remedy is to properly "forward" the \`save\`/\`restore\` methods in all model wrappers. We can exclude any wrapper model implemented as \`NetworkComposite\` (ie, using learning networks) as they already overload \`save\` and \`restore\` \[properly\](https://github.com/JuliaAI/MLJBase.jl/pull/960).
> 
> To do:
> 
> \- \[\] \`TunedModel\`
> \- \[\] \`IteratedModel\`
> \- \[\] \`EnsembleModel\`
> \- \[\] \`BinaryThresholdPredictor\` ?

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [March 7, 2024, 8:06pm UTC](https://discourse.julialang.org/t/mlj-save-and-restoring-with-machine-dont-work/110535/12 "2024-03-07T20:06:47Z")

</div>

Above issue resolved by MLJTuning 0.8.2