TypeError: Expected UnionAll, got Type{Machine{...}}

Specific Domains Machine Learning
1

I want to save a trained MLJ machine (EvoTreeRegressor) on the cluster and then open it locally on my computer. On the cluster, I save the machine with

MLJ.save("path/on/cluster/machine.jls", mach)

and then try to load it with

mach = machine("path/on/local/computer/machine.jls")

However, I get the following error message:

TypeError: in Type{...} expression, expected UnionAll, got Type{Machine{EvoTreeRegressor{EvoTrees.MSE}, EvoTreeRegressor{EvoTrees.MSE}}}

and the stacktrace

[1] deserialize_datatype(s::Serialization.Serializer{IOStream}, full::Bool)
    @ Serialization /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:1423
  [2] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:867
  [3] deserialize(s::Serialization.Serializer{IOStream})
    @ Serialization /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:814
  [4] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:882
  [5] deserialize(s::Serialization.Serializer{IOStream})
    @ Serialization /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:814
  [6] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:920
  [7] deserialize
    @ Serialization /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:814 [inlined]
  [8] deserialize(s::IOStream)
    @ Serialization /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:801
  [9] open(f::typeof(Serialization.deserialize), args::String; kwargs::@Kwargs{})
    @ Base ./io.jl:396
 [10] open
    @ ./io.jl:393 [inlined]
 [11] deserialize
    @ /cluster/easybuild/broadwell/software/Julia/1.10.0-linux-x86_64/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:811 [inlined]
 [12] machine(file::String)
    @ MLJBase ~/.julia/packages/MLJBase/iIhiI/src/machines.jl:398
 [13] top-level scope
    @ REPL[13]:1

On the cluster where the machine is saved, I am using the following versions:
MLJ v0.20.3
MLJTuning v0.8.4
EvoTrees v0.16.7

(note that the Model I am saving is not a tuned model, but I import it anyways for a training a different model in the same skript)

On the local computer where I want to load the machine I am using

MLJ v0.20.3
EvoTrees v0.16.7.

Do you have any idea what is going on here?

2

Is it possible to confirm that the issue persists when you ensure the two machines are using identical package environments (Manifest.toml files) and of course identical Julia versions?

1 Like
3

Give versioninfo() of both machines.

Have you tried loading the machine on the cluster itself, without involving your local machine?

4

It seems from the error message that the issue is a mismatch of MLJBase.jl versions. Please give the version of MLJBase in the manifest on both machines.

In particular, this commit introduces a third type parameter to Machine:

So it seems the version of MLJBase on the cluster is newer than on your local computer.

1 Like
5

Ahhh. Good observation @nsajko . That’s seems like the likely issue.

1 Like
6

Very good observation, I used MLJBase 1.2.1 and MLJBase 1.3.0. Making sure the MLJBase version is the same indeed solves the issue. Thank you very much

1 Like
7

In case someone else (or future me) stumbles into this problem again: If you save the machine with MLJBase 1.3.0 and load it with MLJBase 1.2.1, you get the TypeError mentioned above.
However, if you save the machine with MLJBase 1.2.1 and then try to load it with MLJBase 1.3.0, you get the error message

ERROR: MethodError: no method matching deserialize(::Serialization.Serializer{IOStream}, ::Type{Machine{EvoTreeRegressor{EvoTrees.MSE}, true}})

Closest candidates are:
  deserialize(::Serialization.AbstractSerializer, ::Type{Core.MethodInstance})
   @ Serialization ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:1090
  deserialize(::Serialization.AbstractSerializer, ::Type{Core.PhiNode})
   @ Serialization ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:1121
  deserialize(::Serialization.AbstractSerializer, ::Type{Core.TypeName})
   @ Serialization ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:1297

with the stacktrace

[1] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:883
  [2] deserialize(s::Serialization.Serializer{IOStream})
    @ Serialization ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:814
  [3] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:920
  [4] deserialize
    @ ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:814 [inlined]
  [5] deserialize(s::IOStream)
    @ Serialization ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:801
  [6] open(f::typeof(Serialization.deserialize), args::String; kwargs::@Kwargs{})
    @ Base ./io.jl:396
  [7] open
    @ ./io.jl:393 [inlined]
  [8] deserialize
    @ ~/software/julia-1.10.2/share/julia/stdlib/v1.10/Serialization/src/Serialization.jl:811 [inlined]
  [9] machine(file::String)
    @ MLJBase ~/.julia/packages/MLJBase/hoZmq/src/machines.jl:404
 [10] top-level scope
    @ REPL[6]:1

As mentioned by @nsajko make sure that the MLJBase version is the same on both computers.

1 Like