How to make EvoTrees.jl more performant?

General Usage Performance
1

Hello Folks:

Code Snippets obtained from Alan Turing Institute @ablaom

Using Pluto.jl, Windows, Julia v 1.6.x

I am using EvoTrees.jl Evaluation

First I am loading and Instantiating the Gradient Tree
Boosting Model

Booster = @load EvoTreeRegressor

Second, I am wrapping the model to make it self-repeating

itr_boost = InteratedModel(model=Booster, resampling=Holdout(fraction_train=0.8)...)

Then I combine with the categorical feature encoder

pipe = @pipeline Continuous Encoder itr_boost

After, I define the hyperparameter range

max_depth_range = MLJIteration.range(pipe, :deterministic_iterated_model.model.max.depth),
                                  lower = 1, 
                                  upper = 30)

I wrap the model in a optimization container (for self-tuning)

self_tuning_pipe = MLJ.TunedModel(model=pipe, 
                                                           tuning = RandomSearch(), 
                                                           ranges = max_depth_range, 
                                                           resampling=CV(nfolds=3, rng=456),
                                                           measure=l1,
                                                           acceleration=CPUThreads(),
                                                           n=5)

I bind the data with

X, y = @load_reduced_ames

Wrap self_tuning pipe in a machine

EVO = machine(self_tuning_pipe, X, y)

Lastly, when I attempt to evaluate wrapped self-tuning pipe

MLJ.evaluate!(EVO,
                 measures=[l1, l2],
                 resampling=CV(nfolds=5, rng=123),
                 acceleration=CPUThreads(),
                 verbosity=2)

I am returning the following error:

Distributed.ProcessExitedException(10)

I have attempted to decrease the folds and
make other parameter adjustments, but am
experiencing the same issue. The processes
exit and my Pluto session terminates.

Is this a parameter issue or a performance
issue? Or maybe something else? How
would you approach a solution to this?

Thanks,

2

@YummyPampers2 thanks for reporting!

Can you please post a full stack trace?

In the meantime, there is a known issue that popped up with EvoTreeRegressor fixed only this week in this version:

https://github.com/Evovest/EvoTrees.jl/releases/tag/v0.8.3

So make sure you’re using this version.

3

@ablaom Thank you for sharing this! I saw about 2 days ago,
you posted a similar response but I was not sure how to fix.

As for the stack-trace am attempting to copy it, but the browser
freezes when I run that last block.

4

Can you try to reproduce the issue by executing your code in the Repl without using Pluto?

And paste the output of the command
]status

?

1 Like
5

@ablaom and @ufechner7

Good Day Professors:

The error-log reads:

image
image883×355 30.7 KB

This is with the v0.8.3 EvoTrees. When the EvoTrees version was
0.8.2 the error message from the REPL was:

image
image1500×261 12 KB

6

As an alternative (for anyone observing using Pluto.jl):

  1. select F12 OR right-click in the browser (select ‘Inspect Element’),
  2. select ‘Console’ tab, then on the far right select either ‘Errors’ or ‘Log’,
  3. screenshot the messages after executing the MLJ.evaluate! method
7

There is a version of the code you are running here, which also includes a set of environment files Project.tom and Manifest.toml, for reproducing the same environment. (If you just run the script there as is, in the same folder as these files, it should use that environment automatically). I’m unable to reproduce your problems with that script/env. I also updated the environment there with using Pkg.update() and again, no problems. In this case I have these versions installed:

Pkg.status()
      Status `~/Google Drive/Julia/MLJ/MLJ/examples/lightning_tour/Project.toml`
  [f6006082] EvoTrees v0.8.3
  [98b081ad] Literate v2.9.1
  [add582a8] MLJ v0.16.7
  [614be32b] MLJIteration v0.3.1

Does this match your status?

Also, is your stack trace complete? It does not appear to be.

BTW, You may want to drop the acceleration=... option to simplify diagnosis.

8

Hello Dr. Blaom:

The full log reads:

image
image887×553 41.1 KB

Currently testing without the acceleration parameter and different
workspace combinations. Will post results soon.

9

Hello: This is the full error log:

image
image1438×1544 156 KB

1 Like
10

Can you please report the output of using Pkg; Pkg.status()?

11

From the REPL or with in a Pluto process?

12

should they be the same? wherever you’re running this. The point is to know the versions of packages that lead you to this error

14

@ablaom and @jling

First,
image

Second,
image

Third,
image
image

Fourth: I resolved the packages

Fifth: I instantiated all the changes

15

EvoTrees is not the only package in play here is it? Are you not using MLJ? Please list all the package versions

16

Also note that if you use the current Pluto version, the package versions in your notebook might be different from those in your default (or any other) environment, as Pluto’s built-in package manager will create a notebook-specific environment once you put using somewhere in your notebook.

17

not confusing at all… yeah

18

I think it’s nice for reproducibility and mitigates compat issues especially for new users which might not use environments otherwise, but I agree it can be surprising for people that do ] st or other package operations in the REPL to find that they have no effect on the code in their notebooks…

More details here: 🎁 Package management · fonsp/Pluto.jl Wiki · GitHub

1 Like
19

image

image

image

image

image

20

By the way, it’s usually more convenient if you can copy and paste code snippets into code blocks, rather than posting screenshots. See here for more info:

2 Likes
21

Thanks Cameron.

The original post had the format that was convenient for you.
The follow-up posts had screen shots simply for QA not
reproducibility.