@Pere Thanks for your interest in MLJ.
The intended use of evaluate! is to estimate the generalisation error associated with some supervised learning model, by subsampling observations, as in cross-validation, a common use-case. I’m afraid there is no natural way for evaluate! do feature subsampling.
https://alan-turing-institute.github.io/MLJ.jl/dev/evaluating_model_performance/
FYI: There is a version of kernel regression implementing the MLJ model interface, namely kernel partial least squares regression from the package https://github.com/lalvim/PartialLeastSquaresRegressor.jl .