It seems like you’re trying to fit a square peg into a round hole. ![]()
The resampling strategy interface in MLJ is designed for conducting model performance evaluation on fully trained models. (Although the resampling strategies sort of perform double duty, since you also have to pick one to use for TunedModel.) If you’re trying to do some sort of active sampling algorithm to learn your model, then I think it might make more sense to put all of that into the fit function. Even if training your model is an iterative process, the entire training process should normally be encapsulated in the fit function, so that the output of the fit function is a fully trained model.
Hope that helps!