# XGBoost reduce overfitting | k-fold cross validation

**URL:** <https://discourse.julialang.org/t/xgboost-reduce-overfitting-k-fold-cross-validation/88423>\
**Category:** Machine Learning\
**Tags:** question\
**Created:** [October 8, 2022, 2:08am UTC](https://discourse.julialang.org/t/xgboost-reduce-overfitting-k-fold-cross-validation/88423 "2022-10-08T02:08:36Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Seth16225](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seth16225/32/31117_2.png) [@Seth16225](https://discourse.julialang.org/u/Seth16225)\
**Post date:** [October 8, 2022, 2:08am UTC](https://discourse.julialang.org/t/xgboost-reduce-overfitting-k-fold-cross-validation/88423/1 "2022-10-08T02:08:36Z")

</div>

Hello, I’m pretty new to machine learning, and I was using XGBoost.jl Regression. I’m in the process of trying to reduce overfitting through k-fold cross validation.

I came across the nfold\_cv function, and I was wondering if anyone knew how it worked? I was working off this example:

> <https://github.com/dmlc/XGBoost.jl/blob/master/demo/cross_validation.jl>

I was looking up some videos on k-fold cross validation and how it worked in theory, but I’m not sure how I would use this nfold\_cv function to draw out-of-fold predictions.

Thanks for your help.

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [October 8, 2022, 12:25pm UTC](https://discourse.julialang.org/t/xgboost-reduce-overfitting-k-fold-cross-validation/88423/2 "2022-10-08T12:25:06Z")

</div>

I believe [this link](https://sylvaticus.github.io/SPMLJ/stable/03_-_ML1_-_Introduction_to_Machine_Learning/0301_-_Machine_learning_main_concepts.html#K-folds-cross-validation) explains quite well the idea behind k-fold cross validation.

I don’t know the specific xgboost implementation, but the basic idea is that you use various partitions of the same dataset to divide between training and test sample, and “test” your model, with the specific hyperparameters, on all these different attempts and average the losses, and then you choose the hyperparameters with the lowest averaged loss.

Note that you can separate the problem of finding the best parameters (training of the model, in this case xgboost) with those of finding the best hyperparameters (hyperparameter tuning via cross-validation) using a machine learning library like [`MLJ`](https://github.com/alan-turing-institute/MLJ.jl) or [`BetaML`](https://github.com/sylvaticus/BetaML.jl) (disclaimer: I am the author of the second one).

Note that that function seems not to do just cross validation but to already do a hyperparameter tuning (I assume through a grid search) looping over all the parameter space and on each specific combination it apply cross validation to judge its value…