# Batch size and early stopping in Flux

**URL:** <https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517>\
**Category:** Machine Learning\
**Created:** [June 16, 2020, 12:59pm UTC](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517 "2020-06-16T12:59:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mariana](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mariana/32/15064_2.png) [@Mariana](https://discourse.julialang.org/u/Mariana)\
**Post date:** [June 16, 2020, 12:59pm UTC](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517/1 "2020-06-16T12:59:10Z")

</div>

I´m starting to work with Flux and I have a neural network with 63 entries, 2 hidden layers and 1 exit. Each entry and exit is a vector of 113344 elements, from which 78801 elements belong to the training set and the remainder, to the test set.

I trained the network with 5000 epochs, and it took around 8 hours to complete the training. So, in order to decrease this time, I thought about using early stopping and defining batch sizes, but I´m not sure how to do it. At this moment, I´m using the following code to train the network:

function loss\_training(x\_train::Array{Float64,2}, y\_train::Array{Float64,2})  
model = Chain(Dense(63,63,sigmoid),Dense(63,63,sigmoid),Dense(63,1,sigmoid))  
loss(x,y) = Flux.mae(model(x),y)  
ps = params(model)  
dataset = [(x\_train’,y\_train’)]  
opt = ADAGrad()  
cb = () → println(loss(x\_train’,y\_train’))  
Flux.@epochs 5000 Flux.train!(loss,ps,dataset,opt,cb=cb)  
y\_hat = model(x\_train’)’  
return y\_hat, model  
end

y\_hat, model = @time loss\_training(x\_train, y\_train)

y\_test\_hat = @time model(x\_test’)’

---

<div class="post-metadata">

**Author:** ![dmolina](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dmolina/32/5246_2.png) [@dmolina](https://discourse.julialang.org/u/dmolina)\
**Post date:** [June 16, 2020, 1:48pm UTC](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517/2 "2020-06-16T13:48:36Z")

</div>

@Mariana I do not know about the early stopping, but for batch size, I recommend you to use [DataLoader](https://fluxml.ai/Flux.jl/stable/data/dataloader/) (in Flux.Data). With the parameters batchsize, and shuffle it is very simple to use:

```julia
        data = Flux.Data.DataLoader(X, Y, batchsize=32, shuffle=true)
        opt = ADAM()
        loss(x, y) = Flux.mse(model(x), y)
        ps = Flux.params(model)
        evalcb() = @show loss(X,Y)
        
        for epoch in 1:8
            println("Epoch $epoch")
            time = @elapsed train!(loss, ps, data, opt, cb=throttle(evalcb, 3))
            println("Echo $epoch: $time secs")
            @show epoch, loss(X,Y)
        end

```

The code is an example, it is not complete.  
If you have more problems, do not hesitate in ask again.

---

<div class="post-metadata">

**Author:** ![Mariana](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mariana/32/15064_2.png) [@Mariana](https://discourse.julialang.org/u/Mariana)\
**Post date:** [June 16, 2020, 4:46pm UTC](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517/3 "2020-06-16T16:46:34Z")

</div>

@dmolina, thank you very much for your answer! I tried a code similar to yours in my problem and it worked! One question, though: what does cb=throttle(evalcb,3) do in the code?

---

<div class="post-metadata">

**Author:** ![dmolina](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dmolina/32/5246_2.png) [@dmolina](https://discourse.julialang.org/u/dmolina)\
**Post date:** [June 16, 2020, 8:17pm UTC](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517/4 "2020-06-16T20:17:28Z")

</div>

@Mariana Throttle is a function in Flux that allow you to run a function, evalcb in this case, each X seconds. Thus, each 3 seconds, more or less, evalcb is run. This is good to show the improving but not for each iteration. See the [documentation of the function](https://fluxml.ai/Flux.jl/stable/utilities/#Flux.throttle). Also, there is a Flux.stop() that also could be useful.

---

<div class="post-metadata">

**Author:** ![Mariana](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mariana/32/15064_2.png) [@Mariana](https://discourse.julialang.org/u/Mariana)\
**Post date:** [June 17, 2020, 10:10am UTC](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517/5 "2020-06-17T10:10:16Z")

</div>

@dmolina, thank you very much!!

---

<div class="post-metadata">

**Author:** ![HenriDeh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrideh/32/8316_2.png) [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Post date:** [June 17, 2020, 10:56am UTC](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517/6 "2020-06-17T10:56:58Z")

</div>

Please use backticks ( ` ) around your code in the future to make it easier to read for us. See [Please read: make it easier to help you](https://discourse.julialang.org/t/psa-make-it-easier-to-help-you/14757)