# Flux - support for mini-batches

**URL:** <https://discourse.julialang.org/t/flux-support-for-mini-batches/14718>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [September 8, 2018, 10:09pm UTC](https://discourse.julialang.org/t/flux-support-for-mini-batches/14718 "2018-09-08T22:09:30Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sheevy](https://avatars.discourse-cdn.com/v4/letter/s/dc4da7/32.png) [@sheevy](https://discourse.julialang.org/u/sheevy)\
**Post date:** [September 8, 2018, 10:09pm UTC](https://discourse.julialang.org/t/flux-support-for-mini-batches/14718/1 "2018-09-08T22:09:31Z")

</div>

Hi,

Does Flux.train! have support for mini-batches out of the box? If it does I couldn’t see it.

Does it mean in need to restructure my data and rewrite the loss function to act on the mini-batch at once?

Thanks

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [September 8, 2018, 10:38pm UTC](https://discourse.julialang.org/t/flux-support-for-mini-batches/14718/2 "2018-09-08T22:38:47Z")

</div>

Check out MlDataUtils.jl, it should have the support you need and works well with flux

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [September 20, 2018, 7:13am UTC](https://discourse.julialang.org/t/flux-support-for-mini-batches/14718/3 "2018-09-20T07:13:56Z")

</div>

Minibatch is supported, more-or-less automatically. One just has to structure the data provided to the `train!` method in the appropriate way. Here is my understanding:

In ordinary SVD one calls train! with a `data` argument of the form

```
data = [(x1, y1), (x2, y2), ... , (xN, yN)]

```

For `k` mini-batches of size `3` (say) use instead

```
data = [(X1, Y1), (X2, Y2), ... (Xk, Yk)]

```

where

```
X1 = cat(x1, x2, x3, dims=px), X2 = cat(x4, x5, x6, dims=px), etc
Y1 = cat(y1, y2, y3, dims=py), Y2 = cat(y4, y6, y7, dimes=py), etc

```

Here `px` is one more than the dimension of inputs (e.g., 3 for grey-scale images) and `py` is one more than the dimension of outputs. For an example, see [this example from the Flux model-zoo](https://github.com/FluxML/model-zoo/blob/master/vision/mnist/conv.jl)

This “just works” because methods like the provided loss functions and the model functions (e.g., `model = Dense(2,3)`) can be called on multiple instances of data arguments by just concatenating the data along the last dimension.

(In this version of mini-batch the gradients are _summed_ rather than averaged, so you may want divide the SVD learning rate by the batch-size.)