# Flux data parallelism

**URL:** <https://discourse.julialang.org/t/flux-data-parallelism/113627>\
**Category:** Machine Learning\
**Tags:** gpu, cuda, flux\
**Created:** [April 29, 2024, 8:01pm UTC](https://discourse.julialang.org/t/flux-data-parallelism/113627 "2024-04-29T20:01:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jafagervik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jafagervik/32/208594_2.png) [@jafagervik](https://discourse.julialang.org/u/jafagervik)\
**Post date:** [April 29, 2024, 8:01pm UTC](https://discourse.julialang.org/t/flux-data-parallelism/113627/1 "2024-04-29T20:01:48Z")

</div>

Hi, I’ve tried finding resources regarding Flux/Lux data parallelism. Im still unsure what the current SOTA solution for data parallel training would be. Im creating a different kinds of autoencoders, with the main one being a LSTM VAE, and due to the amount of data I want to process, dataparallelism would be great.

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [April 30, 2024, 1:34am UTC](https://discourse.julialang.org/t/flux-data-parallelism/113627/2 "2024-04-30T01:34:14Z")

</div>

Lux has it built in [Distributed Data Parallel Training | Lux.jl Documentation](https://lux.csail.mit.edu/stable/manual/distributed_utils). I had an older package now deprecated which still works with Flux [GitHub - avik-pal/FluxMPI.jl: Distributed Data Parallel Training of Deep Neural Networks](https://github.com/avik-pal/FluxMPI.jl)

---

<div class="post-metadata">

**Author:** ![johnbb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnbb/32/34233_2.png) [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Post date:** [April 30, 2024, 8:29am UTC](https://discourse.julialang.org/t/flux-data-parallelism/113627/3 "2024-04-30T08:29:46Z")

</div>

Is this also the recommended way forward for single node data parallelism? Is there an alternative for Flux?