# PyTorch DataLoader equivalent for training large models with Flux

**URL:** <https://discourse.julialang.org/t/pytorch-dataloader-equivalent-for-training-large-models-with-flux/30763>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [November 5, 2019, 6:25pm UTC](https://discourse.julialang.org/t/pytorch-dataloader-equivalent-for-training-large-models-with-flux/30763 "2019-11-05T18:25:02Z")\
**Posts on this page:** 1\
**Showing post:** 14

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [November 20, 2019, 5:55pm UTC](https://discourse.julialang.org/t/pytorch-dataloader-equivalent-for-training-large-models-with-flux/30763/14 "2019-11-20T17:55:42Z")

</div>

I have been using extensively this one `https://github.com/pevnak/DataIterators.jl`, but it uses processes rather than threads (as at the time of writing threads were not available). I would like to consolidate this effort, as I would like to write this kind of thing for threads as well.

---

_[View the full topic](https://discourse.julialang.org/t/pytorch-dataloader-equivalent-for-training-large-models-with-flux/30763)._