Flux allows for parallel training with multiple GPUs: GPU Support · Flux
In my use case, I need to run inference of a single model on multiple GPUs. Is there a way to load balance inference calls to the model so all GPUs are maximally utilized?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Can Flux handle multiple GPUs? | 16 | 2612 | August 5, 2022 | |
| Resources for Distributed Training w/ Flux | 1 | 745 | April 2, 2022 | |
| Flux parallel execution | 3 | 2789 | March 29, 2019 | |
| Training Deep Neural Network using Data Parallel? | 5 | 1363 | January 24, 2022 | |
| Flux: GPU not working as expected | 6 | 2232 | July 28, 2020 |