# Mix-Precision training using Models of Metalhead (Flux.jl) or Boltz (Lux.jl)

**URL:** https://discourse.julialang.org/t/mix-precision-training-using-models-of-metalhead-flux-jl-or-boltz-lux-jl/102857
**Category:** Machine Learning
**Tags:** flux, lux
**Created:** [August 16, 2023, 10:02am UTC](https://discourse.julialang.org/t/mix-precision-training-using-models-of-metalhead-flux-jl-or-boltz-lux-jl/102857 "2023-08-16T10:02:57Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![lgmendesDynamics](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lgmendesdynamics/32/52281_2.png) [@lgmendesDynamics](https://discourse.julialang.org/u/lgmendesDynamics)
#### Post date: [August 16, 2023, 10:02am UTC](https://discourse.julialang.org/t/mix-precision-training-using-models-of-metalhead-flux-jl-or-boltz-lux-jl/102857/1 "2023-08-16T10:02:57Z")

</div>

It is possible to train Deep Learning models for computer vision available in packages like Metalhead (Flux) or Boltz (Lux) with mix-precision?

---

<div class="post-metadata">

### Author: ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [August 16, 2023, 3:18pm UTC](https://discourse.julialang.org/t/mix-precision-training-using-models-of-metalhead-flux-jl-or-boltz-lux-jl/102857/2 "2023-08-16T15:18:47Z")

</div>

Mixed precision training of any model is possible, but you will have to write a lot of the logic such as the loss scaling yourself. [Rule for mixed precision training by CarloLucibello · Pull Request #152 · FluxML/Optimisers.jl · GitHub](https://github.com/FluxML/Optimisers.jl/pull/152) may help with some of that.

---

<div class="post-metadata">

### Author: ![lgmendesDynamics](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lgmendesdynamics/32/52281_2.png) [@lgmendesDynamics](https://discourse.julialang.org/u/lgmendesDynamics)
#### Post date: [August 16, 2023, 9:23pm UTC](https://discourse.julialang.org/t/mix-precision-training-using-models-of-metalhead-flux-jl-or-boltz-lux-jl/102857/3 "2023-08-16T21:23:07Z")

</div>

Thanks for the tip!

---

<div class="post-metadata">

### Author: ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [August 16, 2023, 11:30pm UTC](https://discourse.julialang.org/t/mix-precision-training-using-models-of-metalhead-flux-jl-or-boltz-lux-jl/102857/4 "2023-08-16T23:30:20Z")

</div>

If do you come up with a full example of mixed precision training and can share it, please do! I think it would help with developing some utilities and abstractions around this going forwards.