# \#flux

**URL:** https://discourse.julialang.org/tag/flux/162.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [Post-doc position at the GHER, ULiège, Belgium](https://discourse.julialang.org/t/post-doc-position-at-the-gher-uliege-belgium/139159)

<div class="topic-metadata">

**Author:** [@Alexander-Barth](https://discourse.julialang.org/u/Alexander-Barth)\
**Replies:** 0\
**Last updated:** [September 2, 2026, 8:38am UTC](https://discourse.julialang.org/t/post-doc-position-at-the-gher-uliege-belgium/139159 "2026-09-02T08:38:24Z")

</div>

We invite applications for a project scientist (post-doc) position at the GeoHydrodynamics and Environment Research group (GHER, https://www.gher.uliege.be) of the University of Liège in Belgium. The candidate will work …

---

## [Invertible Neural Networks](https://discourse.julialang.org/t/invertible-neural-networks/137909)

<div class="topic-metadata">

**Author:** [@langestefan](https://discourse.julialang.org/u/langestefan)\
**Replies:** 0\
**Last updated:** [July 2, 2026, 2:40pm UTC](https://discourse.julialang.org/t/invertible-neural-networks/137909 "2026-07-02T14:40:20Z")

</div>

I was looking at doing some modelling work using invertible neural-networks in Julia with Lux.jl. There aren’t many options, but I found GitHub - slimgroup/InvertibleNetworks.jl: A Julia framework for invertible neural …

---

## [Memory allocations in Flux evaluation/training (w/MVE)](https://discourse.julialang.org/t/memory-allocations-in-flux-evaluation-training-w-mve/137175)

<div class="topic-metadata">

**Author:** [@CrashBurnRepeat](https://discourse.julialang.org/u/CrashBurnRepeat)\
**Replies:** 39\
**Last updated:** [June 6, 2026, 10:00am UTC](https://discourse.julialang.org/t/memory-allocations-in-flux-evaluation-training-w-mve/137175 "2026-06-06T10:00:29Z")

</div>

I’m currently using Flux to do some GAN experiments and I’m finding that the training is painfully slow. On the CPU, a single epoch can take ~30min for a 286x32 sample dataloader. GPU evaluation is faster, but still feel…

---

## [My journey training an LLM from scratch in Julia (and why I see huge potential)](https://discourse.julialang.org/t/my-journey-training-an-llm-from-scratch-in-julia-and-why-i-see-huge-potential/136843)

<div class="topic-metadata">

**Author:** [@mouad-tarif](https://discourse.julialang.org/u/mouad-tarif)\
**Replies:** 5\
**Last updated:** [April 24, 2026, 5:07pm UTC](https://discourse.julialang.org/t/my-journey-training-an-llm-from-scratch-in-julia-and-why-i-see-huge-potential/136843 "2026-04-24T17:07:59Z")

</div>

I started training a language model from scratch in Julia. Not using pre-built libraries for the core - building my own BPE tokenizer, my own training loop, facing hallucinations, and rebuilding. I tried Flux and Lux. B…

---

## [Why is Flux/NNlib falling back to im2col instead of MIOpen on AMD GPU?](https://discourse.julialang.org/t/why-is-flux-nnlib-falling-back-to-im2col-instead-of-miopen-on-amd-gpu/135867)

<div class="topic-metadata">

**Author:** [@ajrohr2](https://discourse.julialang.org/u/ajrohr2)\
**Replies:** 3\
**Last updated:** [March 10, 2026, 1:30am UTC](https://discourse.julialang.org/t/why-is-flux-nnlib-falling-back-to-im2col-instead-of-miopen-on-amd-gpu/135867 "2026-03-10T01:30:35Z")

</div>

Hello! I’ve been trying to get Flux’s convolutional layers, specifically the Flux.NNlib.conv! function, to use the MIOpen version rather than im2col or direct. I’ve verified that I have MIOpen available with AMDGPU.funct…

---

## [Multi-GPU inference in Flux.jl](https://discourse.julialang.org/t/multi-gpu-inference-in-flux-jl/135210)

<div class="topic-metadata">

**Author:** [@Chrysoberyl](https://discourse.julialang.org/u/Chrysoberyl)\
**Replies:** 2\
**Last updated:** [January 24, 2026, 12:07am UTC](https://discourse.julialang.org/t/multi-gpu-inference-in-flux-jl/135210 "2026-01-24T00:07:26Z")

</div>

Flux allows for parallel training with multiple GPUs: GPU Support · Flux In my use case, I need to run inference of a single model on multiple GPUs. Is there a way to load balance inference calls to the model so all GPU…

---

## [Identical input in Zygote leads to different outputs](https://discourse.julialang.org/t/identical-input-in-zygote-leads-to-different-outputs/135120)

<div class="topic-metadata">

**Author:** [@Chrysoberyl](https://discourse.julialang.org/u/Chrysoberyl)\
**Replies:** 1\
**Last updated:** [January 19, 2026, 12:27am UTC](https://discourse.julialang.org/t/identical-input-in-zygote-leads-to-different-outputs/135120 "2026-01-19T00:27:54Z")

</div>

In this script, the forward pass has no error: using GNNGraphs, GraphNeuralNetworks, NNlib, Flux graph = GNNHeteroGraph( Dict( (:A, :a, :B) =\> (\[1, 2\], \[3, 4\]), (:B, :a, :A) =\> (\[1\], \[2\]), ); …

---

## [Reset model parameters Flux.jl](https://discourse.julialang.org/t/reset-model-parameters-flux-jl/35021)

<div class="topic-metadata">

**Author:** [@luboshanus](https://discourse.julialang.org/u/luboshanus)\
**Replies:** 3\
**Last updated:** [January 15, 2026, 4:05pm UTC](https://discourse.julialang.org/t/reset-model-parameters-flux-jl/35021 "2026-01-15T16:05:48Z")

</div>

Hi, I’d like to ask how to reset model parameters without saving initial parameters of the model such as init\_theta = Flux.params(model) and then loading them to the model when reseting. I would like to use built in Flu…

---

## [CUDA Warning about freeing DeviceMemory](https://discourse.julialang.org/t/cuda-warning-about-freeing-devicememory/135042)

<div class="topic-metadata">

**Author:** [@Chrysoberyl](https://discourse.julialang.org/u/Chrysoberyl)\
**Replies:** 1\
**Last updated:** [January 15, 2026, 7:37am UTC](https://discourse.julialang.org/t/cuda-warning-about-freeing-devicememory/135042 "2026-01-15T07:37:09Z")

</div>

I’m using Flux.jl for machine learning. When I run training, I got this error (seems to be generated from compiled code in Zygote) ERROR: LoadError: CUDA error: an illegal memory access was encountered (code 700, ERROR\_…

---

## [Machine Learning and Deep Learning Course with Julia](https://discourse.julialang.org/t/machine-learning-and-deep-learning-course-with-julia/134890)

<div class="topic-metadata">

**Author:** [@rajgoel](https://discourse.julialang.org/u/rajgoel)\
**Replies:** 1\
**Last updated:** [January 5, 2026, 4:13pm UTC](https://discourse.julialang.org/t/machine-learning-and-deep-learning-course-with-julia/134890 "2026-01-05T16:13:47Z")

</div>

I created a course on Machine Learning and Deep Learning that is using Julia and Flux.jl in particular. I wanted the course to be mathematically precise while providing concise implementations that match the notation use…

---

## [Flux newbie: simple Markov](https://discourse.julialang.org/t/flux-newbie-simple-markov/134073)

<div class="topic-metadata">

**Author:** [@Anders\_Holtsberg](https://discourse.julialang.org/u/Anders_Holtsberg)\
**Replies:** 5\
**Last updated:** [December 3, 2025, 8:30am UTC](https://discourse.julialang.org/t/flux-newbie-simple-markov/134073 "2025-12-03T08:30:32Z")

</div>

Note: Flux newbie here. I am trying to learn Flux with the goal in the end to experiment with different GRU-like models, oneHot to oneHot with crossentropy, background is NLP. I started learning Flux by trying to set …

---

## [Using \`CartesianIndex\` with Flux.jl](https://discourse.julialang.org/t/using-cartesianindex-with-flux-jl/134279)

<div class="topic-metadata">

**Author:** [@Chrysoberyl](https://discourse.julialang.org/u/Chrysoberyl)\
**Replies:** 3\
**Last updated:** [December 3, 2025, 3:49am UTC](https://discourse.julialang.org/t/using-cartesianindex-with-flux-jl/134279 "2025-12-03T03:49:15Z")

</div>

Is it possible to use CartesianIndex with Flux.jl within a Flux.with\_gradient block? Specifically I have a::Vector{Float32} and b::Vector{CartesianIndex}, and a\[b\] is in the gradient block. When I used this I got ERROR…

---

## [DiffOpt / JuMP: zero gradient for variable fixed by equality constraint](https://discourse.julialang.org/t/diffopt-jump-zero-gradient-for-variable-fixed-by-equality-constraint/134041)

<div class="topic-metadata">

**Author:** [@yeomoon](https://discourse.julialang.org/u/yeomoon)\
**Replies:** 12\
**Last updated:** [November 28, 2025, 5:36am UTC](https://discourse.julialang.org/t/diffopt-jump-zero-gradient-for-variable-fixed-by-equality-constraint/134041 "2025-11-28T05:36:03Z")

</div>

Hi all, I’m trying to train a neural network that is followed by a differentiable optimization layer in Julia. Conceptually it’s: Flux NN → Economic Dispatch layer (ED layer) → Loss The ED layer is implemented as a …

---

## [Issue: Enzyme + Flux double differentiation on custom layer](https://discourse.julialang.org/t/issue-enzyme-flux-double-differentiation-on-custom-layer/133503)

<div class="topic-metadata">

**Author:** [@Gianmarco](https://discourse.julialang.org/u/Gianmarco)\
**Replies:** 3\
**Last updated:** [October 30, 2025, 1:52am UTC](https://discourse.julialang.org/t/issue-enzyme-flux-double-differentiation-on-custom-layer/133503 "2025-10-30T01:52:28Z")

</div>

Hi, I’m using Flux and Enzyme for a project, and the entire architecture is built on top of these two libraries. So unless strictly necessary, I’d like to stay within the Flux/Enzyme ecosystem. For both libraries i’m u…

---

## [Best options for Graph Neural Network integration with Flux](https://discourse.julialang.org/t/best-options-for-graph-neural-network-integration-with-flux/133368)

<div class="topic-metadata">

**Author:** [@AriMarkowitz](https://discourse.julialang.org/u/AriMarkowitz)\
**Replies:** 5\
**Last updated:** [October 24, 2025, 7:15am UTC](https://discourse.julialang.org/t/best-options-for-graph-neural-network-integration-with-flux/133368 "2025-10-24T07:15:42Z")

</div>

Hello! I built a library that utilizes GraphNeuralNetworks.jl and Flux.jl, but the GraphNeuralNetworks.jl tests are currently failing in Julia 1.12 (and the same failure is causing my custom library to crash when used).…

---

## [Fitting neural ODE to periodic time series](https://discourse.julialang.org/t/fitting-neural-ode-to-periodic-time-series/42263)

<div class="topic-metadata">

**Author:** [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)\
**Replies:** 18\
**Last updated:** [October 13, 2025, 6:31pm UTC](https://discourse.julialang.org/t/fitting-neural-ode-to-periodic-time-series/42263 "2025-10-13T18:31:03Z")

</div>

Hello, I am trying to fit a neural ODE adapted from an example in the docs to the Mauna Loa dataset. but am running in to some problems. Fitting the model to too many time steps at once causes it to underfit dramatical…

---

## [Wrapping a Flux FCN model with MLJFlux](https://discourse.julialang.org/t/wrapping-a-flux-fcn-model-with-mljflux/132024)

<div class="topic-metadata">

**Author:** [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Replies:** 0\
**Last updated:** [September 1, 2025, 3:51pm UTC](https://discourse.julialang.org/t/wrapping-a-flux-fcn-model-with-mljflux/132024 "2025-09-01T15:51:49Z")

</div>

Cheers, I wonder if it is possible to wrap a fully convolutional network written in Flux (e.g. a U-Net) with MLJFlux, and benefit from the high-level interface from MLJ. FCN outputs are typically multi-dimensional array…

---

## [Is it possible to evaluate the objective function within the "Optimisers.apply!" function?](https://discourse.julialang.org/t/is-it-possible-to-evaluate-the-objective-function-within-the-optimisers-apply-function/130826)

<div class="topic-metadata">

**Author:** [@lorena](https://discourse.julialang.org/u/lorena)\
**Replies:** 2\
**Last updated:** [August 6, 2025, 11:35am UTC](https://discourse.julialang.org/t/is-it-possible-to-evaluate-the-objective-function-within-the-optimisers-apply-function/130826 "2025-08-06T11:35:08Z")

</div>

I am working on implementing the MoMo method for adaptive learning rates in momentum optimizers, and it requires an evaluation of the objective function within the update rule. Until now I have been implementing my own …

---

## [Using Enzyme.jl with Flux: Issues Computing Gradients of a Model with Duplicated Parameters and Mixed Forward/Reverse AD](https://discourse.julialang.org/t/using-enzyme-jl-with-flux-issues-computing-gradients-of-a-model-with-duplicated-parameters-and-mixed-forward-reverse-ad/131135)

<div class="topic-metadata">

**Author:** [@Gianmarco](https://discourse.julialang.org/u/Gianmarco)\
**Replies:** 11\
**Last updated:** [July 29, 2025, 11:23pm UTC](https://discourse.julialang.org/t/using-enzyme-jl-with-flux-issues-computing-gradients-of-a-model-with-duplicated-parameters-and-mixed-forward-reverse-ad/131135 "2025-07-29T23:23:35Z")

</div>

Hi everyone, I’m trying to use Enzyme.jl to compute gradients for a Flux model during training. My goal is to differentiate a custom loss function that combines a regular mean squared error term with an additional gradi…

---

## [Why is the loss function increasing when fitting a line?](https://discourse.julialang.org/t/why-is-the-loss-function-increasing-when-fitting-a-line/130600)

<div class="topic-metadata">

**Author:** [@\_jovian](https://discourse.julialang.org/u/_jovian)\
**Replies:** 2\
**Last updated:** [July 10, 2025, 1:37pm UTC](https://discourse.julialang.org/t/why-is-the-loss-function-increasing-when-fitting-a-line/130600 "2025-07-10T13:37:12Z")

</div>

Hello everyone, I’m following the “Fitting a Line” from the Flux.jl guide. It works as expected with the given data, as well as when modifying the predicted function (e.g. actual(x) = 3x - 1). I was trying to change th…

---

## [PReLU with long compilation time](https://discourse.julialang.org/t/prelu-with-long-compilation-time/129783)

<div class="topic-metadata">

**Author:** [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Replies:** 1\
**Last updated:** [June 12, 2025, 6:24pm UTC](https://discourse.julialang.org/t/prelu-with-long-compilation-time/129783 "2025-06-12T18:24:35Z")

</div>

Cheers, I have implemented a PReLU nonlinearity that seems to be working quite well in my models, except for the fact the model’s compilation time (\*) is way longer than the equivalent model with ReLU instead. Given my…

---

## [\[pre-ANN\] Differentiable FDTD for inverse design in photonics, acoustics and RF](https://discourse.julialang.org/t/pre-ann-differentiable-fdtd-for-inverse-design-in-photonics-acoustics-and-rf/105405)

<div class="topic-metadata">

**Author:** [@pxshen](https://discourse.julialang.org/u/pxshen)\
**Replies:** 54\
**Last updated:** [May 10, 2025, 5:04am UTC](https://discourse.julialang.org/t/pre-ann-differentiable-fdtd-for-inverse-design-in-photonics-acoustics-and-rf/105405 "2025-05-10T05:04:24Z")

</div>

Planning to release a differentiable FDTD package for inverse design in photonics, acoustics and RF. I’ll include examples on designing stacks, gratings and couplers as well as meta-materials and meta-surfaces. The elect…

---

## [Warning: Package cuDNN not found in current path](https://discourse.julialang.org/t/warning-package-cudnn-not-found-in-current-path/104242)

<div class="topic-metadata">

**Author:** [@wsshin](https://discourse.julialang.org/u/wsshin)\
**Replies:** 5\
**Last updated:** [May 8, 2025, 10:22pm UTC](https://discourse.julialang.org/t/warning-package-cudnn-not-found-in-current-path/104242 "2025-05-08T22:22:09Z")

</div>

I get this warning while using Flux on a CUDA GPU: ┌ Warning: Package cuDNN not found in current path. │ - Run \`import Pkg; Pkg.add("cuDNN")\` to install the cuDNN package, then restart julia. │ - If cuDNN is not install…

---

## [.== and .\<= inside Zygote.gradient() are inaccurate on GPU](https://discourse.julialang.org/t/and-inside-zygote-gradient-are-inaccurate-on-gpu/128269)

<div class="topic-metadata">

**Author:** [@ivanightingale](https://discourse.julialang.org/u/ivanightingale)\
**Replies:** 10\
**Last updated:** [April 30, 2025, 3:07am UTC](https://discourse.julialang.org/t/and-inside-zygote-gradient-are-inaccurate-on-gpu/128269 "2025-04-30T03:07:05Z")

</div>

Hello, I’ve encountered some very bizzare behaviors with equality .== and inequality .\<= when using Flux.jl on GPU. Below is a small example: using CUDA using Flux device = gpu\_device() # device = cpu\_device() y = \[0,…

---

## [Memory usage increasing with each epoch](https://discourse.julialang.org/t/memory-usage-increasing-with-each-epoch/121798)

<div class="topic-metadata">

**Author:** [@JoshuaBillson](https://discourse.julialang.org/u/JoshuaBillson)\
**Replies:** 18\
**Last updated:** [April 14, 2025, 12:30pm UTC](https://discourse.julialang.org/t/memory-usage-increasing-with-each-epoch/121798 "2025-04-14T12:30:43Z")

</div>

I’m having a problem where memory usage is gradually increasing with each epoch when training large neural networks with Flux (v0.14.22) and CUDA (v5.5.2). At the same time, training appears to get progressively slower a…

---

## [Improve performance in Flux.jl](https://discourse.julialang.org/t/improve-performance-in-flux-jl/127939)

<div class="topic-metadata">

**Author:** [@Wuaphi](https://discourse.julialang.org/u/Wuaphi)\
**Replies:** 0\
**Last updated:** [April 10, 2025, 4:42pm UTC](https://discourse.julialang.org/t/improve-performance-in-flux-jl/127939 "2025-04-10T16:42:33Z")

</div>

I tried to train a LSTM model in Flux.jl, but whatever I do, the gpu usage only reaches about 50%. But I can’t make the batchsize bigger, as the vram is nearly full. I would appreciate any kind of help. module Lstm …

---

## [GPU backend-agnostic way to create efficiently random number on the GPU](https://discourse.julialang.org/t/gpu-backend-agnostic-way-to-create-efficiently-random-number-on-the-gpu/126110)

<div class="topic-metadata">

**Author:** [@Alexander-Barth](https://discourse.julialang.org/u/Alexander-Barth)\
**Replies:** 3\
**Last updated:** [March 3, 2025, 7:44am UTC](https://discourse.julialang.org/t/gpu-backend-agnostic-way-to-create-efficiently-random-number-on-the-gpu/126110 "2025-03-03T07:44:34Z")

</div>

I am trying to make my code Flux code portable across CUDA and AMDGPU. I would like to know if there is a GPU backend-agnostic way to create efficiently random number on the GPU? AMDGPU.randn is by far the fastest, but I…

---

## [Distributed Data Parallel training with 2 GPUs fails with Flux.jl on AMD GPUs](https://discourse.julialang.org/t/distributed-data-parallel-training-with-2-gpus-fails-with-flux-jl-on-amd-gpus/125993)

<div class="topic-metadata">

**Author:** [@Alexander-Barth](https://discourse.julialang.org/u/Alexander-Barth)\
**Replies:** 7\
**Last updated:** [February 18, 2025, 8:41am UTC](https://discourse.julialang.org/t/distributed-data-parallel-training-with-2-gpus-fails-with-flux-jl-on-amd-gpus/125993 "2025-02-18T08:41:29Z")

</div>

I am using Flux (with this PR Fix missing imports in FluxMPIExt by Alexander-Barth · Pull Request #2589 · FluxML/Flux.jl · GitHub to fix some MPI related imports) on two AMD GPUs with Distributed Data Parallel (DDP). Ho…

---

## [Save model when training with Distributed Data Parallel](https://discourse.julialang.org/t/save-model-when-training-with-distributed-data-parallel/125790)

<div class="topic-metadata">

**Author:** [@niltsz](https://discourse.julialang.org/u/niltsz)\
**Replies:** 0\
**Last updated:** [February 11, 2025, 1:24pm UTC](https://discourse.julialang.org/t/save-model-when-training-with-distributed-data-parallel/125790 "2025-02-11T13:24:45Z")

</div>

Hey all, I want to parallelize my machine learning model on GPUs using DDP from Flux. I get the code to run with adapting what is written on the Flux GPU Support page, but when I try to save the model I get the error u…

---

## [Segfault using Flux + Intel integrated GPU](https://discourse.julialang.org/t/segfault-using-flux-intel-integrated-gpu/125626)

<div class="topic-metadata">

**Author:** [@ed123](https://discourse.julialang.org/u/ed123)\
**Replies:** 2\
**Last updated:** [February 6, 2025, 4:17pm UTC](https://discourse.julialang.org/t/segfault-using-flux-intel-integrated-gpu/125626 "2025-02-06T16:17:44Z")

</div>

I am trying to use Flux on my laptop’s GPU (Intel Iris Xe, Linux) and it immediately segfaults. I realise that this isn’t a powerful GPU but it even happens on a trivial model (see Quick Start · Flux from the Flux docs). …

[Next page](https://discourse.julialang.org/tag/flux/162.md?match_all_tags=true&page=1&tags%5B%5D=flux)