# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=2

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 3

---

## [Fitting neural ODE to periodic time series](https://discourse.julialang.org/t/fitting-neural-ode-to-periodic-time-series/42263)

<div class="topic-metadata">

**Author:** [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)\
**Replies:** 18\
**Last updated:** [October 13, 2025, 6:31pm UTC](https://discourse.julialang.org/t/fitting-neural-ode-to-periodic-time-series/42263 "2025-10-13T18:31:03Z")

</div>

Hello, I am trying to fit a neural ODE adapted from an example in the docs to the Mauna Loa dataset. but am running in to some problems. Fitting the model to too many time steps at once causes it to underfit dramatical…

---

## [How to manipulate gradients in Lux.jl](https://discourse.julialang.org/t/how-to-manipulate-gradients-in-lux-jl/133102)

<div class="topic-metadata">

**Author:** [@IBArbitrary](https://discourse.julialang.org/u/IBArbitrary)\
**Replies:** 0\
**Last updated:** [October 13, 2025, 7:20am UTC](https://discourse.julialang.org/t/how-to-manipulate-gradients-in-lux-jl/133102 "2025-10-13T07:20:22Z")

</div>

I am using Lux.jl, and I want to do algebra on the gradients before applying. Currently I convert them to ComponentArray and convert them back to NamedTuple after I do the necessary algebra. But this requires me to move …

---

## [Type instability in custom Lux masking layer when using Enzyme/Reactant](https://discourse.julialang.org/t/type-instability-in-custom-lux-masking-layer-when-using-enzyme-reactant/133042)

<div class="topic-metadata">

**Author:** [@pietro-butti](https://discourse.julialang.org/u/pietro-butti)\
**Replies:** 5\
**Last updated:** [October 10, 2025, 5:25pm UTC](https://discourse.julialang.org/t/type-instability-in-custom-lux-masking-layer-when-using-enzyme-reactant/133042 "2025-10-10T17:25:01Z")

</div>

I’m encountering issues when implementing a custom Lux layer that performs masking (as commonly used in RealNVPs). Example in 1D with checkerboard masking: \[x1, x2, x3, x4\] → (\[x1,0,x3,0\], \[0,x2,0,x4\]) Goal I want to …

---

## [Speeding up per-sample gradients?](https://discourse.julialang.org/t/speeding-up-per-sample-gradients/109322)

<div class="topic-metadata">

**Author:** [@AlexLewandowski](https://discourse.julialang.org/u/AlexLewandowski)\
**Replies:** 25\
**Last updated:** [October 7, 2025, 5:14pm UTC](https://discourse.julialang.org/t/speeding-up-per-sample-gradients/109322 "2025-10-07T17:14:36Z")

</div>

I am trying to speed up the calculation of per-sample (or per-example) gradients. This can be done via a loop or a jacobian in Julia, but this is slow. The calculation of individual gradients can be batched in several py…

---

## [What lessons could Julia's autodiff ecosystem learn from Stan's TinyGrad?](https://discourse.julialang.org/t/what-lessons-could-julias-autodiff-ecosystem-learn-from-stans-tinygrad/103070)

<div class="topic-metadata">

**Author:** [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Replies:** 45\
**Last updated:** [September 26, 2025, 1:16am UTC](https://discourse.julialang.org/t/what-lessons-could-julias-autodiff-ecosystem-learn-from-stans-tinygrad/103070 "2025-09-26T01:16:10Z")

</div>

This came up in the What steps should the Julia community take to bring Julia to the next level of popularity? megathread and I think it’s worth having a dedicated topic on the point. I’ll merge some existing posts into…

---

## [Reactant.jl compile times and memory usage](https://discourse.julialang.org/t/reactant-jl-compile-times-and-memory-usage/132655)

<div class="topic-metadata">

**Author:** [@f.ij](https://discourse.julialang.org/u/f.ij)\
**Replies:** 9\
**Last updated:** [September 25, 2025, 9:39pm UTC](https://discourse.julialang.org/t/reactant-jl-compile-times-and-memory-usage/132655 "2025-09-25T21:39:41Z")

</div>

I’m trying to do some ML experiments using Lux and Reactant. Without posting all of the code, I was wondering if I’m using Reactant properly, because I’m getting compile times of around 3 minutes, and gigabytes of memory…

---

## [What is the correct way to autodiff a simple CUDA kernel using Reactant.jl and Enzyme.jl](https://discourse.julialang.org/t/what-is-the-correct-way-to-autodiff-a-simple-cuda-kernel-using-reactant-jl-and-enzyme-jl/132572)

<div class="topic-metadata">

**Author:** [@TiangeHuang-Cedric](https://discourse.julialang.org/u/TiangeHuang-Cedric)\
**Replies:** 20\
**Last updated:** [September 23, 2025, 7:58pm UTC](https://discourse.julialang.org/t/what-is-the-correct-way-to-autodiff-a-simple-cuda-kernel-using-reactant-jl-and-enzyme-jl/132572 "2025-09-23T19:58:25Z")

</div>

I’m new to Reactant.jl and Enzyme.jl, and I want to use them to calculate the grad of a CUDA kernel. Here is my script: using Random, Reactant, CUDA, Test, Enzyme const ReactantCUDAExt = Base.get\_extension(Reactant, :R…

---

## [SciMLSensitivity fails on GPU?](https://discourse.julialang.org/t/scimlsensitivity-fails-on-gpu/132575)

<div class="topic-metadata">

**Author:** [@Iris\_Allevi](https://discourse.julialang.org/u/Iris_Allevi)\
**Replies:** 5\
**Last updated:** [September 22, 2025, 9:37pm UTC](https://discourse.julialang.org/t/scimlsensitivity-fails-on-gpu/132575 "2025-09-22T21:37:51Z")

</div>

Hi all, I’m running into a problem when trying to compute gradients of an ODE solution on GPU (with Metal.jl) using SciMLSensitivity (SciMLSensitivity: Automatic Differentiation and Adjoints for (Differential) Equation …

---

## [Make Lux.jl/Zygote.jl choose gradient type based on the superset of the weight type](https://discourse.julialang.org/t/make-lux-jl-zygote-jl-choose-gradient-type-based-on-the-superset-of-the-weight-type/132453)

<div class="topic-metadata">

**Author:** [@IBArbitrary](https://discourse.julialang.org/u/IBArbitrary)\
**Replies:** 0\
**Last updated:** [September 17, 2025, 5:46am UTC](https://discourse.julialang.org/t/make-lux-jl-zygote-jl-choose-gradient-type-based-on-the-superset-of-the-weight-type/132453 "2025-09-17T05:46:01Z")

</div>

I am using Lux.jl to implement models whose weights are complex, and the output can be real or complex. If i keep the output real and the weights complex, I cannot compute gradients because the gradient container is dete…

---

## [\`apply\` do not work anymore with StatefulLuxLayer (by Luxv1.19)](https://discourse.julialang.org/t/apply-do-not-work-anymore-with-statefulluxlayer-by-luxv1-19/132317)

<div class="topic-metadata">

**Author:** [@dmetivie](https://discourse.julialang.org/u/dmetivie)\
**Replies:** 5\
**Last updated:** [September 16, 2025, 1:52pm UTC](https://discourse.julialang.org/t/apply-do-not-work-anymore-with-statefulluxlayer-by-luxv1-19/132317 "2025-09-16T13:52:15Z")

</div>

using Lux, Random nn = Lux.Dense(10, 10, tanh) ps, st = Lux.setup(Xoshiro(2024), nn) nn\_st = Lux.StatefulLuxLayer{true}(nn, nothing, st, nothing) input\_data = rand(10) Lux.apply(nn\_st, input\_data, ps) # do not w…

---

## [NamedTuple from the ComponentArray](https://discourse.julialang.org/t/namedtuple-from-the-componentarray/132000)

<div class="topic-metadata">

**Author:** [@IBArbitrary](https://discourse.julialang.org/u/IBArbitrary)\
**Replies:** 3\
**Last updated:** [September 11, 2025, 6:36am UTC](https://discourse.julialang.org/t/namedtuple-from-the-componentarray/132000 "2025-09-11T06:36:28Z")

</div>

I am using Lux.jl and I want to access the flattened parameter vector. I use ComponentArrays.jl to achieve this. Now, if I have a flattened model vector m0, how do I get the ps back from this? # flattened vector from th…

---

## [Sequence language models in Julia](https://discourse.julialang.org/t/sequence-language-models-in-julia/130306)

<div class="topic-metadata">

**Author:** [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Replies:** 6\
**Last updated:** [August 18, 2025, 1:26am UTC](https://discourse.julialang.org/t/sequence-language-models-in-julia/130306 "2025-08-18T01:26:26Z")

</div>

Dear All, has anyone implemented / ported sequence models like Mamba, Hyena, S4 / S5 to Julia? I would be interested especially in already trained models. Thanks a lot for answer in advance. Tomas

---

## [I heard lux.jl is moving to reactant.jl. Where can I help?](https://discourse.julialang.org/t/i-heard-lux-jl-is-moving-to-reactant-jl-where-can-i-help/131625)

<div class="topic-metadata">

**Author:** [@Tarny\_GG\_Channie](https://discourse.julialang.org/u/Tarny_GG_Channie)\
**Replies:** 4\
**Last updated:** [August 15, 2025, 8:43am UTC](https://discourse.julialang.org/t/i-heard-lux-jl-is-moving-to-reactant-jl-where-can-i-help/131625 "2025-08-15T08:43:05Z")

</div>

I heard lux.jl is moving to reactant.jl. Personally, that’s a really good news. Not only would it probably be something actionable for me (like implementing machine learning layers), it could also help me get closer to r…

---

## [Is it possible to evaluate the objective function within the "Optimisers.apply!" function?](https://discourse.julialang.org/t/is-it-possible-to-evaluate-the-objective-function-within-the-optimisers-apply-function/130826)

<div class="topic-metadata">

**Author:** [@lorena](https://discourse.julialang.org/u/lorena)\
**Replies:** 2\
**Last updated:** [August 6, 2025, 11:35am UTC](https://discourse.julialang.org/t/is-it-possible-to-evaluate-the-objective-function-within-the-optimisers-apply-function/130826 "2025-08-06T11:35:08Z")

</div>

I am working on implementing the MoMo method for adaptive learning rates in momentum optimizers, and it requires an evaluation of the objective function within the update rule. Until now I have been implementing my own …

---

## [Loading Lux params and states in a new script](https://discourse.julialang.org/t/loading-lux-params-and-states-in-a-new-script/131370)

<div class="topic-metadata">

**Author:** [@imantha](https://discourse.julialang.org/u/imantha)\
**Replies:** 1\
**Last updated:** [August 5, 2025, 11:49am UTC](https://discourse.julialang.org/t/loading-lux-params-and-states-in-a-new-script/131370 "2025-08-05T11:49:29Z")

</div>

To save a model the Training a Simple Lstm tutorial shows the following using JDL2 package # train.jl using JLD2 @save "trained\_model.jld2" ps\_trained st\_trained @load "trained\_model.jld2" ps\_trained st\_trained But th…

---

## [How to use Lux.apply for a composite model](https://discourse.julialang.org/t/how-to-use-lux-apply-for-a-composite-model/131331)

<div class="topic-metadata">

**Author:** [@imantha](https://discourse.julialang.org/u/imantha)\
**Replies:** 1\
**Last updated:** [August 5, 2025, 12:44am UTC](https://discourse.julialang.org/t/how-to-use-lux-apply-for-a-composite-model/131331 "2025-08-05T00:44:07Z")

</div>

I am following the CVAE tutorial in Lux (Convolutional VAE for MNIST using Reactant | Lux.jl Docs) to construct a VAE . Everything was all good till I came to the loss function which looks like this function loss\_funct…

---

## [Using Enzyme.jl with Flux: Issues Computing Gradients of a Model with Duplicated Parameters and Mixed Forward/Reverse AD](https://discourse.julialang.org/t/using-enzyme-jl-with-flux-issues-computing-gradients-of-a-model-with-duplicated-parameters-and-mixed-forward-reverse-ad/131135)

<div class="topic-metadata">

**Author:** [@Gianmarco](https://discourse.julialang.org/u/Gianmarco)\
**Replies:** 11\
**Last updated:** [July 29, 2025, 11:23pm UTC](https://discourse.julialang.org/t/using-enzyme-jl-with-flux-issues-computing-gradients-of-a-model-with-duplicated-parameters-and-mixed-forward-reverse-ad/131135 "2025-07-29T23:23:35Z")

</div>

Hi everyone, I’m trying to use Enzyme.jl to compute gradients for a Flux model during training. My goal is to differentiate a custom loss function that combines a regular mean squared error term with an additional gradi…

---

## [Just-in-time gradient optimization idea](https://discourse.julialang.org/t/just-in-time-gradient-optimization-idea/131170)

<div class="topic-metadata">

**Author:** [@Tarny\_GG\_Channie](https://discourse.julialang.org/u/Tarny_GG_Channie)\
**Replies:** 2\
**Last updated:** [July 29, 2025, 10:17pm UTC](https://discourse.julialang.org/t/just-in-time-gradient-optimization-idea/131170 "2025-07-29T22:17:28Z")

</div>

Disclaimer: The idea might look simple in theory, but it might be very hard to implement. Imagine you had a machine learning model comprising multiple batches of parameters. Normally, you differentiate through the entir…

---

## [Accepted mask type for Augmentor.jl](https://discourse.julialang.org/t/accepted-mask-type-for-augmentor-jl/131142)

<div class="topic-metadata">

**Author:** [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Replies:** 0\
**Last updated:** [July 28, 2025, 5:56pm UTC](https://discourse.julialang.org/t/accepted-mask-type-for-augmentor-jl/131142 "2025-07-28T17:56:11Z")

</div>

Cheers, I’m a new Augmentor.jl user. Need to apply same augmentation to images and masks for training a semantic segmentation model. Augmentation of images has worked out smoothly. Issue is when augmenting masks as the …

---

## [Enzyme throws \`alwaysinline mustprogress\` when using GraphNeuralNetworks to mutate Graphs](https://discourse.julialang.org/t/enzyme-throws-alwaysinline-mustprogress-when-using-graphneuralnetworks-to-mutate-graphs/130684)

<div class="topic-metadata">

**Author:** [@alonsoC1s](https://discourse.julialang.org/u/alonsoC1s)\
**Replies:** 6\
**Last updated:** [July 15, 2025, 11:53am UTC](https://discourse.julialang.org/t/enzyme-throws-alwaysinline-mustprogress-when-using-graphneuralnetworks-to-mutate-graphs/130684 "2025-07-15T11:53:46Z")

</div>

I am trying to use GraphNeuralNetworks with Flux in a Deep RL style problem. What I want to achieve by using GraphNeuralNetworks is to obtain new weights for a modifed graph that changes every epoch, and run a simulation…

---

## [NeuralPDE on GPU throws NaNs when I use a source term elevated to some power](https://discourse.julialang.org/t/neuralpde-on-gpu-throws-nans-when-i-use-a-source-term-elevated-to-some-power/114048)

<div class="topic-metadata">

**Author:** [@PetrosStefanou](https://discourse.julialang.org/u/PetrosStefanou)\
**Replies:** 2\
**Last updated:** [July 6, 2025, 1:20am UTC](https://discourse.julialang.org/t/neuralpde-on-gpu-throws-nans-when-i-use-a-source-term-elevated-to-some-power/114048 "2025-07-06T01:20:00Z")

</div>

Hello to everyone, I have a problem with NeuralPDE when trying to use the GPU on the following problem using NeuralPDE using Optimization, OptimizationOptimJL, OptimizationOptimisers import ModelingToolkit: Interval us…

---

## [How to use Lux's \`apply\` function?](https://discourse.julialang.org/t/how-to-use-luxs-apply-function/130339)

<div class="topic-metadata">

**Author:** [@Torkel](https://discourse.julialang.org/u/Torkel)\
**Replies:** 2\
**Last updated:** [June 30, 2025, 5:01pm UTC](https://discourse.julialang.org/t/how-to-use-luxs-apply-function/130339 "2025-06-30T17:01:08Z")

</div>

I am trying to get Lux to work, but am stumbling at a relatively early stage of the tutorials. I am trying to use a combination of the quickstart and getting started pages (Getting Started | Lux.jl Docs, Getting Started …

---

## [Custom differentiation with Flux](https://discourse.julialang.org/t/custom-differentiation-with-flux/130249)

<div class="topic-metadata">

**Author:** [@Fabrice\_Rosay](https://discourse.julialang.org/u/Fabrice_Rosay)\
**Replies:** 2\
**Last updated:** [June 26, 2025, 7:50pm UTC](https://discourse.julialang.org/t/custom-differentiation-with-flux/130249 "2025-06-26T19:50:32Z")

</div>

Currently the derivative of roundis 0. I would like it to be 1 instead and also that it works when broadcasted. So i wrote a custom my\_roundand tried to write the new rule with adjoint and @scalar\_rule but it does not fu…

---

## [Training Neural ODEs - Advice](https://discourse.julialang.org/t/training-neural-odes-advice/130141)

<div class="topic-metadata">

**Author:** [@julianewbsourcream](https://discourse.julialang.org/u/julianewbsourcream)\
**Replies:** 2\
**Last updated:** [June 24, 2025, 1:49pm UTC](https://discourse.julialang.org/t/training-neural-odes-advice/130141 "2025-06-24T13:49:26Z")

</div>

Hi, I am looking for some advice about training neural ODE models. I am looking to train a neural ODE model on a sample of ~25000, 10x4000 matrices. The columns represent time so I was looking to at use an increasing am…

---

## [A custom type-stable regularization function in Lux & Enzyme](https://discourse.julialang.org/t/a-custom-type-stable-regularization-function-in-lux-enzyme/129984)

<div class="topic-metadata">

**Author:** [@user22](https://discourse.julialang.org/u/user22)\
**Replies:** 0\
**Last updated:** [June 18, 2025, 9:44am UTC](https://discourse.julialang.org/t/a-custom-type-stable-regularization-function-in-lux-enzyme/129984 "2025-06-18T09:44:39Z")

</div>

We are trying to develop a custom regularization function for the biases and weights of a simple dense neural network in Lux. However, we have not found a way to establish type-stable gradient computation for Lux and Enz…

---

## [Loading resampled data for PINN in Reactant + Enzyme + Lux](https://discourse.julialang.org/t/loading-resampled-data-for-pinn-in-reactant-enzyme-lux/129974)

<div class="topic-metadata">

**Author:** [@NoFishLikeIan](https://discourse.julialang.org/u/NoFishLikeIan)\
**Replies:** 2\
**Last updated:** [June 17, 2025, 10:58pm UTC](https://discourse.julialang.org/t/loading-resampled-data-for-pinn-in-reactant-enzyme-lux/129974 "2025-06-17T22:58:15Z")

</div>

I am implementing a neural network using Lux and Reactant + Enzyme. At each epoch, I need to resample the data, divide it in batches, and take a training step. For now, I am working on a CPU. What is the best way to do t…

---

## [Issue with backprop in a custom recurrent \`Lux\` layer](https://discourse.julialang.org/t/issue-with-backprop-in-a-custom-recurrent-lux-layer/128968)

<div class="topic-metadata">

**Author:** [@pietro-butti](https://discourse.julialang.org/u/pietro-butti)\
**Replies:** 6\
**Last updated:** [June 16, 2025, 3:44pm UTC](https://discourse.julialang.org/t/issue-with-backprop-in-a-custom-recurrent-lux-layer/128968 "2025-06-16T15:44:58Z")

</div>

I am trying to implement an autoencoder in Lux, inspired by the excellent one that I found in githun.com/lkulowski/LSTM\_encoder\_decoder, coded with PyTorch. (diagram taken from the link) Model overview The encoder i…

---

## [Weird behavior of GraphNeuralNetworks.jl \`SGConv\` layer within \`Flux.withgradient\`](https://discourse.julialang.org/t/weird-behavior-of-graphneuralnetworks-jl-sgconv-layer-within-flux-withgradient/129701)

<div class="topic-metadata">

**Author:** [@MackHarry](https://discourse.julialang.org/u/MackHarry)\
**Replies:** 5\
**Last updated:** [June 14, 2025, 4:34pm UTC](https://discourse.julialang.org/t/weird-behavior-of-graphneuralnetworks-jl-sgconv-layer-within-flux-withgradient/129701 "2025-06-14T16:34:32Z")

</div>

First, I am quite new to Julia’s ML ecosystem, so I might have made stupid mistakes.. Recently, I tried to debug a very training loop involving GraphNeuralNetworks.jl’s SGConv layer, and found successively longer runti…

---

## [Native Julia FID (Fréchet Inception Distance) Computation](https://discourse.julialang.org/t/native-julia-fid-frechet-inception-distance-computation/129798)

<div class="topic-metadata">

**Author:** [@josemanuel22](https://discourse.julialang.org/u/josemanuel22)\
**Replies:** 0\
**Last updated:** [June 11, 2025, 10:00am UTC](https://discourse.julialang.org/t/native-julia-fid-frechet-inception-distance-computation/129798 "2025-06-11T10:00:18Z")

</div>

Is there a relatively simple way to calculate the FID (Fréchet Inception Distance) metric in Julia? Metalhead doesn’t include a pre-trained Inception-v3 model. I know you can try ONNX or PythonCall (though all of those h…

---

## [Type stability of \`Lux.batched\_jacobian\`](https://discourse.julialang.org/t/type-stability-of-lux-batched-jacobian/128526)

<div class="topic-metadata">

**Author:** [@NoFishLikeIan](https://discourse.julialang.org/u/NoFishLikeIan)\
**Replies:** 5\
**Last updated:** [June 10, 2025, 8:56am UTC](https://discourse.julialang.org/t/type-stability-of-lux-batched-jacobian/128526 "2025-06-10T08:56:01Z")

</div>

How can I make the call to Lux.batched\_jacobian of a StatefulLuxLayer type stable? I am using Lux v1.12.4. Consider this MWE: using Lux using Random; rng = Xoshiro(42) input, output = 6, 2 model = Chain(Dense(input =\> …

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=1)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=3)