# \#neural-network

**URL:** https://discourse.julialang.org/tag/neural-network/654.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [Spiking Neural Networks](https://discourse.julialang.org/t/spiking-neural-networks/63270)

<div class="topic-metadata">

**Author:** [@mefob](https://discourse.julialang.org/u/mefob)\
**Replies:** 33\
**Last updated:** [July 9, 2026, 4:32pm UTC](https://discourse.julialang.org/t/spiking-neural-networks/63270 "2026-07-09T16:32:19Z")

</div>

What is the state of the art for modeling spiking neural networks in Julia? I haven’t found much recent work: Spiking Neural Network (Discussion from 2019 on implementing a SNN as an ODE) GitHub - AStupidBear/Spikin…

---

## [Invertible Neural Networks](https://discourse.julialang.org/t/invertible-neural-networks/137909)

<div class="topic-metadata">

**Author:** [@langestefan](https://discourse.julialang.org/u/langestefan)\
**Replies:** 0\
**Last updated:** [July 2, 2026, 2:40pm UTC](https://discourse.julialang.org/t/invertible-neural-networks/137909 "2026-07-02T14:40:20Z")

</div>

I was looking at doing some modelling work using invertible neural-networks in Julia with Lux.jl. There aren’t many options, but I found GitHub - slimgroup/InvertibleNetworks.jl: A Julia framework for invertible neural …

---

## [How to train dense nets several times faster than with Adam](https://discourse.julialang.org/t/how-to-train-dense-nets-several-times-faster-than-with-adam/137009)

<div class="topic-metadata">

**Author:** [@uwestoehr](https://discourse.julialang.org/u/uwestoehr)\
**Replies:** 4\
**Last updated:** [May 8, 2026, 10:36am UTC](https://discourse.julialang.org/t/how-to-train-dense-nets-several-times-faster-than-with-adam/137009 "2026-05-08T10:36:59Z")

</div>

I had the pleasure to give a lecture about the basics of neural nets at a university and purposely used Julia also for the practical lessons. As a side effect I wanted to explain my students how an optimizer can be deri…

---

## [\[ANN\] NeuralEstimators.jl: Efficient simulation-based inference (SBI) using neural networks](https://discourse.julialang.org/t/ann-neuralestimators-jl-efficient-simulation-based-inference-sbi-using-neural-networks/136917)

<div class="topic-metadata">

**Author:** [@MattSainsbury-Dale](https://discourse.julialang.org/u/MattSainsbury-Dale)\
**Replies:** 3\
**Last updated:** [April 30, 2026, 11:47am UTC](https://discourse.julialang.org/t/ann-neuralestimators-jl-efficient-simulation-based-inference-sbi-using-neural-networks/136917 "2026-04-30T11:47:20Z")

</div>

NeuralEstimators.jl uses neural networks for fast simulation-based inference (SBI) for any model for which simulation is feasible. It supports: Neural posterior estimation (NPE): directly learn the posterior distributi…

---

## [NeatEvolution.jl: Neuroevolution of Augmenting Topologies](https://discourse.julialang.org/t/neatevolution-jl-neuroevolution-of-augmenting-topologies/136107)

<div class="topic-metadata">

**Author:** [@CodeReclaimers](https://discourse.julialang.org/u/CodeReclaimers)\
**Replies:** 0\
**Last updated:** [March 9, 2026, 8:19pm UTC](https://discourse.julialang.org/t/neatevolution-jl-neuroevolution-of-augmenting-topologies/136107 "2026-03-09T20:19:03Z")

</div>

Hi folks, I’m the maintainer of the neat-python package, and I finally got around to porting it to Julia. I tried to make sure it adheres to the original NEAT paper and has about the same set of capabilities as neat-py…

---

## [Correctness Issue in NeuralOperators.jl: Mode truncation in SpectralConv excludes negative low-frequency modes](https://discourse.julialang.org/t/correctness-issue-in-neuraloperators-jl-mode-truncation-in-spectralconv-excludes-negative-low-frequency-modes/134403)

<div class="topic-metadata">

**Author:** [@Azamat](https://discourse.julialang.org/u/Azamat)\
**Replies:** 2\
**Last updated:** [December 9, 2025, 8:26am UTC](https://discourse.julialang.org/t/correctness-issue-in-neuraloperators-jl-mode-truncation-in-spectralconv-excludes-negative-low-frequency-modes/134403 "2025-12-09T08:26:17Z")

</div>

I was looking at the mode truncation logic in NeuralOperators.jl and have a question about correctness. Link: NeuralOperators.jl/src/transform.jl at 39f8a3c6dc974e711c8de8a8fde804e096098157 · SciML/NeuralOperators.jl · …

---

## [Has anyone tried implementing the LiMuon Optimizer?](https://discourse.julialang.org/t/has-anyone-tried-implementing-the-limuon-optimizer/132742)

<div class="topic-metadata">

**Author:** [@DoktorMike](https://discourse.julialang.org/u/DoktorMike)\
**Replies:** 2\
**Last updated:** [November 7, 2025, 8:33pm UTC](https://discourse.julialang.org/t/has-anyone-tried-implementing-the-limuon-optimizer/132742 "2025-11-07T20:33:54Z")

</div>

Has anyone tried implementing the LiMuon optimizer? I made an issue here in the Optimisers.jl package but thought I’d ask here as well if someone has implemented it elsewhere? Also any hands on experience with the metho…

---

## [Lotka Volterra Neural ODE](https://discourse.julialang.org/t/lotka-volterra-neural-ode/133159)

<div class="topic-metadata">

**Author:** [@Aditya\_Deshpande](https://discourse.julialang.org/u/Aditya_Deshpande)\
**Replies:** 5\
**Last updated:** [October 19, 2025, 9:36am UTC](https://discourse.julialang.org/t/lotka-volterra-neural-ode/133159 "2025-10-19T09:36:47Z")

</div>

Hello All, Just started learning about neural ODEs, i was trying to solve the Lotka Volterra equation using neural ODE. here is my code using JLD, Lux, DiffEqFlux, DifferentialEquations, Optimization, OptimizationOpt…

---

## [Is it possible to evaluate the objective function within the "Optimisers.apply!" function?](https://discourse.julialang.org/t/is-it-possible-to-evaluate-the-objective-function-within-the-optimisers-apply-function/130826)

<div class="topic-metadata">

**Author:** [@lorena](https://discourse.julialang.org/u/lorena)\
**Replies:** 2\
**Last updated:** [August 6, 2025, 11:35am UTC](https://discourse.julialang.org/t/is-it-possible-to-evaluate-the-objective-function-within-the-optimisers-apply-function/130826 "2025-08-06T11:35:08Z")

</div>

I am working on implementing the MoMo method for adaptive learning rates in momentum optimizers, and it requires an evaluation of the objective function within the update rule. Until now I have been implementing my own …

---

## [NeuralIntegrator: Spiking neural network and synaptic plasticity in DifferentialEquations.jl](https://discourse.julialang.org/t/neuralintegrator-spiking-neural-network-and-synaptic-plasticity-in-differentialequations-jl/130295)

<div class="topic-metadata">

**Author:** [@flmuk](https://discourse.julialang.org/u/flmuk)\
**Replies:** 1\
**Last updated:** [July 1, 2025, 8:09am UTC](https://discourse.julialang.org/t/neuralintegrator-spiking-neural-network-and-synaptic-plasticity-in-differentialequations-jl/130295 "2025-07-01T08:09:13Z")

</div>

NeuralIntegrator: simulating spiking neural networks with discontinuities I work in the field of computational neuroscience and model spiking neural networks (SNNs) with different synaptic plasticity rules, i.e. rules o…

---

## [How to use Lux's \`apply\` function?](https://discourse.julialang.org/t/how-to-use-luxs-apply-function/130339)

<div class="topic-metadata">

**Author:** [@Torkel](https://discourse.julialang.org/u/Torkel)\
**Replies:** 2\
**Last updated:** [June 30, 2025, 5:01pm UTC](https://discourse.julialang.org/t/how-to-use-luxs-apply-function/130339 "2025-06-30T17:01:08Z")

</div>

I am trying to get Lux to work, but am stumbling at a relatively early stage of the tutorials. I am trying to use a combination of the quickstart and getting started pages (Getting Started | Lux.jl Docs, Getting Started …

---

## [Using \`Lux.WrappedFunction\` for pre/post processing in Lux model](https://discourse.julialang.org/t/using-lux-wrappedfunction-for-pre-post-processing-in-lux-model/129254)

<div class="topic-metadata">

**Author:** [@facusapienza](https://discourse.julialang.org/u/facusapienza)\
**Replies:** 5\
**Last updated:** [May 22, 2025, 11:36pm UTC](https://discourse.julialang.org/t/using-lux-wrappedfunction-for-pre-post-processing-in-lux-model/129254 "2025-05-22T23:36:22Z")

</div>

Hi! I recently learned about Lux.WrappedFunction() layer (docs) that allows to prescribed a parameter less function inside the Chain. I think this is a great feature to allow building of preprocessing and postprocessing…

---

## [Error when a neural ode is implemented](https://discourse.julialang.org/t/error-when-a-neural-ode-is-implemented/119963)

<div class="topic-metadata">

**Author:** [@Ashima\_Kalathingal](https://discourse.julialang.org/u/Ashima_Kalathingal)\
**Replies:** 3\
**Last updated:** [May 2, 2025, 8:09pm UTC](https://discourse.julialang.org/t/error-when-a-neural-ode-is-implemented/119963 "2025-05-02T20:09:03Z")

</div>

I am trying to implement a neural ode in Julia. The loss function reduces when training. But after the training the parameters are not updated. It goes back to initial value. When I run the algorithm the following error…

---

## [Getting a LoadError: GPUCompiler.InvalidIRError on trying to use LuxCUDA with EnsembleProblem](https://discourse.julialang.org/t/getting-a-loaderror-gpucompiler-invalidirerror-on-trying-to-use-luxcuda-with-ensembleproblem/128328)

<div class="topic-metadata">

**Author:** [@sunaina2821](https://discourse.julialang.org/u/sunaina2821)\
**Replies:** 3\
**Last updated:** [April 25, 2025, 11:02am UTC](https://discourse.julialang.org/t/getting-a-loaderror-gpucompiler-invalidirerror-on-trying-to-use-luxcuda-with-ensembleproblem/128328 "2025-04-25T11:02:16Z")

</div>

Here’s a MWE for my problem. I’ve been trying to use Lux with GPU and have been working on a similar problem as the MWE. The error results from the U(u, p, st)\[1\] line in most cases. At times I get it due to scalar index…

---

## [What is the best way to implement PINN in Julia](https://discourse.julialang.org/t/what-is-the-best-way-to-implement-pinn-in-julia/127713)

<div class="topic-metadata">

**Author:** [@Yang-yang](https://discourse.julialang.org/u/Yang-yang)\
**Replies:** 18\
**Last updated:** [April 6, 2025, 2:37pm UTC](https://discourse.julialang.org/t/what-is-the-best-way-to-implement-pinn-in-julia/127713 "2025-04-06T14:37:28Z")

</div>

Dear all, I want to use PINNs to solve a PDE. There are two packages in Julia that can do this NeuralPDE.jl and Sophon.jl Both of these packages use the finite difference method to realize derivatives. I want to know i…

---

## [Type instability for simple MLP call using ComponentArrays](https://discourse.julialang.org/t/type-instability-for-simple-mlp-call-using-componentarrays/126191)

<div class="topic-metadata">

**Author:** [@lasse](https://discourse.julialang.org/u/lasse)\
**Replies:** 2\
**Last updated:** [March 6, 2025, 5:17pm UTC](https://discourse.julialang.org/t/type-instability-for-simple-mlp-call-using-componentarrays/126191 "2025-03-06T17:17:58Z")

</div>

Hi, Is there a way to make Zygote gradients type stable when optimizing over a ComponentArray of parameters? If not - what is a good alternative that is compatible with the SciML ecosystem? Using @NamedTuple does as I …

---

## [Training Universal PINNs using NeuralPDE?](https://discourse.julialang.org/t/training-universal-pinns-using-neuralpde/125948)

<div class="topic-metadata">

**Author:** [@John\_Nardini](https://discourse.julialang.org/u/John_Nardini)\
**Replies:** 4\
**Last updated:** [February 18, 2025, 2:34pm UTC](https://discourse.julialang.org/t/training-universal-pinns-using-neuralpde/125948 "2025-02-18T14:34:44Z")

</div>

I am interested Universal PINNs (UPINNs) to describe some data. UPINNs are similar to a PINN, but we assume that the underlying DE model has known and unknown dynamics: \\dfrac{d\\vec{u}}{dt} = F\_{known}(\\vec{u}) + F\_{unk…

---

## [Some thoughts and questions on smooth non-linear optimization, degrees of freedom, optimizers, MLPs](https://discourse.julialang.org/t/some-thoughts-and-questions-on-smooth-non-linear-optimization-degrees-of-freedom-optimizers-mlps/125892)

<div class="topic-metadata">

**Author:** [@BdeKoning](https://discourse.julialang.org/u/BdeKoning)\
**Replies:** 0\
**Last updated:** [February 14, 2025, 7:02am UTC](https://discourse.julialang.org/t/some-thoughts-and-questions-on-smooth-non-linear-optimization-degrees-of-freedom-optimizers-mlps/125892 "2025-02-14T07:02:43Z")

</div>

Although I’d like to pose my thoughts and questions on a quite general group of optimization problems, my thoughts on these topics came from working on the following optimization problem: The second lens surface is a…

---

## [Lux.jl QuickStart: why \`Lux.apply\` returns the state?](https://discourse.julialang.org/t/lux-jl-quickstart-why-lux-apply-returns-the-state/125607)

<div class="topic-metadata">

**Author:** [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Replies:** 1\
**Last updated:** [February 6, 2025, 9:55am UTC](https://discourse.julialang.org/t/lux-jl-quickstart-why-lux-apply-returns-the-state/125607 "2025-02-06T09:55:10Z")

</div>

In the QuickStart section of Lux.jl documentation, there is the line y, st = Lux.apply(model, x, ps, st) Why does Lux.apply return not only y but also st? How can inference computations change the neural network state?…

---

## [How to compute TanhExp / TeLU function accurately?](https://discourse.julialang.org/t/how-to-compute-tanhexp-telu-function-accurately/124464)

<div class="topic-metadata">

**Author:** [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Replies:** 10\
**Last updated:** [January 6, 2025, 5:07pm UTC](https://discourse.julialang.org/t/how-to-compute-tanhexp-telu-function-accurately/124464 "2025-01-06T17:07:34Z")

</div>

A new paper advocates using the TeLU activation function, x \\cdot \\tanh (e^x), for neural networks, also proposed in this paper under the name TanhExp. The intermediate result e^x grows very fast for large x, even thoug…

---

## [Forecast Time series using Neural ODE for supervised learning](https://discourse.julialang.org/t/forecast-time-series-using-neural-ode-for-supervised-learning/124268)

<div class="topic-metadata">

**Author:** [@roilsed0903](https://discourse.julialang.org/u/roilsed0903)\
**Replies:** 0\
**Last updated:** [December 30, 2024, 5:59am UTC](https://discourse.julialang.org/t/forecast-time-series-using-neural-ode-for-supervised-learning/124268 "2024-12-30T05:59:43Z")

</div>

I’m aiming for time series prediction. I’ve seen a good prediction of time series through the post forcast weather NODE(“Forecasting the weather with neural ODEs” taked Sebastian Callh personal blog and “Forcasting time …

---

## [Scaling issue (probably) while solving simple PINN problem](https://discourse.julialang.org/t/scaling-issue-probably-while-solving-simple-pinn-problem/124183)

<div class="topic-metadata">

**Author:** [@nico](https://discourse.julialang.org/u/nico)\
**Replies:** 3\
**Last updated:** [December 27, 2024, 4:04pm UTC](https://discourse.julialang.org/t/scaling-issue-probably-while-solving-simple-pinn-problem/124183 "2024-12-27T16:04:14Z")

</div>

Hi All! I am trying to solve a simple ODE with the PINN technology. There might be better ways to do that, but I am using it to test and understand few practical things. This code worked perfectly as long as the probl…

---

## [How to optimize neural network parameters with non-gradient algorithms?](https://discourse.julialang.org/t/how-to-optimize-neural-network-parameters-with-non-gradient-algorithms/124186)

<div class="topic-metadata">

**Author:** [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Replies:** 2\
**Last updated:** [December 26, 2024, 7:41pm UTC](https://discourse.julialang.org/t/how-to-optimize-neural-network-parameters-with-non-gradient-algorithms/124186 "2024-12-26T19:41:58Z")

</div>

Is there any simple example code for optimizing neural network (e.g. from Flux.jl) parameters using non-gradient algorithms such as simulated annealing and genetic algorithms?

---

## [How to generate values after training with Lux.LayerNorm and Lux.BatchNorm](https://discourse.julialang.org/t/how-to-generate-values-after-training-with-lux-layernorm-and-lux-batchnorm/102227)

<div class="topic-metadata">

**Author:** [@Ethan\_Tran](https://discourse.julialang.org/u/Ethan_Tran)\
**Replies:** 5\
**Last updated:** [December 26, 2024, 10:50am UTC](https://discourse.julialang.org/t/how-to-generate-values-after-training-with-lux-layernorm-and-lux-batchnorm/102227 "2024-12-26T10:50:15Z")

</div>

I tried to use Lux.LayerNorm and Dense to train the ODE system on NeuralPDE as follows: domains = \[t ∈ Interval(0.0,25.0)\] chain =\[Lux.Chain(Lux.LayerNorm((1,),Lux.relu),Dense(1,10,Lux.tanh),Dense(10,20,Lux.tanh),Dense…

---

## [NeuralPDE.jl slow with integro diff. equations](https://discourse.julialang.org/t/neuralpde-jl-slow-with-integro-diff-equations/122511)

<div class="topic-metadata">

**Author:** [@nico](https://discourse.julialang.org/u/nico)\
**Replies:** 3\
**Last updated:** [November 11, 2024, 8:42pm UTC](https://discourse.julialang.org/t/neuralpde-jl-slow-with-integro-diff-equations/122511 "2024-11-11T20:42:40Z")

</div>

Is there a reason why with integro-differential equations NeuralPDE.jl becomes very slow? Is there a better way to deal with integrals than with, e.g., Ii = Symbolics.Integral(t in DomainSets.ClosedInterval(0, t)) ?

---

## [Performing max-min optimization with optimization.jl](https://discourse.julialang.org/t/performing-max-min-optimization-with-optimization-jl/121991)

<div class="topic-metadata">

**Author:** [@KianH](https://discourse.julialang.org/u/KianH)\
**Replies:** 0\
**Last updated:** [October 30, 2024, 12:36pm UTC](https://discourse.julialang.org/t/performing-max-min-optimization-with-optimization-jl/121991 "2024-10-30T12:36:01Z")

</div>

Hello, I was wondering if it is possible to solve minimax optimization problems using Optimization.jl for neural networks? I am currently performing the optimization using Optimization.solve(...). I am specifically inte…

---

## [Error using example for NeuralODE](https://discourse.julialang.org/t/error-using-example-for-neuralode/121697)

<div class="topic-metadata">

**Author:** [@agme2014](https://discourse.julialang.org/u/agme2014)\
**Replies:** 4\
**Last updated:** [October 25, 2024, 1:45pm UTC](https://discourse.julialang.org/t/error-using-example-for-neuralode/121697 "2024-10-25T13:45:53Z")

</div>

Getting an error in the last optimization step : result\_neuralode2 = Optimization.solve( optprob2, Optim.BFGS(; initial\_stepnorm = 0.01); callback, allow\_f\_increases = false) \*\*Error message : \*\* \*\*ERROR: Output …

---

## [CUDA Error : ArgumentError: Objects are on devices with different types: CPUDevice and CUDADevice](https://discourse.julialang.org/t/cuda-error-argumenterror-objects-are-on-devices-with-different-types-cpudevice-and-cudadevice/121651)

<div class="topic-metadata">

**Author:** [@agme2014](https://discourse.julialang.org/u/agme2014)\
**Replies:** 4\
**Last updated:** [October 23, 2024, 6:11pm UTC](https://discourse.julialang.org/t/cuda-error-argumenterror-objects-are-on-devices-with-different-types-cpudevice-and-cudadevice/121651 "2024-10-23T18:11:49Z")

</div>

Hi, I am trying to use CUDA to speed up my code. I am running into an error at the last step, where optimization is used for the iterations This is the message I am getting : ERROR: ArgumentError: Objects are on device…

---

## [Using trained neural networks inside GPU computations](https://discourse.julialang.org/t/using-trained-neural-networks-inside-gpu-computations/121463)

<div class="topic-metadata">

**Author:** [@tduretz](https://discourse.julialang.org/u/tduretz)\
**Replies:** 0\
**Last updated:** [October 18, 2024, 4:31pm UTC](https://discourse.julialang.org/t/using-trained-neural-networks-inside-gpu-computations/121463 "2024-10-18T16:31:01Z")

</div>

Hi all, We run simulations which rely on iterative solvers on GPU using tools like ParallelStencil or Chmy. Part of computations involve the evaluation of heavy non-linear functions that can be shortcut by (1) training…

---

## [Nested and different AD methods altogether: How to add AD calculations inside my loss function when using neural differential equations?](https://discourse.julialang.org/t/nested-and-different-ad-methods-altogether-how-to-add-ad-calculations-inside-my-loss-function-when-using-neural-differential-equations/108985)

<div class="topic-metadata">

**Author:** [@facusapienza](https://discourse.julialang.org/u/facusapienza)\
**Replies:** 9\
**Last updated:** [September 28, 2024, 12:16am UTC](https://discourse.julialang.org/t/nested-and-different-ad-methods-altogether-how-to-add-ad-calculations-inside-my-loss-function-when-using-neural-differential-equations/108985 "2024-09-28T00:16:46Z")

</div>

Hi all, I am implementing regularization penalties inside Universal Differential Equations (also applicable to Physics-Informed neural networks) where I need to differentiate a (loss) function that includes in its calcu…

[Next page](https://discourse.julialang.org/tag/neural-network/654.md?match_all_tags=true&page=1&tags%5B%5D=neural-network)