# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=31

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 32

---

## [Flux is lagging far beyond tensorflow with a pretty basic use case](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942)

<div class="topic-metadata">

**Author:** [@egolep](https://discourse.julialang.org/u/egolep)\
**Replies:** 5\
**Last updated:** [September 9, 2021, 4:43pm UTC](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942 "2021-09-09T16:43:36Z")

</div>

I was trying to move from a Python ML/DL stack to Julia (so from something like sklearn+pytorch/tensorflow to MLJ+Flux). So I decided to rewrite the Kaggle courses in Julia, switching from the Python libraries to the J…

---

## [UndefVarError while importing CUDA](https://discourse.julialang.org/t/undefvarerror-while-importing-cuda/67872)

<div class="topic-metadata">

**Author:** [@Hamza\_Coban](https://discourse.julialang.org/u/Hamza_Coban)\
**Replies:** 3\
**Last updated:** [September 8, 2021, 3:15pm UTC](https://discourse.julialang.org/t/undefvarerror-while-importing-cuda/67872 "2021-09-08T15:15:11Z")

</div>

Hi all, I have issues on importing CUDA package. Only active packages are Knet and CUDA. The error as follows when I run “import CUDA” : Info: Precompiling CUDA \[052768ef-5323-5732-b1bb-66c8b64840ba\] └ @ Base loading.…

---

## [State of automatic differentiation in Julia](https://discourse.julialang.org/t/state-of-automatic-differentiation-in-julia/43083)

<div class="topic-metadata">

**Author:** [@Samuel\_Ainsworth](https://discourse.julialang.org/u/Samuel_Ainsworth)\
**Replies:** 56\
**Last updated:** [September 8, 2021, 2:16pm UTC](https://discourse.julialang.org/t/state-of-automatic-differentiation-in-julia/43083 "2021-09-08T14:16:12Z")

</div>

There seems to be a plethora of AD frameworks in Julia. I’m curious what the state of the ecosystem looks like currently. I’m aware of the following packages: FowardDiff ForwardDiff2 Nabla Tracker Yota Zygote ReverseD…

---

## [Simple Regression using DiffEqFlux](https://discourse.julialang.org/t/simple-regression-using-diffeqflux/66842)

<div class="topic-metadata">

**Author:** [@amit1](https://discourse.julialang.org/u/amit1)\
**Replies:** 6\
**Last updated:** [September 8, 2021, 6:27am UTC](https://discourse.julialang.org/t/simple-regression-using-diffeqflux/66842 "2021-09-08T06:27:53Z")

</div>

I am trying to work out a simple application of NeuralODE for regression problems, based on the following tutorial. Core idea here being (as per my understanding) that regression problems can be modelled as y = \\int\_{…

---

## [ChainRulesCore: Custom adjoint ignored in certain cases](https://discourse.julialang.org/t/chainrulescore-custom-adjoint-ignored-in-certain-cases/67814)

<div class="topic-metadata">

**Author:** [@Rasmus\_Hoier](https://discourse.julialang.org/u/Rasmus_Hoier)\
**Replies:** 3\
**Last updated:** [September 7, 2021, 2:32pm UTC](https://discourse.julialang.org/t/chainrulescore-custom-adjoint-ignored-in-certain-cases/67814 "2021-09-07T14:32:10Z")

</div>

Hi, A colleague of mine ran into the issue that custom adjoints get ignored in certain cases in recent versions of ChainRulesCore. Here is a minimal working example: using Flux, ChainRulesCore my\_fun(x) = x\*x …

---

## [Getting strange "UndefVarError: xs not defined" in Zygote / Flux](https://discourse.julialang.org/t/getting-strange-undefvarerror-xs-not-defined-in-zygote-flux/67707)

<div class="topic-metadata">

**Author:** [@outlace](https://discourse.julialang.org/u/outlace)\
**Replies:** 3\
**Last updated:** [September 5, 2021, 10:18pm UTC](https://discourse.julialang.org/t/getting-strange-undefvarerror-xs-not-defined-in-zygote-flux/67707 "2021-09-05T22:18:41Z")

</div>

I’ve got a fairly simple custom 2-layer MLP implementation but I get this strange error when trying to get the gradient: LoadError: UndefVarError: xs not defined UndefVarError: xs not defined Here’s my full code using…

---

## [Will Flux/Zygote compute gradients sparsely?](https://discourse.julialang.org/t/will-flux-zygote-compute-gradients-sparsely/67542)

<div class="topic-metadata">

**Author:** [@outlace](https://discourse.julialang.org/u/outlace)\
**Replies:** 7\
**Last updated:** [September 4, 2021, 10:30pm UTC](https://discourse.julialang.org/t/will-flux-zygote-compute-gradients-sparsely/67542 "2021-09-04T22:30:03Z")

</div>

So in short I want to train an MLP by sparsely updating it’s parameters. I do this on the forward pass by randomly subsampling the columns of my parameter matrices. E.g. for a simple 2 layer model on MINST with input dat…

---

## [Use ForwardDiff instead of Zygote with Flux?](https://discourse.julialang.org/t/use-forwarddiff-instead-of-zygote-with-flux/67572)

<div class="topic-metadata">

**Author:** [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Replies:** 10\
**Last updated:** [September 3, 2021, 12:35pm UTC](https://discourse.julialang.org/t/use-forwarddiff-instead-of-zygote-with-flux/67572 "2021-09-03T12:35:36Z")

</div>

I’m embedding a Flux model into a physical simulation but Zygote doesn’t seem to be able to handle taking the gradient. I got ForwardDiff.gradient working with the code yesterday. Can I use ForwardDiff as the backend in…

---

## [CUDNN operators not supporting Views/SubArrays](https://discourse.julialang.org/t/cudnn-operators-not-supporting-views-subarrays/67463)

<div class="topic-metadata">

**Author:** [@jeremiedb](https://discourse.julialang.org/u/jeremiedb)\
**Replies:** 3\
**Last updated:** [September 3, 2021, 8:05am UTC](https://discourse.julialang.org/t/cudnn-operators-not-supporting-views-subarrays/67463 "2021-09-03T08:05:58Z")

</div>

Is there a reason for Convolution (and other CUDNN related functions) to be restricted to DenseCuArray as found in the NNlibCUDA.jl repo? For example, taking a view here will result in an error as CUDA has to resort to …

---

## [Neural ODE with Evolutionary Parameters](https://discourse.julialang.org/t/neural-ode-with-evolutionary-parameters/65736)

<div class="topic-metadata">

**Author:** [@Saad](https://discourse.julialang.org/u/Saad)\
**Replies:** 7\
**Last updated:** [August 31, 2021, 4:17pm UTC](https://discourse.julialang.org/t/neural-ode-with-evolutionary-parameters/65736 "2021-08-31T16:17:02Z")

</div>

Hello everyone, I would like to know if there is a package on Julia adapted to Neural ODEs with time dependent parameters, or if there is a practical example of code use to allow evolution of the neural network paramete…

---

## [Flux: Hidden problem broadcasting convolution layer on GPU?](https://discourse.julialang.org/t/flux-hidden-problem-broadcasting-convolution-layer-on-gpu/67316)

<div class="topic-metadata">

**Author:** [@MattSainsbury-Dale](https://discourse.julialang.org/u/MattSainsbury-Dale)\
**Replies:** 1\
**Last updated:** [August 30, 2021, 3:13am UTC](https://discourse.julialang.org/t/flux-hidden-problem-broadcasting-convolution-layer-on-gpu/67316 "2021-08-30T03:13:42Z")

</div>

I found the following comment in the source code for the Conv layer for Flux.jl: function (c::Conv)(x::AbstractArray) # TODO: breaks gpu broadcast :( It’s a bit concerning, given that I am currently working on a proj…

---

## [Speeding up gradients for custom neural network - currently much slower than in PyTorch](https://discourse.julialang.org/t/speeding-up-gradients-for-custom-neural-network-currently-much-slower-than-in-pytorch/50637)

<div class="topic-metadata">

**Author:** [@Andreas\_Schlaginhauf](https://discourse.julialang.org/u/Andreas_Schlaginhauf)\
**Replies:** 16\
**Last updated:** [August 28, 2021, 4:52pm UTC](https://discourse.julialang.org/t/speeding-up-gradients-for-custom-neural-network-currently-much-slower-than-in-pytorch/50637 "2021-08-28T16:52:49Z")

</div>

I am currently trying to implement stable neural ODEs according to https://arxiv.org/pdf/2001.06116.pdf. This includes an ODE function, (m::StableDynamics)(x) = begin if x == zero(x) return 0 end vx …

---

## [How to save optimiser state and the model of CGAN?](https://discourse.julialang.org/t/how-to-save-optimiser-state-and-the-model-of-cgan/67232)

<div class="topic-metadata">

**Author:** [@extremety1989](https://discourse.julialang.org/u/extremety1989)\
**Replies:** 0\
**Last updated:** [August 28, 2021, 4:07pm UTC](https://discourse.julialang.org/t/how-to-save-optimiser-state-and-the-model-of-cgan/67232 "2021-08-28T16:07:22Z")

</div>

link : https://github.com/FluxML/model-zoo/blob/master/vision/cdcgan\_mnist/cGAN\_mnist.jl i tried inside training loop: @save "gen-model-$(now()).bson" gen opt\_gen @save "dscr-model-$(now()).bson" dscr opt\_dscr ERROR: L…

---

## [Need help to translate this Pytorch part to Flux](https://discourse.julialang.org/t/need-help-to-translate-this-pytorch-part-to-flux/67032)

<div class="topic-metadata">

**Author:** [@extremety1989](https://discourse.julialang.org/u/extremety1989)\
**Replies:** 6\
**Last updated:** [August 26, 2021, 6:56pm UTC](https://discourse.julialang.org/t/need-help-to-translate-this-pytorch-part-to-flux/67032 "2021-08-26T18:56:30Z")

</div>

i already made some progress Pytorch code class ResidualBlock(nn.Module): def \_\_init\_\_(self, in\_channels, channels=32, residual\_beta=0.2): super().\_\_init\_\_() self.residual\_beta = residual\_beta …

---

## [NeuralODE not taking multiple inputs although neural network is able to do so](https://discourse.julialang.org/t/neuralode-not-taking-multiple-inputs-although-neural-network-is-able-to-do-so/67042)

<div class="topic-metadata">

**Author:** [@DiracFermion1](https://discourse.julialang.org/u/DiracFermion1)\
**Replies:** 0\
**Last updated:** [August 26, 2021, 2:58pm UTC](https://discourse.julialang.org/t/neuralode-not-taking-multiple-inputs-although-neural-network-is-able-to-do-so/67042 "2021-08-26T14:58:15Z")

</div>

(this question is related to How to use multiple inputs in \`NeuralODE\` object? with a concrete example) Hi, I am trying to define a NeuralODE that takes two inputs (one input is a Float array, another is an Int array). …

---

## [Approximating sine function using Flux for Solving PINN problem](https://discourse.julialang.org/t/approximating-sine-function-using-flux-for-solving-pinn-problem/66323)

<div class="topic-metadata">

**Author:** [@Chaitanya](https://discourse.julialang.org/u/Chaitanya)\
**Replies:** 5\
**Last updated:** [August 26, 2021, 6:01am UTC](https://discourse.julialang.org/t/approximating-sine-function-using-flux-for-solving-pinn-problem/66323 "2021-08-26T06:01:06Z")

</div>

In this code I am trying to predict u(x,t)= NN(\\theta) so that after I predict this I can use the neural network to solve PINN problems. However, I’m not able to train the model and reduce the loss function. import Flux…

---

## [The right way to implement a residual network (ResNet)](https://discourse.julialang.org/t/the-right-way-to-implement-a-residual-network-resnet/66937)

<div class="topic-metadata">

**Author:** [@weiqi](https://discourse.julialang.org/u/weiqi)\
**Replies:** 2\
**Last updated:** [August 25, 2021, 5:14pm UTC](https://discourse.julialang.org/t/the-right-way-to-implement-a-residual-network-resnet/66937 "2021-08-25T17:14:24Z")

</div>

I am not sure if the following implementation of ResNet is correct. Any suggestion? What I am not sure are: (1) for the network of nn\_x, will the ResNet block sm share parameters? I don’t expect they share the same par…

---

## [How to make parameters of function within Flux.Chain trainable?](https://discourse.julialang.org/t/how-to-make-parameters-of-function-within-flux-chain-trainable/66983)

<div class="topic-metadata">

**Author:** [@DiracFermion1](https://discourse.julialang.org/u/DiracFermion1)\
**Replies:** 2\
**Last updated:** [August 25, 2021, 3:36pm UTC](https://discourse.julialang.org/t/how-to-make-parameters-of-function-within-flux-chain-trainable/66983 "2021-08-25T15:36:13Z")

</div>

Hi, I have a NeuralODE defined as follows: dudt = Chain(x -\> Dense(10, 20, tanh)(x\[1\]) + Dense(10, 20, tanh)(x\[2\])) n\_ode = NeuralODE(dudt, (0., 1.), Tsit5(), saveat=range(0., 1., length=100), reltol=1e-7, abstol=1e-9) …

---

## [Flux.Chain vs expand everything in a function](https://discourse.julialang.org/t/flux-chain-vs-expand-everything-in-a-function/66982)

<div class="topic-metadata">

**Author:** [@DiracFermion1](https://discourse.julialang.org/u/DiracFermion1)\
**Replies:** 1\
**Last updated:** [August 25, 2021, 2:59pm UTC](https://discourse.julialang.org/t/flux-chain-vs-expand-everything-in-a-function/66982 "2021-08-25T14:59:47Z")

</div>

Hi, I was trying to understand what the difference is between Flux.Chain vs expanding all operations in a single function. For instance, given following object: Chain(Dense(d\_in, d\_hidden, tanh), Dense(d\_hidden, d\_hidd…

---

## [How to use multiple inputs in \`NeuralODE\` object?](https://discourse.julialang.org/t/how-to-use-multiple-inputs-in-neuralode-object/66955)

<div class="topic-metadata">

**Author:** [@DiracFermion1](https://discourse.julialang.org/u/DiracFermion1)\
**Replies:** 2\
**Last updated:** [August 25, 2021, 1:43pm UTC](https://discourse.julialang.org/t/how-to-use-multiple-inputs-in-neuralode-object/66955 "2021-08-25T13:43:33Z")

</div>

Hi, I am trying to use multiple inputs in NeuralODE object and wonder what is the best way to do this. Specifically, I have the following object: n\_ode = NeuralODE(dudt, tspan, Tsit5(), saveat=t, reltol=1e-7, abstol=1e…

---

## [Train neural ODE with mini-batch in different initial conditions](https://discourse.julialang.org/t/train-neural-ode-with-mini-batch-in-different-initial-conditions/66854)

<div class="topic-metadata">

**Author:** [@DiracFermion1](https://discourse.julialang.org/u/DiracFermion1)\
**Replies:** 5\
**Last updated:** [August 25, 2021, 2:53am UTC](https://discourse.julialang.org/t/train-neural-ode-with-mini-batch-in-different-initial-conditions/66854 "2021-08-25T02:53:08Z")

</div>

Hi, I am trying to train a neural ODE with mini-batch in different initial conditions. I found some tutorials regarding mini-batch but they are mostly focusing on batching different time points with the a single initial…

---

## [neuralODE not using multiple CPUs](https://discourse.julialang.org/t/neuralode-not-using-multiple-cpus/66873)

<div class="topic-metadata">

**Author:** [@DiracFermion1](https://discourse.julialang.org/u/DiracFermion1)\
**Replies:** 3\
**Last updated:** [August 24, 2021, 2:22pm UTC](https://discourse.julialang.org/t/neuralode-not-using-multiple-cpus/66873 "2021-08-24T14:22:40Z")

</div>

Hi, I am trying to run neuralODE with multiple CPUs, but I found that when I set export JULIA\_NUM\_THREADS=10, and Threads.nthreads() outputs 10, the neuralODE model is still using only 1 CPU. I am wondering if anyone ha…

---

## [Computational performance when broadcasting a convolution layer](https://discourse.julialang.org/t/computational-performance-when-broadcasting-a-convolution-layer/66587)

<div class="topic-metadata">

**Author:** [@MattSainsbury-Dale](https://discourse.julialang.org/u/MattSainsbury-Dale)\
**Replies:** 5\
**Last updated:** [August 24, 2021, 1:11am UTC](https://discourse.julialang.org/t/computational-performance-when-broadcasting-a-convolution-layer/66587 "2021-08-24T01:11:45Z")

</div>

For 2D images, the usual input format for Conv layers is Array{Float32, 4}. I’m currently experimenting with broadcasting a Conv layer over a Vector{Array{Float32, 4}}, however I have found the performance to be signific…

---

## [How to handle dataloader consisting of data with different dimensions?](https://discourse.julialang.org/t/how-to-handle-dataloader-consisting-of-data-with-different-dimensions/66847)

<div class="topic-metadata">

**Author:** [@kengyangyao](https://discourse.julialang.org/u/kengyangyao)\
**Replies:** 3\
**Last updated:** [August 23, 2021, 5:50pm UTC](https://discourse.julialang.org/t/how-to-handle-dataloader-consisting-of-data-with-different-dimensions/66847 "2021-08-23T17:50:41Z")

</div>

Hi, I was trying to use Flux.DataLoader with following code: X1 = rand(85761, 8, 24); X2 = rand(85761, 8, 1); y\_1 = rand(85761, 8, 1); emb = rand(85761, 1) train\_loader = Flux.DataLoader((X1=X1, emb=emb, X2=X2, y\_1=y\_1,…

---

## [Pytorch's Identity() implementation/alternative in Fluxml?](https://discourse.julialang.org/t/pytorchs-identity-implementation-alternative-in-fluxml/66810)

<div class="topic-metadata">

**Author:** [@extremety1989](https://discourse.julialang.org/u/extremety1989)\
**Replies:** 8\
**Last updated:** [August 23, 2021, 4:17pm UTC](https://discourse.julialang.org/t/pytorchs-identity-implementation-alternative-in-fluxml/66810 "2021-08-23T16:17:32Z")

</div>

description: A placeholder identity operator that is argument-insensitive. example pseudocode: self.layer = nn.Sequential( nn.Dropout(0.5) if dropout else nn.Identity(), )

---

## [Flux.jl: params() and gradient() ocnfusion](https://discourse.julialang.org/t/flux-jl-params-and-gradient-ocnfusion/66775)

<div class="topic-metadata">

**Author:** [@Alex-ZA](https://discourse.julialang.org/u/Alex-ZA)\
**Replies:** 4\
**Last updated:** [August 23, 2021, 4:13pm UTC](https://discourse.julialang.org/t/flux-jl-params-and-gradient-ocnfusion/66775 "2021-08-23T16:13:04Z")

</div>

Hello everyone, I apologise if this is not the area to ask Flux questions. If not, please direct to the correct place. I have been going through the Flux.jl documentation and came across this code. julia\> x = \[2, 1\];…

---

## [ERROR: LoadError: ArgumentError: cannot take the CPU address of a CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}](https://discourse.julialang.org/t/error-loaderror-argumenterror-cannot-take-the-cpu-address-of-a-cuarray-float32-2-cuda-mem-devicebuffer/66801)

<div class="topic-metadata">

**Author:** [@extremety1989](https://discourse.julialang.org/u/extremety1989)\
**Replies:** 12\
**Last updated:** [August 23, 2021, 7:35am UTC](https://discourse.julialang.org/t/error-loaderror-argumenterror-cannot-take-the-cpu-address-of-a-cuarray-float32-2-cuda-mem-devicebuffer/66801 "2021-08-23T07:35:42Z")

</div>

the problem is in " x\_batch, y\_batch = x\_batch|\>gpu, y\_batch|\>gpu " when i convert batches to gpu, removing it will work, but how can i make it work on gpu? pullback function from Zygote. here is fullcode using Flux us…

---

## [Where can i find documentation about Flux modular (approach) for building models?](https://discourse.julialang.org/t/where-can-i-find-documentation-about-flux-modular-approach-for-building-models/66780)

<div class="topic-metadata">

**Author:** [@extremety1989](https://discourse.julialang.org/u/extremety1989)\
**Replies:** 4\
**Last updated:** [August 21, 2021, 7:17pm UTC](https://discourse.julialang.org/t/where-can-i-find-documentation-about-flux-modular-approach-for-building-models/66780 "2021-08-21T19:17:02Z")

</div>

i found this example ,but no documentation how to build this way struct FFNetwork fc\_1 dropout fc\_2 FFNetwork( input\_dim::Int, hidden\_dim::Int, dropout::Float32, num\_classes::Int ) = new( …

---

## [Batch evaluation for NeuralODE?](https://discourse.julialang.org/t/batch-evaluation-for-neuralode/66753)

<div class="topic-metadata">

**Author:** [@kengyangyao](https://discourse.julialang.org/u/kengyangyao)\
**Replies:** 1\
**Last updated:** [August 21, 2021, 1:26pm UTC](https://discourse.julialang.org/t/batch-evaluation-for-neuralode/66753 "2021-08-21T13:26:06Z")

</div>

Hi, I am new to Julia and am trying to use NeuralODE as follows: d\_in, dim\_hidden = 25, 16 tspan = (0., 1.) t = range(tspan\[1\], tspan\[2\], length=100) dxdt = Chain(Dense(d\_in, dim\_hidden, tanh), Dense(dim\_hidden, dim\_hid…

---

## [Interface for loading GBIF datasets into Flux](https://discourse.julialang.org/t/interface-for-loading-gbif-datasets-into-flux/65976)

<div class="topic-metadata">

**Author:** [@banksiaboy](https://discourse.julialang.org/u/banksiaboy)\
**Replies:** 3\
**Last updated:** [August 20, 2021, 6:47am UTC](https://discourse.julialang.org/t/interface-for-loading-gbif-datasets-into-flux/65976 "2021-08-20T06:47:12Z")

</div>

Here is an example of a dataset resulting from my query against the GBIF occurrences database. As you can see it has its own DOI, is well-structured, and should be a good candidate for loading as a data source into Fl…

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=30)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=32)