# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=28

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 29

---

## [Zygote/Flux not computing gradient with view/slice of tensor properly for MLP](https://discourse.julialang.org/t/zygote-flux-not-computing-gradient-with-view-slice-of-tensor-properly-for-mlp/67835)

<div class="topic-metadata">

**Author:** [@outlace](https://discourse.julialang.org/u/outlace)\
**Replies:** 19\
**Last updated:** [December 20, 2021, 4:57pm UTC](https://discourse.julialang.org/t/zygote-flux-not-computing-gradient-with-view-slice-of-tensor-properly-for-mlp/67835 "2021-12-20T16:57:02Z")

</div>

I am working on an implementation of an ML algorithm called SLIDE (sub-linear deep learning engine) from an academic paper. I’ve included a link to a gist of the code: https://gist.github.com/outlace/c81a15ab77c1c9cebb56…

---

## [Dropout for the recurrent signal on LSTM units](https://discourse.julialang.org/t/dropout-for-the-recurrent-signal-on-lstm-units/73335)

<div class="topic-metadata">

**Author:** [@Maxim\_Lubov](https://discourse.julialang.org/u/Maxim_Lubov)\
**Replies:** 1\
**Last updated:** [December 20, 2021, 4:26pm UTC](https://discourse.julialang.org/t/dropout-for-the-recurrent-signal-on-lstm-units/73335 "2021-12-20T16:26:40Z")

</div>

Is there a possibility in Flux to apply dropout procedure to the recurrent input signal on the LSTM units, like recurrent\_dropout in Keras?

---

## [Computing discrete Wasserstein (EMD) distance in Julia?](https://discourse.julialang.org/t/computing-discrete-wasserstein-emd-distance-in-julia/9600)

<div class="topic-metadata">

**Author:** [@raff](https://discourse.julialang.org/u/raff)\
**Replies:** 16\
**Last updated:** [December 20, 2021, 3:49pm UTC](https://discourse.julialang.org/t/computing-discrete-wasserstein-emd-distance-in-julia/9600 "2021-12-20T15:49:47Z")

</div>

Is there any library in Julia for computing discrete Wasserstein (EMD) given two discrete distributions? Python seems to have a tool for that http://pot.readthedocs.io/en/stable/index.html, but if a pure Julia solution …

---

## [Training a MLJ model on a large dataset](https://discourse.julialang.org/t/training-a-mlj-model-on-a-large-dataset/73321)

<div class="topic-metadata">

**Author:** [@jlwoolf](https://discourse.julialang.org/u/jlwoolf)\
**Replies:** 2\
**Last updated:** [December 19, 2021, 7:44pm UTC](https://discourse.julialang.org/t/training-a-mlj-model-on-a-large-dataset/73321 "2021-12-19T19:44:26Z")

</div>

Hello, I am somewhat new to Julia and the MLJ library. I am currently using a preexisting model, and would like to train it on a large dataset. However, I am running out of memory when calling fit with all the data. Eac…

---

## [Automatic/Black-box variational inference replication vs Autograd](https://discourse.julialang.org/t/automatic-black-box-variational-inference-replication-vs-autograd/73020)

<div class="topic-metadata">

**Author:** [@lf28](https://discourse.julialang.org/u/lf28)\
**Replies:** 9\
**Last updated:** [December 16, 2021, 10:25pm UTC](https://discourse.julialang.org/t/automatic-black-box-variational-inference-replication-vs-autograd/73020 "2021-12-16T22:25:26Z")

</div>

Hello all, I have been trying to replicate a simple example (Fig 2) presented in the paper: Black-box stochastic variational inference in five lines of Python (https://www.cs.toronto.edu/~duvenaud/papers/blackbox.pdf). …

---

## [Neural SDE vs Neural ODE](https://discourse.julialang.org/t/neural-sde-vs-neural-ode/73189)

<div class="topic-metadata">

**Author:** [@ignace](https://discourse.julialang.org/u/ignace)\
**Replies:** 2\
**Last updated:** [December 16, 2021, 11:24am UTC](https://discourse.julialang.org/t/neural-sde-vs-neural-ode/73189 "2021-12-16T11:24:10Z")

</div>

May I please ask a question, please redirect me when this is the wrong place to ask :slight\_smile: What is the main (dis)advantage of neural SDEs over neural ODEs? Can they somehow tackle ‘more difficult’ problems or s…

---

## [Collect model outputs into multi-dimensional array](https://discourse.julialang.org/t/collect-model-outputs-into-multi-dimensional-array/72977)

<div class="topic-metadata">

**Author:** [@Tengrath](https://discourse.julialang.org/u/Tengrath)\
**Replies:** 5\
**Last updated:** [December 16, 2021, 10:25am UTC](https://discourse.julialang.org/t/collect-model-outputs-into-multi-dimensional-array/72977 "2021-12-16T10:25:42Z")

</div>

Consider the following model which creates an output after every layer ’ function track(model, input) track::Vector{Vector{Float64}} = p = input for layer in model p = layer(p) track = vcat(track, \[p,\]) end …

---

## [Habana Gaudi AI Processors - Julia support?](https://discourse.julialang.org/t/habana-gaudi-ai-processors-julia-support/73093)

<div class="topic-metadata">

**Author:** [@anon28281882](https://discourse.julialang.org/u/anon28281882)\
**Replies:** 4\
**Last updated:** [December 15, 2021, 12:40am UTC](https://discourse.julialang.org/t/habana-gaudi-ai-processors-julia-support/73093 "2021-12-15T00:40:24Z")

</div>

Hi, Do you think that there is any chance that Habana is going to support Julia on their processors in the future (https://developer.habana.ai/) or is it rather very unlikely?

---

## [Need help proofreading my BLSTM implementation](https://discourse.julialang.org/t/need-help-proofreading-my-blstm-implementation/73015)

<div class="topic-metadata">

**Author:** [@Qfl3x](https://discourse.julialang.org/u/Qfl3x)\
**Replies:** 4\
**Last updated:** [December 14, 2021, 5:55pm UTC](https://discourse.julialang.org/t/need-help-proofreading-my-blstm-implementation/73015 "2021-12-14T17:55:40Z")

</div>

Hi, I’m implementing a BLSTM for a personal (soon finished) project of mine and I’m currently very confused as to how BLSTM’s are implemented and found various kinds of implementations in papers online, mine should conca…

---

## [Graph Neural Networks in Flux.](https://discourse.julialang.org/t/graph-neural-networks-in-flux/21932)

<div class="topic-metadata">

**Author:** [@FernandoHildebrand](https://discourse.julialang.org/u/FernandoHildebrand)\
**Replies:** 5\
**Last updated:** [December 14, 2021, 7:44am UTC](https://discourse.julialang.org/t/graph-neural-networks-in-flux/21932 "2021-12-14T07:44:12Z")

</div>

Hello, Im starting my master course (Brazil- UFPR) in Bioinformatics. My background is mostly in IT and I´ve been studing Neural Nets(CNN, RNN…) to my upmost abilities. It seems that Graphs Neural Networks and/or Graph…

---

## [OCR with Julia](https://discourse.julialang.org/t/ocr-with-julia/4945)

<div class="topic-metadata">

**Author:** [@John\_Hearns](https://discourse.julialang.org/u/John_Hearns)\
**Replies:** 7\
**Last updated:** [December 12, 2021, 10:08pm UTC](https://discourse.julialang.org/t/ocr-with-julia/4945 "2021-12-12T22:08:49Z")

</div>

I am looking for suggestion on doing Optical Character Recognition. This does not necessarily have to be native Julia (see below). Also I am not looking for a full-on Machine Learnign framework which will recognise image…

---

## [How to move optimiser from gpu to cpu?](https://discourse.julialang.org/t/how-to-move-optimiser-from-gpu-to-cpu/69622)

<div class="topic-metadata">

**Author:** [@weiqi](https://discourse.julialang.org/u/weiqi)\
**Replies:** 10\
**Last updated:** [December 12, 2021, 6:29pm UTC](https://discourse.julialang.org/t/how-to-move-optimiser-from-gpu-to-cpu/69622 "2021-12-12T18:29:18Z")

</div>

Currently, cpu(optimiser) won’t move it. For instance, the state still consists of variables in GPU.

---

## [Deepcopy Flux Model](https://discourse.julialang.org/t/deepcopy-flux-model/72930)

<div class="topic-metadata">

**Author:** [@tknopp](https://discourse.julialang.org/u/tknopp)\
**Replies:** 8\
**Last updated:** [December 12, 2021, 6:28pm UTC](https://discourse.julialang.org/t/deepcopy-flux-model/72930 "2021-12-12T18:28:02Z")

</div>

Hi, I am struggling with warmstarting Flux models. My idea is to deepcopy both the model and the optimizer. However, if I do that the training error goes significantly up when retraining. If I use BSON: @save/@load it w…

---

## [How to pass each column of a Matrix(which is the output of an NN) onto a layer seperately](https://discourse.julialang.org/t/how-to-pass-each-column-of-a-matrix-which-is-the-output-of-an-nn-onto-a-layer-seperately/72574)

<div class="topic-metadata">

**Author:** [@Qfl3x](https://discourse.julialang.org/u/Qfl3x)\
**Replies:** 2\
**Last updated:** [December 5, 2021, 4:46pm UTC](https://discourse.julialang.org/t/how-to-pass-each-column-of-a-matrix-which-is-the-output-of-an-nn-onto-a-layer-seperately/72574 "2021-12-05T16:46:14Z")

</div>

Hello, I have a net as follows: model = Chain(Embed(200,37),Dense(200,150),Dense(150,100)) The output of Embed (From Transformers.jl) is an embedding matrix of size (200,100) for a size 100 input vector. Meaning at t…

---

## [Multilayer perceptron with time dependent multidimensional data using Flux](https://discourse.julialang.org/t/multilayer-perceptron-with-time-dependent-multidimensional-data-using-flux/72011)

<div class="topic-metadata">

**Author:** [@FrootLoops](https://discourse.julialang.org/u/FrootLoops)\
**Replies:** 7\
**Last updated:** [December 2, 2021, 5:20pm UTC](https://discourse.julialang.org/t/multilayer-perceptron-with-time-dependent-multidimensional-data-using-flux/72011 "2021-12-02T17:20:58Z")

</div>

Hi, I have some problems coding a ( multilayer perceptron using Flux. My data is given in the form D\_t = (x\_i, yi)\_{i=1}^n where x\_i is a scalar and y\_i a $3-$dim vector and t=1,\\ldots,T the timesteps. One shold notice …

---

## [Updating only a subset of parameters in a neural network using Flux](https://discourse.julialang.org/t/updating-only-a-subset-of-parameters-in-a-neural-network-using-flux/72312)

<div class="topic-metadata">

**Author:** [@Handam](https://discourse.julialang.org/u/Handam)\
**Replies:** 2\
**Last updated:** [December 1, 2021, 5:25pm UTC](https://discourse.julialang.org/t/updating-only-a-subset-of-parameters-in-a-neural-network-using-flux/72312 "2021-12-01T17:25:53Z")

</div>

I am currently playing around with actor-critic methods. In my current problem, the actor has four outputs: pol = Chain(Dense(6, 32, relu), Dense(32, 4)) However, depending on the state only a subset of the actions are…

---

## [Significant compile time latency in Flux with a GAN](https://discourse.julialang.org/t/significant-compile-time-latency-in-flux-with-a-gan/68518)

<div class="topic-metadata">

**Author:** [@Alexander-Barth](https://discourse.julialang.org/u/Alexander-Barth)\
**Replies:** 28\
**Last updated:** [November 29, 2021, 12:00am UTC](https://discourse.julialang.org/t/significant-compile-time-latency-in-flux-with-a-gan/68518 "2021-11-29T00:00:32Z")

</div>

I am trying to implement a SRGAN (super-resolution generative adversarial networks) in Flux (I am happy to share once it works… :-)) The generator network is composed of 5 residual blocks (function resblock below). Unf…

---

## [Unstable learning of neural net with few nodes using Flux](https://discourse.julialang.org/t/unstable-learning-of-neural-net-with-few-nodes-using-flux/72115)

<div class="topic-metadata">

**Author:** [@Handam](https://discourse.julialang.org/u/Handam)\
**Replies:** 9\
**Last updated:** [November 28, 2021, 10:18am UTC](https://discourse.julialang.org/t/unstable-learning-of-neural-net-with-few-nodes-using-flux/72115 "2021-11-28T10:18:38Z")

</div>

I have a neural network with only one binary input node, one hidden layer and one output node. When I try to train the network, such that it outputs the value x\_1 when the input is 0 and x\_2 when the input is 1, it will …

---

## [Custom rrule for Feedback Alignment](https://discourse.julialang.org/t/custom-rrule-for-feedback-alignment/72159)

<div class="topic-metadata">

**Author:** [@Rasmus\_Hoier](https://discourse.julialang.org/u/Rasmus_Hoier)\
**Replies:** 2\
**Last updated:** [November 27, 2021, 8:10pm UTC](https://discourse.julialang.org/t/custom-rrule-for-feedback-alignment/72159 "2021-11-27T20:10:36Z")

</div>

Hi, I am trying to use a custom rrule to implement Feedback Alignment learning (which is just like BP except that the error signals are propagated backwards using random feedback weights B, which is supposed to be more …

---

## [SVM Errors With Grid Search](https://discourse.julialang.org/t/svm-errors-with-grid-search/71954)

<div class="topic-metadata">

**Author:** [@Rahul](https://discourse.julialang.org/u/Rahul)\
**Replies:** 24\
**Last updated:** [November 26, 2021, 8:31am UTC](https://discourse.julialang.org/t/svm-errors-with-grid-search/71954 "2021-11-26T08:31:36Z")

</div>

I am trying to perform Grid Search CV with SVM as follows: SVMClassifier = @load SVMClassifier pkg=ScikitLearn svm\_model = SVMClassifier() r\_svm = range(svm\_model, :C, values=\[0.25, 0.5, 1\]) self\_tuning\_svm = TunedMode…

---

## [Automatic Creation of a Grid of Tuning Parameters](https://discourse.julialang.org/t/automatic-creation-of-a-grid-of-tuning-parameters/71902)

<div class="topic-metadata">

**Author:** [@Rahul](https://discourse.julialang.org/u/Rahul)\
**Replies:** 4\
**Last updated:** [November 23, 2021, 9:10pm UTC](https://discourse.julialang.org/t/automatic-creation-of-a-grid-of-tuning-parameters/71902 "2021-11-23T21:10:17Z")

</div>

Currently, in R’s caret package if we do something like this: fitControl \<- trainControl(method = "repeatedcv", number=10,repeats=5) model\_rf = train(responder~ ., data=trainData, method='rf',trControl=fitControl) a gr…

---

## [Flux/Zygote: Gradient with respect to inputs and implicit parameters (in 2021)](https://discourse.julialang.org/t/flux-zygote-gradient-with-respect-to-inputs-and-implicit-parameters-in-2021/71931)

<div class="topic-metadata">

**Author:** [@manuelbb-upb](https://discourse.julialang.org/u/manuelbb-upb)\
**Replies:** 1\
**Last updated:** [November 23, 2021, 12:28pm UTC](https://discourse.julialang.org/t/flux-zygote-gradient-with-respect-to-inputs-and-implicit-parameters-in-2021/71931 "2021-11-23T12:28:14Z")

</div>

Hello all, I have struggled a bit to get the gradient(s) of a loss with a Zygote model both with respect to the model parameters and to the model input. Is it even sensible to do this in one pass? Consider this simple…

---

## [Minimisation of a Bolza form loss function in DiffEqFlux framework](https://discourse.julialang.org/t/minimisation-of-a-bolza-form-loss-function-in-diffeqflux-framework/71920)

<div class="topic-metadata">

**Author:** [@nhcho91](https://discourse.julialang.org/u/nhcho91)\
**Replies:** 3\
**Last updated:** [November 23, 2021, 8:07am UTC](https://discourse.julialang.org/t/minimisation-of-a-bolza-form-loss-function-in-diffeqflux-framework/71920 "2021-11-23T08:07:19Z")

</div>

Brief Summary Problem at hand: minimise J \\left( p \\right) = \\phi \\left( x \\left(t\_{f} \\right) \\right) + \\int\_{t\_{0}}^{t\_{f}} L \\left( \\tau, x\\left(\\tau\\right), p \\right) d\\tau + R\\left( p \\right) subject to \\dot{x}…

---

## [Problems with Flux NN regression](https://discourse.julialang.org/t/problems-with-flux-nn-regression/71714)

<div class="topic-metadata">

**Author:** [@Bitt](https://discourse.julialang.org/u/Bitt)\
**Replies:** 1\
**Last updated:** [November 19, 2021, 7:39am UTC](https://discourse.julialang.org/t/problems-with-flux-nn-regression/71714 "2021-11-19T07:39:03Z")

</div>

I am trying to make a regression with NN using Flux. I have separated the dataset in the train/test parts and implemented a simple neural network. x\_train, x\_test, y\_train, y\_test = train\_test\_split(convert(Array, x), c…

---

## [How to add norm of gradient to a loss function?](https://discourse.julialang.org/t/how-to-add-norm-of-gradient-to-a-loss-function/69873)

<div class="topic-metadata">

**Author:** [@AlexLewandowski](https://discourse.julialang.org/u/AlexLewandowski)\
**Replies:** 16\
**Last updated:** [November 17, 2021, 7:56am UTC](https://discourse.julialang.org/t/how-to-add-norm-of-gradient-to-a-loss-function/69873 "2021-11-17T07:56:08Z")

</div>

I am trying to add a gradient norm penalty to a loss function similar to WGAN-GP (\[1704.00028\] Improved Training of Wasserstein GANs, Alg 1, Lines 7-9). However, I am running into problems with mutation. Is there another…

---

## [Flux: no method matching ndims(::ChainRulesCore.NoTangent)](https://discourse.julialang.org/t/flux-no-method-matching-ndims-chainrulescore-notangent/71479)

<div class="topic-metadata">

**Author:** [@Shuhua](https://discourse.julialang.org/u/Shuhua)\
**Replies:** 3\
**Last updated:** [November 17, 2021, 2:06am UTC](https://discourse.julialang.org/t/flux-no-method-matching-ndims-chainrulescore-notangent/71479 "2021-11-17T02:06:49Z")

</div>

Consider the MWE below. using Flux model = Chain( RNN(1, 128), Dense(128, 1) ); # one feature, sequence length = 3, batch size = 6 x = \[rand(Float32, 1, 6) for \_ = 1:3\] y = rand(Float32, 1, 6) function loss(…

---

## [Implicit gradients with mutable struct returns an error](https://discourse.julialang.org/t/implicit-gradients-with-mutable-struct-returns-an-error/71455)

<div class="topic-metadata">

**Author:** [@a709e9a0c](https://discourse.julialang.org/u/a709e9a0c)\
**Replies:** 1\
**Last updated:** [November 17, 2021, 1:15am UTC](https://discourse.julialang.org/t/implicit-gradients-with-mutable-struct-returns-an-error/71455 "2021-11-17T01:15:28Z")

</div>

Hi, When I try to compute the gradients of a mutable struct via implicit gradients, this code fails, but the same exact code works when the struct is not mutable: using Zygote mutable struct Linear\_ W::Array{Float…

---

## [Multigraph neural networks in Julia](https://discourse.julialang.org/t/multigraph-neural-networks-in-julia/70595)

<div class="topic-metadata">

**Author:** [@bad\_at\_math](https://discourse.julialang.org/u/bad_at_math)\
**Replies:** 4\
**Last updated:** [November 15, 2021, 7:21pm UTC](https://discourse.julialang.org/t/multigraph-neural-networks-in-julia/70595 "2021-11-15T19:21:20Z")

</div>

GeometricFlux.jl (https://fluxml.ai/GeometricFlux.jl/dev/manual/conv/#Crystal-Graph-Convolutional-Network-1) and GraphNeuralNetworks.jl (Convolutional Layers · GraphNeuralNetworks.jl) both implement a layer based on the …

---

## [Training sentence transformers in Julia?](https://discourse.julialang.org/t/training-sentence-transformers-in-julia/71460)

<div class="topic-metadata">

**Author:** [@Alex\_Tantos](https://discourse.julialang.org/u/Alex_Tantos)\
**Replies:** 0\
**Last updated:** [November 14, 2021, 3:57pm UTC](https://discourse.julialang.org/t/training-sentence-transformers-in-julia/71460 "2021-11-14T15:57:45Z")

</div>

Hi! Has anyone tried to train a sentence transformer by fine tuning BERT in julia following the method of the 2019 paper on sentence transformers (and the model is called SBERT)? I know this is too general a question, b…

---

## [Training layers of a Flux model separately](https://discourse.julialang.org/t/training-layers-of-a-flux-model-separately/71342)

<div class="topic-metadata">

**Author:** [@RedDocMD](https://discourse.julialang.org/u/RedDocMD)\
**Replies:** 1\
**Last updated:** [November 13, 2021, 2:47pm UTC](https://discourse.julialang.org/t/training-layers-of-a-flux-model-separately/71342 "2021-11-13T14:47:56Z")

</div>

Suppose I have a model in Flux, which is a bunch of layers that are stacked using Flux.chain. The training algorithm that I have trains layers individually. So would the following (simplified) code work? layers = \[Vect…

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=27)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=29)