# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=42

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 43

---

## [Julia stops during training Flux](https://discourse.julialang.org/t/julia-stops-during-training-flux/43052)

<div class="topic-metadata">

**Author:** [@MousaGhannam](https://discourse.julialang.org/u/MousaGhannam)\
**Replies:** 2\
**Last updated:** [July 14, 2020, 3:58pm UTC](https://discourse.julialang.org/t/julia-stops-during-training-flux/43052 "2020-07-14T15:58:26Z")

</div>

Hi all, I am currently trying to build a recurrent neural network using Flux, and during training Julia stops mid-way without throwing an error. I am using JuliaPro with the atom ide. julia\> versioninfo() Julia Version…

---

## [Computing function Jacobian from Zygote.pullback](https://discourse.julialang.org/t/computing-function-jacobian-from-zygote-pullback/42979)

<div class="topic-metadata">

**Author:** [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)\
**Replies:** 6\
**Last updated:** [July 14, 2020, 5:56am UTC](https://discourse.julialang.org/t/computing-function-jacobian-from-zygote-pullback/42979 "2020-07-14T05:56:03Z")

</div>

Zygote documentation on custom adjoints states dx/dy (in the notation from the docs) is the Jacobian when x, y are vectors. I am struggling in relating this to an implementation. For a function f : R^N -\> R^M modeled wi…

---

## [Constrained weight matrices in Flux](https://discourse.julialang.org/t/constrained-weight-matrices-in-flux/42974)

<div class="topic-metadata">

**Author:** [@grero](https://discourse.julialang.org/u/grero)\
**Replies:** 1\
**Last updated:** [July 13, 2020, 12:43pm UTC](https://discourse.julialang.org/t/constrained-weight-matrices-in-flux/42974 "2020-07-13T12:43:26Z")

</div>

Is there a way to constrain the weight matrices trained by Flux? I’m interested in using Flux to train neural networks where I’d like to e.g. specify the proportion of negative and positive weights, which would simulate …

---

## [Group Convolution Implementation - Work-around mutating arrays in gradient calcs](https://discourse.julialang.org/t/group-convolution-implementation-work-around-mutating-arrays-in-gradient-calcs/42931)

<div class="topic-metadata">

**Author:** [@CTM](https://discourse.julialang.org/u/CTM)\
**Replies:** 8\
**Last updated:** [July 12, 2020, 10:02pm UTC](https://discourse.julialang.org/t/group-convolution-implementation-work-around-mutating-arrays-in-gradient-calcs/42931 "2020-07-12T22:02:09Z")

</div>

I’m attempting to implement group convolutions which applies group transformations to incoming filters and then apply convolution. To do so, I create a set of transformation indices and then create an array with each of…

---

## [Loss function for sequence modeling w/ RNN](https://discourse.julialang.org/t/loss-function-for-sequence-modeling-w-rnn/42452)

<div class="topic-metadata">

**Author:** [@austinbean](https://discourse.julialang.org/u/austinbean)\
**Replies:** 1\
**Last updated:** [July 10, 2020, 7:42pm UTC](https://discourse.julialang.org/t/loss-function-for-sequence-modeling-w-rnn/42452 "2020-07-10T19:42:15Z")

</div>

I’d like advice on the best way to model the loss function in Flux for an RNN where I’m modeling a sequence of words in labeled sentences. (This is for a regression task, but a sentiment analysis task would look similar…

---

## [Comparation with Flux leads to odd results in Flux](https://discourse.julialang.org/t/comparation-with-flux-leads-to-odd-results-in-flux/42839)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 5\
**Last updated:** [July 10, 2020, 1:39pm UTC](https://discourse.julialang.org/t/comparation-with-flux-leads-to-odd-results-in-flux/42839 "2020-07-10T13:39:23Z")

</div>

Hello, I’m developing a toy ML library (BetaML) to learn a bit about ML algorithms (I’m pretty a newbie). I tested BetaML with a bike sharing demand forecast example and compared it with Flux: https://github.com/sylva…

---

## [Flux.jl: Initializing Parameters in Specified Range](https://discourse.julialang.org/t/flux-jl-initializing-parameters-in-specified-range/42621)

<div class="topic-metadata">

**Author:** [@astroboy](https://discourse.julialang.org/u/astroboy)\
**Replies:** 1\
**Last updated:** [July 10, 2020, 12:45pm UTC](https://discourse.julialang.org/t/flux-jl-initializing-parameters-in-specified-range/42621 "2020-07-10T12:45:07Z")

</div>

Hello! I hope you’re all doing well. I’m trying to figure out how to initialize the parameters of a simple Dense layer object uniformly in a range that I specify. I know that I can access how the layer parameters are in…

---

## [Flux.jl: Convolutional VAE throws error after upgrading to 1.4.1/10.4](https://discourse.julialang.org/t/flux-jl-convolutional-vae-throws-error-after-upgrading-to-1-4-1-10-4/39746)

<div class="topic-metadata">

**Author:** [@flo](https://discourse.julialang.org/u/flo)\
**Replies:** 19\
**Last updated:** [July 4, 2020, 7:55pm UTC](https://discourse.julialang.org/t/flux-jl-convolutional-vae-throws-error-after-upgrading-to-1-4-1-10-4/39746 "2020-07-04T19:55:26Z")

</div>

Hello everyone, I am fairly new to Julia and even newer to this forum. I hope, this is the right subforum for my problem. Currently, I am working on a project that involves a Convolutional VAE which I implemented using …

---

## [Flux.jl: Error thrown during gradient calculation in conv VAE](https://discourse.julialang.org/t/flux-jl-error-thrown-during-gradient-calculation-in-conv-vae/42397)

<div class="topic-metadata">

**Author:** [@alecokas](https://discourse.julialang.org/u/alecokas)\
**Replies:** 4\
**Last updated:** [July 4, 2020, 12:18pm UTC](https://discourse.julialang.org/t/flux-jl-error-thrown-during-gradient-calculation-in-conv-vae/42397 "2020-07-04T12:18:01Z")

</div>

Hi, I’m just getting into Flux (and Julia more generally), but I’m finding it impossible to get rid of a very particular error in my convolutional VAE model. The code runs fine until the back(1f0) line where it throws E…

---

## [Flux bilinear upsampling](https://discourse.julialang.org/t/flux-bilinear-upsampling/37717)

<div class="topic-metadata">

**Author:** [@kevinczimmerman](https://discourse.julialang.org/u/kevinczimmerman)\
**Replies:** 3\
**Last updated:** [July 3, 2020, 12:21pm UTC](https://discourse.julialang.org/t/flux-bilinear-upsampling/37717 "2020-07-03T12:21:49Z")

</div>

What is the best way to use Flux to create a bilinear upsampling layer? Something analogous to tensorflow bilinear upsample. I appreciate your insight in advance.

---

## [Efficient Way of Taking First and Second Order Derivatives of a Neural Net](https://discourse.julialang.org/t/efficient-way-of-taking-first-and-second-order-derivatives-of-a-neural-net/42163)

<div class="topic-metadata">

**Author:** [@astroboy](https://discourse.julialang.org/u/astroboy)\
**Replies:** 1\
**Last updated:** [July 2, 2020, 7:42pm UTC](https://discourse.julialang.org/t/efficient-way-of-taking-first-and-second-order-derivatives-of-a-neural-net/42163 "2020-07-02T19:42:20Z")

</div>

Hey everyone, hope you’re all doing well and staying safe! I’m just going to preface this by saying that I am a still solidly a novice when it comes to both machine learning and Julia, so please bear with me here. I am a…

---

## [Given that Zygote does not support mutation, how does Recur gets away with it?](https://discourse.julialang.org/t/given-that-zygote-does-not-support-mutation-how-does-recur-gets-away-with-it/36290)

<div class="topic-metadata">

**Author:** [@Azamat](https://discourse.julialang.org/u/Azamat)\
**Replies:** 7\
**Last updated:** [June 29, 2020, 7:59pm UTC](https://discourse.julialang.org/t/given-that-zygote-does-not-support-mutation-how-does-recur-gets-away-with-it/36290 "2020-06-29T19:59:38Z")

</div>

From Flux.jl’s source code here: mutable struct Recur{T} cell::T init state end Recur(m, h = hidden(m)) = Recur(m, h, h) function (m::Recur)(xs...) h, y = m.cell(m.state, xs...) m.state = h return y end i…

---

## [Batch/batch in Flux?](https://discourse.julialang.org/t/batch-batch-in-flux/41735)

<div class="topic-metadata">

**Author:** [@mefob](https://discourse.julialang.org/u/mefob)\
**Replies:** 2\
**Last updated:** [June 29, 2020, 2:04pm UTC](https://discourse.julialang.org/t/batch-batch-in-flux/41735 "2020-06-29T14:04:09Z")

</div>

I am building my own ODE layer and would like to make sure it works with batches as well as with individual vectors. The input and output to my layer are a NxM 2D arrays, so I will also need to reshape to couple e.g. t…

---

## [LIBSVM crashes after sucessfully running a few times](https://discourse.julialang.org/t/libsvm-crashes-after-sucessfully-running-a-few-times/42047)

<div class="topic-metadata">

**Author:** [@rio](https://discourse.julialang.org/u/rio)\
**Replies:** 1\
**Last updated:** [June 29, 2020, 1:25am UTC](https://discourse.julialang.org/t/libsvm-crashes-after-sucessfully-running-a-few-times/42047 "2020-06-29T01:25:52Z")

</div>

Hi, I am using LIBSVM.jl to train a SVM classification model. Apparently the model in my code below trains successfully. However, after running it a few times it crashes (after about 10 repetitions). It shows a segmenta…

---

## [Should I re-train the neural network if the desired final state is changed while solving optimal control problem?](https://discourse.julialang.org/t/should-i-re-train-the-neural-network-if-the-desired-final-state-is-changed-while-solving-optimal-control-problem/42166)

<div class="topic-metadata">

**Author:** [@XiaodongMa-MRI](https://discourse.julialang.org/u/XiaodongMa-MRI)\
**Replies:** 3\
**Last updated:** [June 28, 2020, 3:19am UTC](https://discourse.julialang.org/t/should-i-re-train-the-neural-network-if-the-desired-final-state-is-changed-while-solving-optimal-control-problem/42166 "2020-06-28T03:19:50Z")

</div>

Hi everyone, I have a very basic question about solving optimal control problems with universal differential equations. Say I want to find a control c(t) to achieve a certain final state u(T), and I can use neural netwo…

---

## [Adding more inputs after convolution in Flux?](https://discourse.julialang.org/t/adding-more-inputs-after-convolution-in-flux/42093)

<div class="topic-metadata">

**Author:** [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)\
**Replies:** 3\
**Last updated:** [June 26, 2020, 3:35pm UTC](https://discourse.julialang.org/t/adding-more-inputs-after-convolution-in-flux/42093 "2020-06-26T15:35:42Z")

</div>

So I have a large spatial vector as input from which I extract some features using convolution, but I also have some others non-spatial inputs that I’d like to concatenate to my features, is there a way to do that with F…

---

## [Augmented neural ODE on GPU](https://discourse.julialang.org/t/augmented-neural-ode-on-gpu/42096)

<div class="topic-metadata">

**Author:** [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)\
**Replies:** 2\
**Last updated:** [June 26, 2020, 2:07pm UTC](https://discourse.julialang.org/t/augmented-neural-ode-on-gpu/42096 "2020-06-26T14:07:14Z")

</div>

Trying to run the augumented neural ODE example from the docs and it works great on CPU. However, moving all the parts with the gpu function like when using a regular Flux model does not seem to do the trick. I am getti…

---

## [Deterministic Clustering in Julia?](https://discourse.julialang.org/t/deterministic-clustering-in-julia/42068)

<div class="topic-metadata">

**Author:** [@mthelm85](https://discourse.julialang.org/u/mthelm85)\
**Replies:** 4\
**Last updated:** [June 26, 2020, 6:54am UTC](https://discourse.julialang.org/t/deterministic-clustering-in-julia/42068 "2020-06-26T06:54:41Z")

</div>

Are there any deterministic clustering methods implemented in Julia? For example: using Clustering using DataFrames df = DataFrame(subject=1:10, height=rand(10), weight=rand(10)) julia\> groups = kmeans(Matrix(df\[:, 2:…

---

## [New Alpha zero replica](https://discourse.julialang.org/t/new-alpha-zero-replica/41432)

<div class="topic-metadata">

**Author:** [@Fabrice\_Rosay](https://discourse.julialang.org/u/Fabrice_Rosay)\
**Replies:** 6\
**Last updated:** [June 19, 2020, 1:01pm UTC](https://discourse.julialang.org/t/new-alpha-zero-replica/41432 "2020-06-19T13:01:27Z")

</div>

Hello, I posted a new alphazero replica on github: https://github.com/fabricerosay/AGZ0 It’s (very(very)) messy but fast and can easily be tweaked provided you already know alphazero. It uses Flux as only backend. Th…

---

## [Custom XGBoost Loss function w/ Zygote. Julia Computing blog post](https://discourse.julialang.org/t/custom-xgboost-loss-function-w-zygote-julia-computing-blog-post/35811)

<div class="topic-metadata">

**Author:** [@Albert\_Zevelev](https://discourse.julialang.org/u/Albert_Zevelev)\
**Replies:** 36\
**Last updated:** [April 29, 2020, 8:07am UTC](https://discourse.julialang.org/t/custom-xgboost-loss-function-w-zygote-julia-computing-blog-post/35811 "2020-04-29T08:07:25Z")

</div>

There is a really cool recent blog post @ Julia Computing about how easily custom loss functions can be implemented in XGBoost w/ Zygote. They use the Titanic survival data from Kaggle & compare scores on the Leaderboard…

---

## [Batch size and early stopping in Flux](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517)

<div class="topic-metadata">

**Author:** [@Mariana](https://discourse.julialang.org/u/Mariana)\
**Replies:** 5\
**Last updated:** [June 17, 2020, 10:56am UTC](https://discourse.julialang.org/t/batch-size-and-early-stopping-in-flux/41517 "2020-06-17T10:56:58Z")

</div>

I´m starting to work with Flux and I have a neural network with 63 entries, 2 hidden layers and 1 exit. Each entry and exit is a vector of 113344 elements, from which 78801 elements belong to the training set and the rem…

---

## [Common RL Environment Interface](https://discourse.julialang.org/t/common-rl-environment-interface/41460)

<div class="topic-metadata">

**Author:** [@zsunberg](https://discourse.julialang.org/u/zsunberg)\
**Replies:** 1\
**Last updated:** [June 15, 2020, 9:12pm UTC](https://discourse.julialang.org/t/common-rl-environment-interface/41460 "2020-06-15T21:12:55Z")

</div>

Hi everyone, after this conversation https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/issues/68 we decided to try to create a common reinforcement learning environment interface with very minimal re…

---

## [My own Feedforward Neural Network library :-)](https://discourse.julialang.org/t/my-own-feedforward-neural-network-library/37198)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 28\
**Last updated:** [June 15, 2020, 9:03pm UTC](https://discourse.julialang.org/t/my-own-feedforward-neural-network-library/37198 "2020-06-15T21:03:36Z")

</div>

I am learning ML, so I created this toy FNN library… Surely is not optimised like in real ML libraries, but it is already pretty flexible and its simplicity may come on hand in implementing custom solutions… using Ra…

---

## [Backpropagation Through Time](https://discourse.julialang.org/t/backpropagation-through-time/40822)

<div class="topic-metadata">

**Author:** [@Freddy\_Chua](https://discourse.julialang.org/u/Freddy_Chua)\
**Replies:** 1\
**Last updated:** [June 14, 2020, 10:50am UTC](https://discourse.julialang.org/t/backpropagation-through-time/40822 "2020-06-14T10:50:58Z")

</div>

Hi, I noted the used of “record” for BPTT for LSTM/Recurrence based Models. Is this in used? I don’t see it being documented anywhere, i also could not find it in the code. How do we do BPTT?

---

## [Speeding up findmin for a matrix for hierarchical clustering](https://discourse.julialang.org/t/speeding-up-findmin-for-a-matrix-for-hierarchical-clustering/41049)

<div class="topic-metadata">

**Author:** [@gobs](https://discourse.julialang.org/u/gobs)\
**Replies:** 4\
**Last updated:** [June 11, 2020, 8:02am UTC](https://discourse.julialang.org/t/speeding-up-findmin-for-a-matrix-for-hierarchical-clustering/41049 "2020-06-11T08:02:25Z")

</div>

I’ve written a hierarchical clustering algorithm and I would like to speed it up if possible. It seems that my limiting step is findmax(D) where D is my dissimilarity matrix. This is a LowerTriangular matrix (since the o…

---

## [Value of gradients with DiffEqFlux and solve](https://discourse.julialang.org/t/value-of-gradients-with-diffeqflux-and-solve/40982)

<div class="topic-metadata">

**Author:** [@MikeKlocCZ](https://discourse.julialang.org/u/MikeKlocCZ)\
**Replies:** 3\
**Last updated:** [June 9, 2020, 7:27am UTC](https://discourse.julialang.org/t/value-of-gradients-with-diffeqflux-and-solve/40982 "2020-06-09T07:27:46Z")

</div>

As recently announced, solve command is now compatible with Flux and should be used. I am confused about the value of the gradients which I obtain in these two very simple examples. using Flux, DiffEqFlux, DifferentialE…

---

## [Why does binarycrossentropy needs an index in a denoising autoencoder?](https://discourse.julialang.org/t/why-does-binarycrossentropy-needs-an-index-in-a-denoising-autoencoder/41013)

<div class="topic-metadata">

**Author:** [@swuyts](https://discourse.julialang.org/u/swuyts)\
**Replies:** 2\
**Last updated:** [June 9, 2020, 7:12am UTC](https://discourse.julialang.org/t/why-does-binarycrossentropy-needs-an-index-in-a-denoising-autoencoder/41013 "2020-06-09T07:12:44Z")

</div>

Hi there, I am still somewhat a rookie in Julia and Flux and I have problem understanding what is going on when I switch between crossentropy and binarycrossentropy loss functions. I coded the following simple denoisin…

---

## [RMSprop and Early stopping in Flux.jl](https://discourse.julialang.org/t/rmsprop-and-early-stopping-in-flux-jl/21065)

<div class="topic-metadata">

**Author:** [@charlesll](https://discourse.julialang.org/u/charlesll)\
**Replies:** 8\
**Last updated:** [June 6, 2020, 2:57pm UTC](https://discourse.julialang.org/t/rmsprop-and-early-stopping-in-flux-jl/21065 "2020-06-06T14:57:19Z")

</div>

Hi, I’m playing with Flux.jl and results are really nice. At the moment, I would like to use the RMSprop solver with Early Stopping. Anyone have experience with implementing those in Flux? Thanks!

---

## [Writing complex Flux Models](https://discourse.julialang.org/t/writing-complex-flux-models/40439)

<div class="topic-metadata">

**Author:** [@irhum](https://discourse.julialang.org/u/irhum)\
**Replies:** 4\
**Last updated:** [June 5, 2020, 7:14pm UTC](https://discourse.julialang.org/t/writing-complex-flux-models/40439 "2020-06-05T19:14:05Z")

</div>

Flux’s typical go to when writing models is Chains, and for most feedforward models (with the occasional skip connection), with one input, it works quite beautifully. Problem is, when defining models as functions, we lo…

---

## [Taking gradients when propagating through an ODE solver](https://discourse.julialang.org/t/taking-gradients-when-propagating-through-an-ode-solver/40652)

<div class="topic-metadata">

**Author:** [@MikeKlocCZ](https://discourse.julialang.org/u/MikeKlocCZ)\
**Replies:** 6\
**Last updated:** [June 3, 2020, 6:45pm UTC](https://discourse.julialang.org/t/taking-gradients-when-propagating-through-an-ode-solver/40652 "2020-06-03T18:45:35Z")

</div>

I need to differentiate parameters of my model (neural network) through an ODE solver. Apparently, DiffEqFlux allows me to do that but it seems that the gradients are computed only if the network parameters enter direct…

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=41)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=43)