# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=33

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 34

---

## [BERT models from huggingface - Transformers.jl](https://discourse.julialang.org/t/bert-models-from-huggingface-transformers-jl/64453)

<div class="topic-metadata">

**Author:** [@Alex\_Tantos](https://discourse.julialang.org/u/Alex_Tantos)\
**Replies:** 1\
**Last updated:** [July 15, 2021, 10:50pm UTC](https://discourse.julialang.org/t/bert-models-from-huggingface-transformers-jl/64453 "2021-07-15T22:50:32Z")

</div>

Hi! @chengchingwen Am planning to use an already pre-trained BERT model for Greek published in huggingface. As we know for using this type of pre-trained models, one depends on python’s hugginface and pytorch. I know t…

---

## [Unable to save and load model (or parameters) with BSON either on GPU or CPU](https://discourse.julialang.org/t/unable-to-save-and-load-model-or-parameters-with-bson-either-on-gpu-or-cpu/64582)

<div class="topic-metadata">

**Author:** [@sguns](https://discourse.julialang.org/u/sguns)\
**Replies:** 2\
**Last updated:** [July 15, 2021, 2:17am UTC](https://discourse.julialang.org/t/unable-to-save-and-load-model-or-parameters-with-bson-either-on-gpu-or-cpu/64582 "2021-07-15T02:17:50Z")

</div>

Hi, I’m training a model on GPU and trying to save intermediate states to disk as described in the Flux docs. Unfortunately it seems like the information about saving GPU models (in the blue box) is wrong - although I c…

---

## [GalacticOptim: Locally running runtests.jl fails](https://discourse.julialang.org/t/galacticoptim-locally-running-runtests-jl-fails/64591)

<div class="topic-metadata">

**Author:** [@StefanMathis](https://discourse.julialang.org/u/StefanMathis)\
**Replies:** 6\
**Last updated:** [July 14, 2021, 12:49pm UTC](https://discourse.julialang.org/t/galacticoptim-locally-running-runtests-jl-fails/64591 "2021-07-14T12:49:37Z")

</div>

Hello all, I want to add constraint support to the FiniteDiff-backend of the GalacticOptim.jl package. However, after creating a local repository of the package, I can’t get runtests.jl to work. All tests using the Ipo…

---

## [Zygote push forward unable to differentiate through generic broadcast](https://discourse.julialang.org/t/zygote-push-forward-unable-to-differentiate-through-generic-broadcast/64364)

<div class="topic-metadata">

**Author:** [@RynoLaubscher](https://discourse.julialang.org/u/RynoLaubscher)\
**Replies:** 10\
**Last updated:** [July 14, 2021, 11:44am UTC](https://discourse.julialang.org/t/zygote-push-forward-unable-to-differentiate-through-generic-broadcast/64364 "2021-07-14T11:44:43Z")

</div>

Good day all, I am struggling a bit with something simple, I hope. I would like to use Zygote.pushforward to AD neural network outputs wrt inputs. For standard fully connected nets everything works fine, but as soon as …

---

## [Zygote: Mutating arrays is not supported (small example)](https://discourse.julialang.org/t/zygote-mutating-arrays-is-not-supported-small-example/64595)

<div class="topic-metadata">

**Author:** [@gianmariomanca](https://discourse.julialang.org/u/gianmariomanca)\
**Replies:** 3\
**Last updated:** [July 13, 2021, 6:57pm UTC](https://discourse.julialang.org/t/zygote-mutating-arrays-is-not-supported-small-example/64595 "2021-07-13T18:57:33Z")

</div>

A small educational example: using Zygote using ForwardDiff Let’s define a simple gaussian (sampled at integer coordinates) x = -20:1.0:20 f2(xf) = 1000.0 .\* exp.(-1.25 .\* (x .- xf).^2) And a simple center of mass co…

---

## [Error with CircularBufferArrays in ReinforcementLearning.jl](https://discourse.julialang.org/t/error-with-circularbufferarrays-in-reinforcementlearning-jl/63769)

<div class="topic-metadata">

**Author:** [@SatvikDuddukuru](https://discourse.julialang.org/u/SatvikDuddukuru)\
**Replies:** 5\
**Last updated:** [July 11, 2021, 3:07am UTC](https://discourse.julialang.org/t/error-with-circularbufferarrays-in-reinforcementlearning-jl/63769 "2021-07-11T03:07:40Z")

</div>

I made a Connect 4 Environment with ReinforcementLearning.jl: begin mutable struct Connect4Env \<: AbstractEnv board::Array{Int,2} current\_player::Int full\_columns::Array{Int,1} is\_terminated::Bool Connect4E…

---

## [Parameters not updating in Flux](https://discourse.julialang.org/t/parameters-not-updating-in-flux/64208)

<div class="topic-metadata">

**Author:** [@a-ill](https://discourse.julialang.org/u/a-ill)\
**Replies:** 20\
**Last updated:** [July 9, 2021, 3:11pm UTC](https://discourse.julialang.org/t/parameters-not-updating-in-flux/64208 "2021-07-09T15:11:12Z")

</div>

I have the following custom layer. struct Activation{F} f::F Activation(f::Function) = new{typeof(f)}(f) end (m::Activation)(x::AbstractArray) = m.f.(x) I test whether it allows a model to be trained using the …

---

## [Flux: Resample model with different initializations](https://discourse.julialang.org/t/flux-resample-model-with-different-initializations/59097)

<div class="topic-metadata">

**Author:** [@v-i-s-h](https://discourse.julialang.org/u/v-i-s-h)\
**Replies:** 9\
**Last updated:** [July 9, 2021, 12:22pm UTC](https://discourse.julialang.org/t/flux-resample-model-with-different-initializations/59097 "2021-07-09T12:22:38Z")

</div>

Hi, I want to do an experiment in Flux where I want to keep the architecture of the model same and want to initialize the parameters with different variance normal distributions. My current set up looks like this funct…

---

## [Weight regularisation which iterates params(m) in Flux -- Mutating arrays is not supported](https://discourse.julialang.org/t/weight-regularisation-which-iterates-params-m-in-flux-mutating-arrays-is-not-supported/64314)

<div class="topic-metadata">

**Author:** [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Replies:** 7\
**Last updated:** [July 9, 2021, 4:42am UTC](https://discourse.julialang.org/t/weight-regularisation-which-iterates-params-m-in-flux-mutating-arrays-is-not-supported/64314 "2021-07-09T04:42:08Z")

</div>

I’m trying to implement weight regularisation which requires adding a weight-dependent penalty to the loss function. When I proceed naively, I run into “Mutating arrays is not supported” but I don’t understand what array…

---

## [Problem with Mutating array not supported in constructing a Wasserstein GAN in Flux](https://discourse.julialang.org/t/problem-with-mutating-array-not-supported-in-constructing-a-wasserstein-gan-in-flux/64203)

<div class="topic-metadata">

**Author:** [@gforchini](https://discourse.julialang.org/u/gforchini)\
**Replies:** 4\
**Last updated:** [July 7, 2021, 1:16pm UTC](https://discourse.julialang.org/t/problem-with-mutating-array-not-supported-in-constructing-a-wasserstein-gan-in-flux/64203 "2021-07-07T13:16:49Z")

</div>

I am trying to construct a Wasserstein GAN as in Gulrajani, et al, Improved Training of Wasserstein GANs. Part of the code is below. The discriminator loss function contains a function of the derivatives of the discrimi…

---

## [(yet another) Zygote: mutating arrays is not supported](https://discourse.julialang.org/t/yet-another-zygote-mutating-arrays-is-not-supported/62294)

<div class="topic-metadata">

**Author:** [@Dhruva2](https://discourse.julialang.org/u/Dhruva2)\
**Replies:** 10\
**Last updated:** [July 7, 2021, 1:11pm UTC](https://discourse.julialang.org/t/yet-another-zygote-mutating-arrays-is-not-supported/62294 "2021-07-07T13:11:25Z")

</div>

Hi all, thanks for the great work on Flux and Zygote! Quick summary: I’m training a simple (feedforward, relu) neural network on MNIST. If I use Flux.Losses.mse as a loss function, my code works If I use Flux.Losses.…

---

## [Flux: How to create a custom multi-layer model with some parameters shared across layers?](https://discourse.julialang.org/t/flux-how-to-create-a-custom-multi-layer-model-with-some-parameters-shared-across-layers/64190)

<div class="topic-metadata">

**Author:** [@IRM](https://discourse.julialang.org/u/IRM)\
**Replies:** 2\
**Last updated:** [July 7, 2021, 4:39am UTC](https://discourse.julialang.org/t/flux-how-to-create-a-custom-multi-layer-model-with-some-parameters-shared-across-layers/64190 "2021-07-07T04:39:40Z")

</div>

Hi all, I’m somewhat new to Julia and Flux, and trying to train a model similar to a standard dense multi-layer neural network, but with sharing of some trainable parameters between layers. To give a concrete example (…

---

## [Accessing a specific layer's weights in a Flux Chain](https://discourse.julialang.org/t/accessing-a-specific-layers-weights-in-a-flux-chain/64180)

<div class="topic-metadata">

**Author:** [@bad\_at\_math](https://discourse.julialang.org/u/bad_at_math)\
**Replies:** 1\
**Last updated:** [July 7, 2021, 1:15am UTC](https://discourse.julialang.org/t/accessing-a-specific-layers-weights-in-a-flux-chain/64180 "2021-07-07T01:15:44Z")

</div>

In PyTorch, I can define a network like this: from torch import nn class Network(nn.Module): def \_\_init\_\_(self, ...): ... self.ln\_1 = nn.Linear(64, 32) self.ln\_2 = nn.Linear(32, 16) …

---

## [Learning Epidemic Model With Neural Differential Equations](https://discourse.julialang.org/t/learning-epidemic-model-with-neural-differential-equations/63386)

<div class="topic-metadata">

**Author:** [@bkuwahara](https://discourse.julialang.org/u/bkuwahara)\
**Replies:** 5\
**Last updated:** [July 5, 2021, 9:26pm UTC](https://discourse.julialang.org/t/learning-epidemic-model-with-neural-differential-equations/63386 "2021-07-05T21:26:22Z")

</div>

I’m trying to use neural networks with DiffEqFlux to learn an SIR model with vaccination dynamics. The equations for this model are function SIRx!(du, u, p, t) β, μ, γ, a, b = Float32.(\[280, 1/50, 365/22, 100, 0.05\]…

---

## [ReinforcementLearning.jl I'm lost .. trying to implement an HVAC time to start system](https://discourse.julialang.org/t/reinforcementlearning-jl-im-lost-trying-to-implement-an-hvac-time-to-start-system/63941)

<div class="topic-metadata">

**Author:** [@mackyo123](https://discourse.julialang.org/u/mackyo123)\
**Replies:** 7\
**Last updated:** [July 4, 2021, 1:31am UTC](https://discourse.julialang.org/t/reinforcementlearning-jl-im-lost-trying-to-implement-an-hvac-time-to-start-system/63941 "2021-07-04T01:31:48Z")

</div>

I’m trying to figure out how to read in data from an HVAC system with 75 units with Zone Temperature (What the temperature is now) , Set Point(What the temperature needs to be at start time), Minutes till start time from…

---

## [Training an Autoencoder with Flux](https://discourse.julialang.org/t/training-an-autoencoder-with-flux/51619)

<div class="topic-metadata">

**Author:** [@Geoffrey](https://discourse.julialang.org/u/Geoffrey)\
**Replies:** 2\
**Last updated:** [July 2, 2021, 6:12pm UTC](https://discourse.julialang.org/t/training-an-autoencoder-with-flux/51619 "2021-07-02T18:12:28Z")

</div>

Hi, I’m trying to train a simple Autoencoder with the objective of using it for data imputation. The idea is that I have matrix of count with zeros values that are caused by a lack of sensitivity of the detector and wan…

---

## [Weights update in Flux](https://discourse.julialang.org/t/weights-update-in-flux/63657)

<div class="topic-metadata">

**Author:** [@Mariana](https://discourse.julialang.org/u/Mariana)\
**Replies:** 2\
**Last updated:** [June 28, 2021, 9:24am UTC](https://discourse.julialang.org/t/weights-update-in-flux/63657 "2021-06-28T09:24:16Z")

</div>

Hello! I am trying to use Flux to train a neural network but, in this case I don’t have the target values for the output. I will try to formulate a simpler version of my problem. Suppose I have an input input = \[0.8 0…

---

## [Size Mismatch Convolution Layer](https://discourse.julialang.org/t/size-mismatch-convolution-layer/63190)

<div class="topic-metadata">

**Author:** [@BroccoliFever](https://discourse.julialang.org/u/BroccoliFever)\
**Replies:** 3\
**Last updated:** [June 24, 2021, 1:02am UTC](https://discourse.julialang.org/t/size-mismatch-convolution-layer/63190 "2021-06-24T01:02:40Z")

</div>

I have some experience using PyTorch, so I decided to try to learn another language for Deep Learning and happened upon Julia and Flux.jl. I am trying to get started with a simple cat/dog classification problem, and I ha…

---

## [Issue with ReinforcementLearning.jl BasicDQN with custom environment](https://discourse.julialang.org/t/issue-with-reinforcementlearning-jl-basicdqn-with-custom-environment/63264)

<div class="topic-metadata">

**Author:** [@SatvikDuddukuru](https://discourse.julialang.org/u/SatvikDuddukuru)\
**Replies:** 3\
**Last updated:** [June 22, 2021, 3:45am UTC](https://discourse.julialang.org/t/issue-with-reinforcementlearning-jl-basicdqn-with-custom-environment/63264 "2021-06-22T03:45:14Z")

</div>

I have recently started working with ReinforcementLearning.jl, and I went through the given examples to familiarize myself with the syntax. I have had success with the built-in environments, and most of my own custom env…

---

## [SLIDE algorithm](https://discourse.julialang.org/t/slide-algorithm/38938)

<div class="topic-metadata">

**Author:** [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)\
**Replies:** 2\
**Last updated:** [June 20, 2021, 3:08pm UTC](https://discourse.julialang.org/t/slide-algorithm/38938 "2021-06-20T15:08:57Z")

</div>

A recent algorithm developed by Rice University + Intel claims a ~3x speedup on CPU only as compared to Nvidia V100 and 10x speedup over Tensorflow CPU. https://insidehpc.com/2020/03/slide-algorithm-for-training-deep-ne…

---

## [Batching in Flux 0.11.1](https://discourse.julialang.org/t/batching-in-flux-0-11-1/63105)

<div class="topic-metadata">

**Author:** [@sapo](https://discourse.julialang.org/u/sapo)\
**Replies:** 7\
**Last updated:** [June 18, 2021, 1:04pm UTC](https://discourse.julialang.org/t/batching-in-flux-0-11-1/63105 "2021-06-18T13:04:28Z")

</div>

Hello, for a dependency issue, I have to use Flux at version 0.11.1. I cannot understand how to use batches: julia\> using Flux julia\> d = Dense(5, 2) Dense(5, 2) julia\> d(rand(Float32, 5, 64)) |\> size (2, 64) julia\>…

---

## [SGD in Flux.jl](https://discourse.julialang.org/t/sgd-in-flux-jl/63130)

<div class="topic-metadata">

**Author:** [@wujinq](https://discourse.julialang.org/u/wujinq)\
**Replies:** 1\
**Last updated:** [June 18, 2021, 10:30am UTC](https://discourse.julialang.org/t/sgd-in-flux-jl/63130 "2021-06-18T10:30:48Z")

</div>

I’m new to Flux.jl, and I’m kind of confused by the fact that there is no stochastic gradient descent algorithm. I checked the source code, and it seemed to me that Descent is just GD instead of SGD. Of course we can ju…

---

## [BoundsError: attempt to access 2-element Vector in DDPG algorithm](https://discourse.julialang.org/t/boundserror-attempt-to-access-2-element-vector-in-ddpg-algorithm/62574)

<div class="topic-metadata">

**Author:** [@GK\_197](https://discourse.julialang.org/u/GK_197)\
**Replies:** 1\
**Last updated:** [June 14, 2021, 4:21am UTC](https://discourse.julialang.org/t/boundserror-attempt-to-access-2-element-vector-in-ddpg-algorithm/62574 "2021-06-14T04:21:05Z")

</div>

Hi, I was implementing DDPG algorithm for 1-D vector action space but, during run it shows error as: BoundsError: attempt to access 2-element Vector{Float64} at index \[\] Now, when looked into DDPG policy on Github # …

---

## [How come Flux.jl's network parameters go to NaN?](https://discourse.julialang.org/t/how-come-flux-jls-network-parameters-go-to-nan/16439)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 10\
**Last updated:** [June 9, 2021, 3:25am UTC](https://discourse.julialang.org/t/how-come-flux-jls-network-parameters-go-to-nan/16439 "2021-06-09T03:25:27Z")

</div>

I am trying to train a neural and the code is quite large and I can’t seem to find a way to reproduce an MWE at the moment. But basically Flux.train!(loss, res\_vec, opt) gives an error of Loss is NaN and when I check the…

---

## [Continuous action space array](https://discourse.julialang.org/t/continuous-action-space-array/62515)

<div class="topic-metadata">

**Author:** [@GK\_197](https://discourse.julialang.org/u/GK_197)\
**Replies:** 3\
**Last updated:** [June 8, 2021, 5:30am UTC](https://discourse.julialang.org/t/continuous-action-space-array/62515 "2021-06-08T05:30:50Z")

</div>

Hi, I have a question regarding implementation of multidimensional array in continuous action spaces. I am trying to implement DDPG algorithm for a problem which requires action space in 1-D vector. A = \[ClosedInterval…

---

## [Zygote - parametrize matrix such that gradient is only performed on selected coefficients](https://discourse.julialang.org/t/zygote-parametrize-matrix-such-that-gradient-is-only-performed-on-selected-coefficients/62393)

<div class="topic-metadata">

**Author:** [@jgbrasier](https://discourse.julialang.org/u/jgbrasier)\
**Replies:** 1\
**Last updated:** [June 5, 2021, 5:34am UTC](https://discourse.julialang.org/t/zygote-parametrize-matrix-such-that-gradient-is-only-performed-on-selected-coefficients/62393 "2021-06-05T05:34:14Z")

</div>

I was wondering if it was possible to parametrize matrices such that zygote’s gradient would only be performed on the selected coefficients of the matrix. Some dummy code would look like: theta = randn(2) W = \[theta\[1\]…

---

## [Custom NeuralODE layer trains innefficiently](https://discourse.julialang.org/t/custom-neuralode-layer-trains-innefficiently/61602)

<div class="topic-metadata">

**Author:** [@Sunny](https://discourse.julialang.org/u/Sunny)\
**Replies:** 2\
**Last updated:** [June 4, 2021, 3:13pm UTC](https://discourse.julialang.org/t/custom-neuralode-layer-trains-innefficiently/61602 "2021-06-04T15:13:21Z")

</div>

I slightly changed a NeuralODE layer and added a few new weights parameters, but it can’t fit the simple data and show the warning ┌ Warning: dt \<= dtmin. Aborting. There is either an error in your model specification …

---

## [Flux came a long way](https://discourse.julialang.org/t/flux-came-a-long-way/62288)

<div class="topic-metadata">

**Author:** [@Alexander-Barth](https://discourse.julialang.org/u/Alexander-Barth)\
**Replies:** 3\
**Last updated:** [June 3, 2021, 9:12pm UTC](https://discourse.julialang.org/t/flux-came-a-long-way/62288 "2021-06-03T21:12:59Z")

</div>

I was following the Keras presentation at the Google IO 2021 by Martin Gorner and François Chollet (creator of Keras) about convolutional varitional autoencoder (VAE). As a way to learn about VAE, I re-implemented the n…

---

## [Flux/DiffEqFlux: error when a loss function does not use all the weights of the NN](https://discourse.julialang.org/t/flux-diffeqflux-error-when-a-loss-function-does-not-use-all-the-weights-of-the-nn/61849)

<div class="topic-metadata">

**Author:** [@SvenDummer](https://discourse.julialang.org/u/SvenDummer)\
**Replies:** 8\
**Last updated:** [June 3, 2021, 5:11pm UTC](https://discourse.julialang.org/t/flux-diffeqflux-error-when-a-loss-function-does-not-use-all-the-weights-of-the-nn/61849 "2021-06-03T17:11:40Z")

</div>

Hi everyone, I have a question that is related to training a Neural Network using the sciml\_train function of DiffEqFlux.jl. Context: I want to train a Neural Stochastic Differential Equation where the loss explicitly …

---

## [Testing your DP model after training](https://discourse.julialang.org/t/testing-your-dp-model-after-training/62268)

<div class="topic-metadata">

**Author:** [@Linda\_tK](https://discourse.julialang.org/u/Linda_tK)\
**Replies:** 1\
**Last updated:** [June 3, 2021, 4:48pm UTC](https://discourse.julialang.org/t/testing-your-dp-model-after-training/62268 "2021-06-03T16:48:14Z")

</div>

Hi there, I have trained a model to approximate a system of differential equations using certain initial conditions and parameters. I would now like to see how my models performs if I use different starting conditions. I…

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=32)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=34)