# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=37

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 38

---

## [Initializing Flux weights the same as PyTorch?](https://discourse.julialang.org/t/initializing-flux-weights-the-same-as-pytorch/54660)

<div class="topic-metadata">

**Author:** [@DevJac](https://discourse.julialang.org/u/DevJac)\
**Replies:** 4\
**Last updated:** [February 9, 2021, 10:04pm UTC](https://discourse.julialang.org/t/initializing-flux-weights-the-same-as-pytorch/54660 "2021-02-09T22:04:38Z")

</div>

I am trying to replicate some results from a PyTorch model. I believe layers in Flux have their weights initialized quite differently than in PyTorch. For example, Flux layers have 0 bias to begin with, but PyTorch layer…

---

## [Turing.jl, differentiation & categorical outputs -- \`isprobvec\` conundrum](https://discourse.julialang.org/t/turing-jl-differentiation-categorical-outputs-isprobvec-conundrum/54083)

<div class="topic-metadata">

**Author:** [@delehef](https://discourse.julialang.org/u/delehef)\
**Replies:** 5\
**Last updated:** [February 8, 2021, 6:12pm UTC](https://discourse.julialang.org/t/turing-jl-differentiation-categorical-outputs-isprobvec-conundrum/54083 "2021-02-08T18:12:51Z")

</div>

Hi all, I met a strange bug, and, to be honest, I’m relatively new to the language, so it could be a mistake from my part. So, I’m having a real blast playing with bayesian NNs, and I’m slowly building up from the basi…

---

## [Flux and automatic differentiation](https://discourse.julialang.org/t/flux-and-automatic-differentiation/54786)

<div class="topic-metadata">

**Author:** [@aquarelleX332](https://discourse.julialang.org/u/aquarelleX332)\
**Replies:** 3\
**Last updated:** [February 8, 2021, 5:16pm UTC](https://discourse.julialang.org/t/flux-and-automatic-differentiation/54786 "2021-02-08T17:16:50Z")

</div>

Hello all, This may be a naive question: how does Flux practice automatic differentiation? (i.e. according to which principle; I am aware we are dealing with Zygote) Where can I possibly find “educational” documentatio…

---

## [Using pretrained pytorch models in Flux](https://discourse.julialang.org/t/using-pretrained-pytorch-models-in-flux/39082)

<div class="topic-metadata">

**Author:** [@tbreloff](https://discourse.julialang.org/u/tbreloff)\
**Replies:** 6\
**Last updated:** [February 8, 2021, 5:11pm UTC](https://discourse.julialang.org/t/using-pretrained-pytorch-models-in-flux/39082 "2021-02-08T17:11:59Z")

</div>

Hi all. I’m excited to be able to dip my toes back into the Julia pond. I’ve missed the community these last few years. My question is about how to best leverage existing pytorch resources, such as the extensive pretrai…

---

## [JLBoost key error](https://discourse.julialang.org/t/jlboost-key-error/54702)

<div class="topic-metadata">

**Author:** [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Replies:** 0\
**Last updated:** [February 5, 2021, 7:25pm UTC](https://discourse.julialang.org/t/jlboost-key-error/54702 "2021-02-05T19:25:33Z")

</div>

Hi All, I am trying to blindly use JLBoost on a big dataset with 2381 features and 600 000 samples. I have automatically converted the feature matrix to the DataFrame using df = DataFrame(x, :auto), add labels as df.y =…

---

## [Sum of Flux.params](https://discourse.julialang.org/t/sum-of-flux-params/54608)

<div class="topic-metadata">

**Author:** [@anon74562486](https://discourse.julialang.org/u/anon74562486)\
**Replies:** 1\
**Last updated:** [February 4, 2021, 8:32pm UTC](https://discourse.julialang.org/t/sum-of-flux-params/54608 "2021-02-04T20:32:58Z")

</div>

I tried these two functions to get the l2 regularization term for a NN model: function ℓ₂(a) s = zero(Float32) for i in a s += sum(i.^2) end s end function l₂(a) s = zero(Float32) Lₐ = l…

---

## [Gradient penalty problems with relu](https://discourse.julialang.org/t/gradient-penalty-problems-with-relu/54594)

<div class="topic-metadata">

**Author:** [@ancorso](https://discourse.julialang.org/u/ancorso)\
**Replies:** 0\
**Last updated:** [February 4, 2021, 6:25am UTC](https://discourse.julialang.org/t/gradient-penalty-problems-with-relu/54594 "2021-02-04T06:25:32Z")

</div>

I am trying to implement the gradient penalty of WGAN-GP in Flux and Zygote, but am getting an error when I have relus in my model. See the following MWE. If I switch the relu to a tanh then it works fine. Thoughts? usi…

---

## [Plotting a decision tree and extracting the split criteria](https://discourse.julialang.org/t/plotting-a-decision-tree-and-extracting-the-split-criteria/54503)

<div class="topic-metadata">

**Author:** [@dataopt](https://discourse.julialang.org/u/dataopt)\
**Replies:** 0\
**Last updated:** [February 3, 2021, 12:46am UTC](https://discourse.julialang.org/t/plotting-a-decision-tree-and-extracting-the-split-criteria/54503 "2021-02-03T00:46:35Z")

</div>

I am new to MLJ coming from R for statistical learning. I have two newbie questions: Is there a way to plot a decision tree? Is there a way to extract the structure of the tree with the split criteria?

---

## [Can I modify Flux parameters manually?](https://discourse.julialang.org/t/can-i-modify-flux-parameters-manually/54344)

<div class="topic-metadata">

**Author:** [@DevJac](https://discourse.julialang.org/u/DevJac)\
**Replies:** 1\
**Last updated:** [January 31, 2021, 8:45pm UTC](https://discourse.julialang.org/t/can-i-modify-flux-parameters-manually/54344 "2021-01-31T20:45:38Z")

</div>

I want to be able to manually modify my Flux parameters, but when I have a Zygote.Params object, I can’t seem to change anything about it. I am trying to average the parameters of two networks that are the same shape (Po…

---

## [Computing Flux.gradient change the model](https://discourse.julialang.org/t/computing-flux-gradient-change-the-model/54284)

<div class="topic-metadata">

**Author:** [@anon74562486](https://discourse.julialang.org/u/anon74562486)\
**Replies:** 10\
**Last updated:** [January 31, 2021, 7:54pm UTC](https://discourse.julialang.org/t/computing-flux-gradient-change-the-model/54284 "2021-01-31T19:54:09Z")

</div>

I don’t speak English well, so sorry for the mistakes. I have an exercise of differential programming. I have an image taken from MNIST and a fully connected NN that classifies correctly that image. Then I want to mod…

---

## [Attempted to port an RL algorithm from PyTorch to Flux and it's 10x slower](https://discourse.julialang.org/t/attempted-to-port-an-rl-algorithm-from-pytorch-to-flux-and-its-10x-slower/54036)

<div class="topic-metadata">

**Author:** [@DevJac](https://discourse.julialang.org/u/DevJac)\
**Replies:** 17\
**Last updated:** [January 29, 2021, 2:08am UTC](https://discourse.julialang.org/t/attempted-to-port-an-rl-algorithm-from-pytorch-to-flux-and-its-10x-slower/54036 "2021-01-29T02:08:29Z")

</div>

I ported an algorithm described in the book Grokking Deep Reinforcement Learning (which I recommend if you’re a beginner to RL) from PyTorch to Julia. This algorithm is described in chapter 8 of the book, and it is meant…

---

## [Zygote differentiation issues](https://discourse.julialang.org/t/zygote-differentiation-issues/54130)

<div class="topic-metadata">

**Author:** [@adhikarirsr](https://discourse.julialang.org/u/adhikarirsr)\
**Replies:** 7\
**Last updated:** [January 28, 2021, 8:12pm UTC](https://discourse.julialang.org/t/zygote-differentiation-issues/54130 "2021-01-28T20:12:32Z")

</div>

In an attempt to replicate what I saw here: " The impact of differentiable programming: how ∂P is enabling new science in Julia" function g(x) if x \< 0 print("Enter function name: ") getfield(Base, S…

---

## [Error with Torch.jl](https://discourse.julialang.org/t/error-with-torch-jl/53712)

<div class="topic-metadata">

**Author:** [@LuisB](https://discourse.julialang.org/u/LuisB)\
**Replies:** 6\
**Last updated:** [January 26, 2021, 8:32am UTC](https://discourse.julialang.org/t/error-with-torch-jl/53712 "2021-01-26T08:32:27Z")

</div>

Hello, I am new using Julia. Currently, I am implementing a very simple example with Flux and Torch.jl, and I get the following error: getindex not defined for Tensor{Float32,2}. The code is the following: using Flux u…

---

## [What is the fastest dimension ordering for convolutions?](https://discourse.julialang.org/t/what-is-the-fastest-dimension-ordering-for-convolutions/46182)

<div class="topic-metadata">

**Author:** [@maxfreu](https://discourse.julialang.org/u/maxfreu)\
**Replies:** 1\
**Last updated:** [January 25, 2021, 12:00pm UTC](https://discourse.julialang.org/t/what-is-the-fastest-dimension-ordering-for-convolutions/46182 "2021-01-25T12:00:57Z")

</div>

Hi! I am wondering what the fastest dimension ordering for convolution operations on the GPU using CUDNN currently is. Flux uses WHCN (column major), pytorch uses NCHW (row major) which is exactly the same. So to say, t…

---

## [Comparisons between Julia (NeuralPDE.jl and DiffEqFlux.jl) and DeepXDE python package?](https://discourse.julialang.org/t/comparisons-between-julia-neuralpde-jl-and-diffeqflux-jl-and-deepxde-python-package/52669)

<div class="topic-metadata">

**Author:** [@bad\_at\_math](https://discourse.julialang.org/u/bad_at_math)\
**Replies:** 3\
**Last updated:** [January 25, 2021, 6:07am UTC](https://discourse.julialang.org/t/comparisons-between-julia-neuralpde-jl-and-diffeqflux-jl-and-deepxde-python-package/52669 "2021-01-25T06:07:18Z")

</div>

Hello everyone, I’ve recently come across the DeepXDE python package (https://github.com/lululxvi/deepxde), which can be used for formulating and solving a variety of neural differential equation - type problems. Beyond …

---

## [Different loss function for training ODE or DDE?](https://discourse.julialang.org/t/different-loss-function-for-training-ode-or-dde/53347)

<div class="topic-metadata">

**Author:** [@Qualium](https://discourse.julialang.org/u/Qualium)\
**Replies:** 5\
**Last updated:** [January 20, 2021, 7:05pm UTC](https://discourse.julialang.org/t/different-loss-function-for-training-ode-or-dde/53347 "2021-01-20T19:05:14Z")

</div>

I am learning to use the DiffEqFlux pkg with the tutorial and playing around and I have a weird problem, I cannot figure it out. In the tutorial they train the model adjusting parameters of the lotka-volterra system so b…

---

## [MLJ: accessing model fields from resampler](https://discourse.julialang.org/t/mlj-accessing-model-fields-from-resampler/53551)

<div class="topic-metadata">

**Author:** [@Pere](https://discourse.julialang.org/u/Pere)\
**Replies:** 4\
**Last updated:** [January 19, 2021, 9:37pm UTC](https://discourse.julialang.org/t/mlj-accessing-model-fields-from-resampler/53551 "2021-01-19T21:37:40Z")

</div>

I have a composite model that has a parameter (alpha) that generates a probability distribution and then the features and samples are weighted according to this distribution. I also want the samples to be taken according…

---

## [Runtime Generated Functions](https://discourse.julialang.org/t/runtime-generated-functions/53507)

<div class="topic-metadata">

**Author:** [@Ruchitha\_Kumar](https://discourse.julialang.org/u/Ruchitha_Kumar)\
**Replies:** 10\
**Last updated:** [January 18, 2021, 12:08am UTC](https://discourse.julialang.org/t/runtime-generated-functions/53507 "2021-01-18T00:08:46Z")

</div>

Blockquote nlsys\_func = generate\_function(ns, \[x,y,z\], \[a,b,c\], expression=Val{false})\[2\] Generates : (::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(Symbol(“##MTIIPVar#873”), Symbol(“##MTKArg#870”), Symbol(“##…

---

## [RNN (LSTM) Training issues with loss function](https://discourse.julialang.org/t/rnn-lstm-training-issues-with-loss-function/53483)

<div class="topic-metadata">

**Author:** [@jparcinski](https://discourse.julialang.org/u/jparcinski)\
**Replies:** 0\
**Last updated:** [January 17, 2021, 12:11pm UTC](https://discourse.julialang.org/t/rnn-lstm-training-issues-with-loss-function/53483 "2021-01-17T12:11:36Z")

</div>

Hi, I have problem with understanding 1 issue, maybe someone will help me. I’m trying to build LSTM or simple RNN for predicting price values. To start understand how to do it in Julia I used the example from here: A Ba…

---

## [Gradient error in Flux model inputs](https://discourse.julialang.org/t/gradient-error-in-flux-model-inputs/53259)

<div class="topic-metadata">

**Author:** [@Elmo](https://discourse.julialang.org/u/Elmo)\
**Replies:** 5\
**Last updated:** [January 13, 2021, 4:13pm UTC](https://discourse.julialang.org/t/gradient-error-in-flux-model-inputs/53259 "2021-01-13T16:13:54Z")

</div>

Hi all, I am trying to construct a loss function in Flux that is composed of a typical mean squared error component, as well as a derivative component. For some reason I get a Can't differentiate foreigncall expression e…

---

## [Take gradient of parameters not working](https://discourse.julialang.org/t/take-gradient-of-parameters-not-working/53250)

<div class="topic-metadata">

**Author:** [@Elmo](https://discourse.julialang.org/u/Elmo)\
**Replies:** 0\
**Last updated:** [January 12, 2021, 8:20pm UTC](https://discourse.julialang.org/t/take-gradient-of-parameters-not-working/53250 "2021-01-12T20:20:58Z")

</div>

Hi all! I am trying to construct a loss function in Flux (v0.11.2) that regularizes the parameters relative to some number (not 0). Below is a minimal working example, does anybody know why the gradients are nothing? us…

---

## [SingularSpectrumAnalysis and CPU load](https://discourse.julialang.org/t/singularspectrumanalysis-and-cpu-load/53212)

<div class="topic-metadata">

**Author:** [@Colin\_Beckingham](https://discourse.julialang.org/u/Colin_Beckingham)\
**Replies:** 3\
**Last updated:** [January 12, 2021, 6:09pm UTC](https://discourse.julialang.org/t/singularspectrumanalysis-and-cpu-load/53212 "2021-01-12T18:09:21Z")

</div>

SSA does a good job of extracting trend and seasonality from a time series. However widening the window on a large dataset puts quite a load on my CPU (GPU is too old to contribute to calculation in Julia) heats up my CP…

---

## [Is it a good time for a PyTorch developer to move to Julia? If so, Flux? Knet?](https://discourse.julialang.org/t/is-it-a-good-time-for-a-pytorch-developer-to-move-to-julia-if-so-flux-knet/38453)

<div class="topic-metadata">

**Author:** [@Alon](https://discourse.julialang.org/u/Alon)\
**Replies:** 52\
**Last updated:** [January 11, 2021, 9:51pm UTC](https://discourse.julialang.org/t/is-it-a-good-time-for-a-pytorch-developer-to-move-to-julia-if-so-flux-knet/38453 "2021-01-11T21:51:35Z")

</div>

For me, the move from Keras to PyTorch was the best decision in my AI career. I am considering moving from Python to Julia because of performance and because the “NumPy” is built-in (as it was in Matlab rest its soul). …

---

## [MLJ - Error - Models support multitarget but measures do not?](https://discourse.julialang.org/t/mlj-error-models-support-multitarget-but-measures-do-not/53091)

<div class="topic-metadata">

**Author:** [@diadora77](https://discourse.julialang.org/u/diadora77)\
**Replies:** 2\
**Last updated:** [January 10, 2021, 1:50pm UTC](https://discourse.julialang.org/t/mlj-error-models-support-multitarget-but-measures-do-not/53091 "2021-01-10T13:50:22Z")

</div>

Hi- I’m relatively new to MLJ but have decent knowledge and experience using ScikitLearn. MLJ looks promising with more flexibility in model creation. In testing it out for my research, I’m struggling to get a basic mod…

---

## [Nonlinear fit with Flux](https://discourse.julialang.org/t/nonlinear-fit-with-flux/53075)

<div class="topic-metadata">

**Author:** [@edwinb-ai](https://discourse.julialang.org/u/edwinb-ai)\
**Replies:** 2\
**Last updated:** [January 10, 2021, 4:10am UTC](https://discourse.julialang.org/t/nonlinear-fit-with-flux/53075 "2021-01-10T04:10:43Z")

</div>

using Flux using Flux.Losses: mse using Flux.Data: DataLoader using Plots using Random gr() nonlinear(x) = sin(x) + sin(2 \* x) + 0.1f0 \* randn() function get\_data() points = 1000 xvals = collect(Float32, range…

---

## [Gradient of NN with respect to inputs](https://discourse.julialang.org/t/gradient-of-nn-with-respect-to-inputs/53037)

<div class="topic-metadata">

**Author:** [@Sunny](https://discourse.julialang.org/u/Sunny)\
**Replies:** 1\
**Last updated:** [January 8, 2021, 1:21pm UTC](https://discourse.julialang.org/t/gradient-of-nn-with-respect-to-inputs/53037 "2021-01-08T13:21:46Z")

</div>

I want to take a gradient of NN but get the error MethodError: no method matching extract\_gradient!(::Type{ForwardDiff.Tag{Chain{Tuple{Dense{typeof(relu),Array{Float32,2},Array{Float32,1}},Dense{typeof(identity),Array{F…

---

## [\[DiffEqFlux.jl\] DimensionMismatch("array could not be broadcast to match destination")](https://discourse.julialang.org/t/diffeqflux-jl-dimensionmismatch-array-could-not-be-broadcast-to-match-destination/52992)

<div class="topic-metadata">

**Author:** [@vkv](https://discourse.julialang.org/u/vkv)\
**Replies:** 0\
**Last updated:** [January 7, 2021, 2:14pm UTC](https://discourse.julialang.org/t/diffeqflux-jl-dimensionmismatch-array-could-not-be-broadcast-to-match-destination/52992 "2021-01-07T14:14:21Z")

</div>

I’m trying to train a NeuralODE model on the MNIST dataset. When I take the gradient of loss function I get the DimensionMismatch error. using DiffEqFlux, OrdinaryDiffEq, Flux, NNlib, MLDataUtils, Printf using Flux: log…

---

## [\[ANN\] Latin hypercube on MLJTuning](https://discourse.julialang.org/t/ann-latin-hypercube-on-mljtuning/52954)

<div class="topic-metadata">

**Author:** [@ludoro](https://discourse.julialang.org/u/ludoro)\
**Replies:** 0\
**Last updated:** [January 6, 2021, 6:05pm UTC](https://discourse.julialang.org/t/ann-latin-hypercube-on-mljtuning/52954 "2021-01-06T18:05:49Z")

</div>

Hello everyone! Just a quick announcement. MLJTuning now has a new tuning strategy: Latin Hypercube.:tada: It will be live when https://github.com/JuliaRegistries/General/pull/27394 gets merged. You can take a look at th…

---

## [RNN Not learning](https://discourse.julialang.org/t/rnn-not-learning/51686)

<div class="topic-metadata">

**Author:** [@Emilio4d46](https://discourse.julialang.org/u/Emilio4d46)\
**Replies:** 9\
**Last updated:** [January 4, 2021, 9:46am UTC](https://discourse.julialang.org/t/rnn-not-learning/51686 "2021-01-04T09:46:32Z")

</div>

I have been trying to build an RNN in Julia using Flux and it does not seem to converge or learn at all. The code is learning on a parsed Stanford Sentiment Treebank (i.e. An array of words and a sentiment score) Th…

---

## [Working implementation of MAML (meta-learning) for FluxML?](https://discourse.julialang.org/t/working-implementation-of-maml-meta-learning-for-fluxml/51954)

<div class="topic-metadata">

**Author:** [@bad\_at\_math](https://discourse.julialang.org/u/bad_at_math)\
**Replies:** 1\
**Last updated:** [January 3, 2021, 5:37pm UTC](https://discourse.julialang.org/t/working-implementation-of-maml-meta-learning-for-fluxml/51954 "2021-01-03T17:37:24Z")

</div>

Hi everyone, I was wondering if there are any public implementations of meta-learning techniques such as MAML (\[1703.03400\] Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks) in Flux. The model zoo has a…

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=36)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=38)