# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=32

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 33

---

## [Flux running slow?](https://discourse.julialang.org/t/flux-running-slow/46032)

<div class="topic-metadata">

**Author:** [@ChrisBlake](https://discourse.julialang.org/u/ChrisBlake)\
**Replies:** 16\
**Last updated:** [August 19, 2021, 5:35am UTC](https://discourse.julialang.org/t/flux-running-slow/46032 "2021-08-19T05:35:24Z")

</div>

Hi, I’m new to Julia and testing it out for ML, hoping to use it for RL which is my main area of work. I noticed that Flux seems to be running much slower than I would have expected. I created a simple example of how I’…

---

## [Conv\_mnist model-zoo example,](https://discourse.julialang.org/t/conv-mnist-model-zoo-example/66592)

<div class="topic-metadata">

**Author:** [@extremety1989](https://discourse.julialang.org/u/extremety1989)\
**Replies:** 9\
**Last updated:** [August 18, 2021, 5:09pm UTC](https://discourse.julialang.org/t/conv-mnist-model-zoo-example/66592 "2021-08-18T17:09:17Z")

</div>

I loaded mnist test data from Kaggle, i wanted to test my pretrained model from conv\_mnist, how to use pretrained model to predict image ? It says: LeNet5 “constructor”. The model can be adapted to any image size and a…

---

## [\[Graph\]CodeBERT; and e.g. (2- to) 8-bit int networks better than float, for Transformers/BERT](https://discourse.julialang.org/t/graph-codebert-and-e-g-2-to-8-bit-int-networks-better-than-float-for-transformers-bert/66559)

<div class="topic-metadata">

**Author:** [@Palli](https://discourse.julialang.org/u/Palli)\
**Replies:** 0\
**Last updated:** [August 17, 2021, 4:49pm UTC](https://discourse.julialang.org/t/graph-codebert-and-e-g-2-to-8-bit-int-networks-better-than-float-for-transformers-bert/66559 "2021-08-17T16:49:01Z")

</div>

A. Pre-trained models for programming language have achieved dramatic empirical improvements on a variety of code-related tasks such as \[…\] However, existing pre-trained models regard a code snippet as a sequence of t…

---

## [Connection between sensitivity algorithm and gradient calculation of the loss function](https://discourse.julialang.org/t/connection-between-sensitivity-algorithm-and-gradient-calculation-of-the-loss-function/66550)

<div class="topic-metadata">

**Author:** [@hardy](https://discourse.julialang.org/u/hardy)\
**Replies:** 6\
**Last updated:** [August 17, 2021, 9:22pm UTC](https://discourse.julialang.org/t/connection-between-sensitivity-algorithm-and-gradient-calculation-of-the-loss-function/66550 "2021-08-17T21:22:31Z")

</div>

I’m new to Julia and ML and try to understand the basics. I have defined an ODE problem with 2 parameters. This is used for the prediction in my loss function. This loss function is now used in turn to optimize the para…

---

## [How to convert a JPEG image (from mobile) to a stantard 60k Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8, 8}}, 2}) (from Flux.Data.MNIST)?](https://discourse.julialang.org/t/how-to-convert-a-jpeg-image-from-mobile-to-a-stantard-60k-array-colortypes-gray-fixedpointnumbers-normed-uint8-8-2-from-flux-data-mnist/66475)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 5\
**Last updated:** [August 17, 2021, 10:20am UTC](https://discourse.julialang.org/t/how-to-convert-a-jpeg-image-from-mobile-to-a-stantard-60k-array-colortypes-gray-fixedpointnumbers-normed-uint8-8-2-from-flux-data-mnist/66475 "2021-08-17T10:20:15Z")

</div>

For an exercise I want to let students draw their own digits, scan them with a mobile and let a MINST-trained flux NN to category them. I see that the single MINST images are of type Matrix{ColorTypes.Gray{FixedPointNum…

---

## [Question on QSIR Model and Flux.train for COVID-19 SIR data](https://discourse.julialang.org/t/question-on-qsir-model-and-flux-train-for-covid-19-sir-data/66422)

<div class="topic-metadata">

**Author:** [@msteele-uofa-ece](https://discourse.julialang.org/u/msteele-uofa-ece)\
**Replies:** 3\
**Last updated:** [August 16, 2021, 9:33pm UTC](https://discourse.julialang.org/t/question-on-qsir-model-and-flux-train-for-covid-19-sir-data/66422 "2021-08-16T21:33:32Z")

</div>

Hello, My name is Manuel Steele. I am a graduate student at St. Mary’s College for data science (Notre Dame, IN). I applied the model at the github site below to a smaller county in North Dakota. The model was succes…

---

## [Flux question: best way to write a function that reuses some terms](https://discourse.julialang.org/t/flux-question-best-way-to-write-a-function-that-reuses-some-terms/66510)

<div class="topic-metadata">

**Author:** [@IRM](https://discourse.julialang.org/u/IRM)\
**Replies:** 1\
**Last updated:** [August 16, 2021, 8:43pm UTC](https://discourse.julialang.org/t/flux-question-best-way-to-write-a-function-that-reuses-some-terms/66510 "2021-08-16T20:43:45Z")

</div>

I have a question about the best (most computationally efficient) way to write functions for automatic differentiation in Flux in which some terms are reused. The particular example I give is motivated by the paper Jauc…

---

## [Flux assumes only trainable parameters should be moved to GPU](https://discourse.julialang.org/t/flux-assumes-only-trainable-parameters-should-be-moved-to-gpu/66429)

<div class="topic-metadata">

**Author:** [@SebastianCallh](https://discourse.julialang.org/u/SebastianCallh)\
**Replies:** 3\
**Last updated:** [August 16, 2021, 4:38pm UTC](https://discourse.julialang.org/t/flux-assumes-only-trainable-parameters-should-be-moved-to-gpu/66429 "2021-08-16T16:38:13Z")

</div>

And that’s not always a good assumption! I am working on a masked dense layer, which looks something like this struct MaskedDense{F, M \<: AbstractMatrix, B \<: AbstractVector} W::M b::B M::AbstractMatrix{Bool} σ:…

---

## [Horovod and Flux / Julia](https://discourse.julialang.org/t/horovod-and-flux-julia/53550)

<div class="topic-metadata">

**Author:** [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Replies:** 7\
**Last updated:** [August 15, 2021, 4:46pm UTC](https://discourse.julialang.org/t/horovod-and-flux-julia/53550 "2021-08-15T16:46:14Z")

</div>

Hi all, I wonder if anyone was trying to use horovod with Flux (or Julia in general). Thanks for a answer. Best wishes, Tomas

---

## [Combining multiple-shooting with out-of-ODE multithreading](https://discourse.julialang.org/t/combining-multiple-shooting-with-out-of-ode-multithreading/66338)

<div class="topic-metadata">

**Author:** [@cmarcotte](https://discourse.julialang.org/u/cmarcotte)\
**Replies:** 1\
**Last updated:** [August 13, 2021, 12:05pm UTC](https://discourse.julialang.org/t/combining-multiple-shooting-with-out-of-ode-multithreading/66338 "2021-08-13T12:05:41Z")

</div>

DiffEqFlux has examples for multiple shooting and out-of-ODE multithreading here and here, respectively. This seems like a natural pairing, but it seems like multiple\_shoot is written for a completely serial implementat…

---

## [AssertionError GCNConv FeaturedGraph GeometricFlux](https://discourse.julialang.org/t/assertionerror-gcnconv-featuredgraph-geometricflux/66061)

<div class="topic-metadata">

**Author:** [@riegel\_gestr](https://discourse.julialang.org/u/riegel_gestr)\
**Replies:** 1\
**Last updated:** [August 13, 2021, 7:09am UTC](https://discourse.julialang.org/t/assertionerror-gcnconv-featuredgraph-geometricflux/66061 "2021-08-13T07:09:24Z")

</div>

I am trying the GeometricFlux library by doing some test (coming from torch\_geometric). So test that I am doing is a simple graphs classification and I have considered very simple input: N = 100 k = 10 g = erdos\_renyi(…

---

## [Recent AI developments: Roformer (transformer w/Rotary Position Embedding) and DL to Rejuvenate Symbolic AI: Neural Production](https://discourse.julialang.org/t/recent-ai-developments-roformer-transformer-w-rotary-position-embedding-and-dl-to-rejuvenate-symbolic-ai-neural-production/66314)

<div class="topic-metadata">

**Author:** [@Palli](https://discourse.julialang.org/u/Palli)\
**Replies:** 4\
**Last updated:** [August 13, 2021, 12:56am UTC](https://discourse.julialang.org/t/recent-ai-developments-roformer-transformer-w-rotary-position-embedding-and-dl-to-rejuvenate-symbolic-ai-neural-production/66314 "2021-08-13T00:56:08Z")

</div>

A. Seems important: ROFORMER: ENHANCED TRANSFORMER WITH ROTARY POSITION EMBEDDING We investigate various methods to encode positional information in transformer-based language models and propose a novel implementati…

---

## [FeaturedGraph Explanation](https://discourse.julialang.org/t/featuredgraph-explanation/66294)

<div class="topic-metadata">

**Author:** [@NFSturm](https://discourse.julialang.org/u/NFSturm)\
**Replies:** 0\
**Last updated:** [August 12, 2021, 8:33pm UTC](https://discourse.julialang.org/t/featuredgraph-explanation/66294 "2021-08-12T20:33:34Z")

</div>

Dear Julia Community, I was recently becoming interested in Machine Learning on graphs I found the GeometricFlux library that seems to be quite useful in that regard. My understanding of graphs in general is rather basi…

---

## [Get training data from saved machine](https://discourse.julialang.org/t/get-training-data-from-saved-machine/66114)

<div class="topic-metadata">

**Author:** [@Vishal\_Sharma1](https://discourse.julialang.org/u/Vishal_Sharma1)\
**Replies:** 1\
**Last updated:** [August 11, 2021, 9:13pm UTC](https://discourse.julialang.org/t/get-training-data-from-saved-machine/66114 "2021-08-11T21:13:08Z")

</div>

Using MLJ, after saving and retrieving a machine - Is there any method to retrieve the training data that was used to train this machine. I am unable to find any such method in documentation.

---

## [NeuralODE: optimal control action in a constrained optimization problem](https://discourse.julialang.org/t/neuralode-optimal-control-action-in-a-constrained-optimization-problem/63372)

<div class="topic-metadata">

**Author:** [@Antonino\_Gandolfo](https://discourse.julialang.org/u/Antonino_Gandolfo)\
**Replies:** 13\
**Last updated:** [July 26, 2021, 5:03pm UTC](https://discourse.julialang.org/t/neuralode-optimal-control-action-in-a-constrained-optimization-problem/63372 "2021-07-26T17:03:01Z")

</div>

Hi, I’m new with Julia. I want to ask if it’s possible to optimize an differential equation using the NeuralODE. I try to create my code but the problems are the physical constrains. I don’t know how to add this equa…

---

## [Any implementation of 'Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs'?](https://discourse.julialang.org/t/any-implementation-of-fixed-smooth-convolutional-layer-for-avoiding-checkerboard-artifacts-in-cnns/65881)

<div class="topic-metadata">

**Author:** [@czexana](https://discourse.julialang.org/u/czexana)\
**Replies:** 2\
**Last updated:** [August 6, 2021, 9:10am UTC](https://discourse.julialang.org/t/any-implementation-of-fixed-smooth-convolutional-layer-for-avoiding-checkerboard-artifacts-in-cnns/65881 "2021-08-06T09:10:54Z")

</div>

Hello, I’m wondering if anyone has implemented the above paper, found at \[2002.02117\] Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs that purports to fix checkerboard artifacts in conv and …

---

## [How to make Zygote avoid differentiating with respect to some fields in struct](https://discourse.julialang.org/t/how-to-make-zygote-avoid-differentiating-with-respect-to-some-fields-in-struct/65520)

<div class="topic-metadata">

**Author:** [@Nikos\_Gianniotis](https://discourse.julialang.org/u/Nikos_Gianniotis)\
**Replies:** 6\
**Last updated:** [August 2, 2021, 8:08pm UTC](https://discourse.julialang.org/t/how-to-make-zygote-avoid-differentiating-with-respect-to-some-fields-in-struct/65520 "2021-08-02T20:08:51Z")

</div>

Hello everyone, I have been always used ForwardDiff for my automatic differentiation needs, but recently I thought I should try out Zygote instead. My main motivation for doing so is the ability of Zygote to work with c…

---

## [Help building a PID example](https://discourse.julialang.org/t/help-building-a-pid-example/27194)

<div class="topic-metadata">

**Author:** [@Ronis\_BR](https://discourse.julialang.org/u/Ronis_BR)\
**Replies:** 11\
**Last updated:** [July 31, 2021, 2:30pm UTC](https://discourse.julialang.org/t/help-building-a-pid-example/27194 "2021-07-31T14:30:59Z")

</div>

Hi guys! I am trying to use Flux.jl to build a PIDNN as shown here: PID Neural Network I started to code using only the proportional part. This, I need an input layer with 2 inputs and 1 output, and one output layer wi…

---

## [CUDA.jl - ERROR: LoadError: LoadError: UndefVarError: parameters not defined](https://discourse.julialang.org/t/cuda-jl-error-loaderror-loaderror-undefvarerror-parameters-not-defined/65501)

<div class="topic-metadata">

**Author:** [@molen89](https://discourse.julialang.org/u/molen89)\
**Replies:** 1\
**Last updated:** [July 30, 2021, 8:58am UTC](https://discourse.julialang.org/t/cuda-jl-error-loaderror-loaderror-undefvarerror-parameters-not-defined/65501 "2021-07-30T08:58:02Z")

</div>

I am having issues with the cuda/flux packages when using the gpu. Example code using Flux, CUDA m = Dense(10,5) |\> gpu x = rand(10) |\> gpu m(x) \> UndefVarError: parameters not defined versions Julia 1.6.1 CUDA 11.…

---

## [Data-formatting in and out of Flux ML model](https://discourse.julialang.org/t/data-formatting-in-and-out-of-flux-ml-model/65383)

<div class="topic-metadata">

**Author:** [@amit1](https://discourse.julialang.org/u/amit1)\
**Replies:** 1\
**Last updated:** [July 28, 2021, 4:15pm UTC](https://discourse.julialang.org/t/data-formatting-in-and-out-of-flux-ml-model/65383 "2021-07-28T16:15:29Z")

</div>

Hi, I am still learning the ropes of Flux framework. I have experience of tensorflow and bit of pytorch, but I am interested in Flux, mostly because of the flexibility and uniformity (no more to\_numpy and vice versa!). I…

---

## [Understanding MultiAgent in ReinforcementLearning.jl](https://discourse.julialang.org/t/understanding-multiagent-in-reinforcementlearning-jl/65269)

<div class="topic-metadata">

**Author:** [@thae](https://discourse.julialang.org/u/thae)\
**Replies:** 2\
**Last updated:** [July 27, 2021, 2:04pm UTC](https://discourse.julialang.org/t/understanding-multiagent-in-reinforcementlearning-jl/65269 "2021-07-27T14:04:44Z")

</div>

Hello, let it be known that I know little about Julia and even less about Reinforcement Learning. I’ve written an environment so trivial it is meaningless just to understand how everything fits together. It’s a four-pla…

---

## [Need Diffractor.jl for State-of-the-Art Deep Learning Model](https://discourse.julialang.org/t/need-diffractor-jl-for-state-of-the-art-deep-learning-model/59423)

<div class="topic-metadata">

**Author:** [@Freddy\_Chua](https://discourse.julialang.org/u/Freddy_Chua)\
**Replies:** 12\
**Last updated:** [July 27, 2021, 11:03am UTC](https://discourse.julialang.org/t/need-diffractor-jl-for-state-of-the-art-deep-learning-model/59423 "2021-07-27T11:03:15Z")

</div>

Hi, I hope to beta test for your Diffractor.jl next generation AD. I am a co-author of DeepCPCFG \[2103.05908\] DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information Extraction. This research is a…

---

## [Sparse Feed Forward NN](https://discourse.julialang.org/t/sparse-feed-forward-nn/33203)

<div class="topic-metadata">

**Author:** [@olszewskip](https://discourse.julialang.org/u/olszewskip)\
**Replies:** 3\
**Last updated:** [July 25, 2021, 1:27pm UTC](https://discourse.julialang.org/t/sparse-feed-forward-nn/33203 "2021-07-25T13:27:10Z")

</div>

Hi! I have a question about (hypothetical) sparse layers in feedforward neural neworks: How would I go about using sparse array in place of W(eights) and b(iases) in the Flux’es Dense layer? What I imagine is that I hav…

---

## [MethodError: No Method Matching Learning Curve](https://discourse.julialang.org/t/methoderror-no-method-matching-learning-curve/65046)

<div class="topic-metadata">

**Author:** [@YummyPampers2](https://discourse.julialang.org/u/YummyPampers2)\
**Replies:** 3\
**Last updated:** [July 22, 2021, 10:33pm UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-learning-curve/65046 "2021-07-22T22:33:32Z")

</div>

Hello Everyone: I am attempting to run: curve\_clf = MLJ.learning\_curve(r\_clf, feat, targ, range=r\_clf, resampling=MLJ.Holdout(fraction\_train=0.8), measure=MLJ.cross\_entropy) But am encounte…

---

## [Suggestions for improved learning in neural network based controller for rotary inverted pendulum](https://discourse.julialang.org/t/suggestions-for-improved-learning-in-neural-network-based-controller-for-rotary-inverted-pendulum/65009)

<div class="topic-metadata">

**Author:** [@shafeeq\_Elanattil](https://discourse.julialang.org/u/shafeeq_Elanattil)\
**Replies:** 2\
**Last updated:** [July 22, 2021, 12:49am UTC](https://discourse.julialang.org/t/suggestions-for-improved-learning-in-neural-network-based-controller-for-rotary-inverted-pendulum/65009 "2021-07-22T00:49:05Z")

</div>

Hello, I am new to Julia, and trying to make use of neural network based controllers for robotics applications. Here I am trying balance an rotary inverted pendulum using neural network based controller. The code is giv…

---

## [Writing pytorch layers in julia](https://discourse.julialang.org/t/writing-pytorch-layers-in-julia/65064)

<div class="topic-metadata">

**Author:** [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Replies:** 0\
**Last updated:** [July 21, 2021, 7:26pm UTC](https://discourse.julialang.org/t/writing-pytorch-layers-in-julia/65064 "2021-07-21T19:26:02Z")

</div>

Hi All, From time to time, I am pushed to rewrite Mill.jl (multi-instance learning library) in pytorch. While that might be possible, i would like to avoid it, as we made heavy use of custom written gradients to help us…

---

## [Unpredictable learning of neural network based controller for dc motor](https://discourse.julialang.org/t/unpredictable-learning-of-neural-network-based-controller-for-dc-motor/64960)

<div class="topic-metadata">

**Author:** [@shafeeq\_Elanattil](https://discourse.julialang.org/u/shafeeq_Elanattil)\
**Replies:** 2\
**Last updated:** [July 20, 2021, 4:22am UTC](https://discourse.julialang.org/t/unpredictable-learning-of-neural-network-based-controller-for-dc-motor/64960 "2021-07-20T04:22:14Z")

</div>

Hello, I am new to the Julia language. I am trying to make neural network based position controller using Julia libraries. My code is given below. J = 0.1999 Ke = 2.1918 Kt = 38.185 L = 0.020453 R = 3.9694 Tq = 0.20142…

---

## [ZygoteRules dispatch on traits?](https://discourse.julialang.org/t/zygoterules-dispatch-on-traits/64869)

<div class="topic-metadata">

**Author:** [@lamorton](https://discourse.julialang.org/u/lamorton)\
**Replies:** 2\
**Last updated:** [July 19, 2021, 5:52pm UTC](https://discourse.julialang.org/t/zygoterules-dispatch-on-traits/64869 "2021-07-19T17:52:05Z")

</div>

Is it possible to specialize a ZygoteRule using a Holy trait? I’m not sure how I would go about doing this.

---

## [Turing.jl - Error running tutorial code](https://discourse.julialang.org/t/turing-jl-error-running-tutorial-code/64867)

<div class="topic-metadata">

**Author:** [@v-i-s-h](https://discourse.julialang.org/u/v-i-s-h)\
**Replies:** 2\
**Last updated:** [July 19, 2021, 1:23pm UTC](https://discourse.julialang.org/t/turing-jl-error-running-tutorial-code/64867 "2021-07-19T13:23:54Z")

</div>

I was try Turing.jl tutorial code for Bayesian Neural networks (Bayesian Neural Networks) When I try to run the Variational inference code (code below) using Bijectors using Turing: Variational using AdvancedVI m = ba…

---

## [Why does \`Flux.stack\` use splatting?](https://discourse.julialang.org/t/why-does-flux-stack-use-splatting/64923)

<div class="topic-metadata">

**Author:** [@bad\_at\_math](https://discourse.julialang.org/u/bad_at_math)\
**Replies:** 1\
**Last updated:** [July 19, 2021, 1:12pm UTC](https://discourse.julialang.org/t/why-does-flux-stack-use-splatting/64923 "2021-07-19T13:12:49Z")

</div>

The Flux.stack function is defined here: https://github.com/FluxML/Flux.jl/blob/b78a27b01c9629099adb059a98657b995760b617/src/utils.jl#L476, and it is very simple: stack(xs, dim) = cat(unsqueeze.(xs, dim)..., dims=dim). …

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=31)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=33)