# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=39

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 40

---

## [MLJFlux much faster for simple MNIST example than TensorFlow?](https://discourse.julialang.org/t/mljflux-much-faster-for-simple-mnist-example-than-tensorflow/49646)

<div class="topic-metadata">

**Author:** [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Replies:** 0\
**Last updated:** [November 5, 2020, 9:57pm UTC](https://discourse.julialang.org/t/mljflux-much-faster-for-simple-mnist-example-than-tensorflow/49646 "2020-11-05T21:57:26Z")

</div>

I am trying to port this TensorFlow MNIST example to MLJFlux.jl. This is what I have so far, it is currently achieving an accuracy of 97.1%, which is pretty similar to the TensorFlow example: using MLJ, MLJFlux, Flux m…

---

## [ERROR: MethodError: no method matching PhysicsInformedNN(::Float64)](https://discourse.julialang.org/t/error-methoderror-no-method-matching-physicsinformednn-float64/49610)

<div class="topic-metadata">

**Author:** [@BaoBaoDeMeng](https://discourse.julialang.org/u/BaoBaoDeMeng)\
**Replies:** 1\
**Last updated:** [November 5, 2020, 4:37am UTC](https://discourse.julialang.org/t/error-methoderror-no-method-matching-physicsinformednn-float64/49610 "2020-11-05T04:37:32Z")

</div>

using Flux, DiffEqFluxModeling,ToolkitDiffEqBase, Plots, NeuralPDETest Example 1, 1D ode 1D ODE eq = Dt(u(t,θ)) ~ t^3 + 2t + (t^2)((1+3\*(t^2))/(1+t+(t^3))) - u(t,θ)(t + ((1+3(t^2))/(1+t+t^3))) Boundary conditions bc…

---

## [Implement Mixed Models with sparse X and Z matrices](https://discourse.julialang.org/t/implement-mixed-models-with-sparse-x-and-z-matrices/49539)

<div class="topic-metadata">

**Author:** [@Antonina\_Klyuyeva](https://discourse.julialang.org/u/Antonina_Klyuyeva)\
**Replies:** 7\
**Last updated:** [November 3, 2020, 7:48pm UTC](https://discourse.julialang.org/t/implement-mixed-models-with-sparse-x-and-z-matrices/49539 "2020-11-03T19:48:53Z")

</div>

Hi there, I use mixed models on a large file (500000 rows). My model formula looks like this: Y ~ 0 + num1:factor1 + num1:factor2 + num2:factor3 + factor4 + (0 + num3|subject) + (0 + num4|subject) + (1|subject), whe…

---

## [Flux con! warning](https://discourse.julialang.org/t/flux-con-warning/49456)

<div class="topic-metadata">

**Author:** [@Krzysztof\_Langner](https://discourse.julialang.org/u/Krzysztof_Langner)\
**Replies:** 4\
**Last updated:** [November 2, 2020, 9:41pm UTC](https://discourse.julialang.org/t/flux-con-warning/49456 "2020-11-02T21:41:59Z")

</div>

Hi, I’m trying to build simple model which works with MNIST dataset. It works fine, but I get some warning and was wandering if this is something which I can improve Here is the model: Chain( Conv((5, 5), 1=\>8…

---

## [Problem using MLJ and Multiclass scitypes](https://discourse.julialang.org/t/problem-using-mlj-and-multiclass-scitypes/49412)

<div class="topic-metadata">

**Author:** [@dmolina](https://discourse.julialang.org/u/dmolina)\
**Replies:** 4\
**Last updated:** [November 2, 2020, 5:23pm UTC](https://discourse.julialang.org/t/problem-using-mlj-and-multiclass-scitypes/49412 "2020-11-02T17:23:05Z")

</div>

Hello everyone, I am teaching a course using Scikit-Learn, and I wanted to translate my materials to MLJ (for a possible future :slight\_smile: ), but I am having many problems. I am using MLJ#dev. First, I have a dataf…

---

## [Regression implementation using FixedEffectModels.jl, InteractiveFixedEffectModels.jl, GLFixedEffectModels.jl packages](https://discourse.julialang.org/t/regression-implementation-using-fixedeffectmodels-jl-interactivefixedeffectmodels-jl-glfixedeffectmodels-jl-packages/49036)

<div class="topic-metadata">

**Author:** [@Antonina\_Klyuyeva](https://discourse.julialang.org/u/Antonina_Klyuyeva)\
**Replies:** 18\
**Last updated:** [November 2, 2020, 12:10pm UTC](https://discourse.julialang.org/t/regression-implementation-using-fixedeffectmodels-jl-interactivefixedeffectmodels-jl-glfixedeffectmodels-jl-packages/49036 "2020-11-02T12:10:05Z")

</div>

Hi guys, I need to implement a regression like Y ~ A : B : C or Y ~ A : B\_C where main effects are missing, in interactions A - numeric variable, B and C - categorical variables (factors)) or in the second case - va…

---

## [Second derivative of custom neural network - gradient error](https://discourse.julialang.org/t/second-derivative-of-custom-neural-network-gradient-error/49311)

<div class="topic-metadata">

**Author:** [@Andreas\_Schlaginhauf](https://discourse.julialang.org/u/Andreas_Schlaginhauf)\
**Replies:** 3\
**Last updated:** [October 31, 2020, 6:10pm UTC](https://discourse.julialang.org/t/second-derivative-of-custom-neural-network-gradient-error/49311 "2020-10-31T18:10:14Z")

</div>

I am trying to take the second derivative of a custom neural network like this, icnn = ICNN(1, \[32, 32\], soft\_relu(0.1)) function df(x) gradient(y-\>sum(icnn(y)), x)\[1\] end df(\[1\]) gradient(y-\>sum(df(y)), \[1\]) This …

---

## [RandomForest with out of bag prediction](https://discourse.julialang.org/t/randomforest-with-out-of-bag-prediction/49377)

<div class="topic-metadata">

**Author:** [@lubo](https://discourse.julialang.org/u/lubo)\
**Replies:** 0\
**Last updated:** [October 31, 2020, 5:50pm UTC](https://discourse.julialang.org/t/randomforest-with-out-of-bag-prediction/49377 "2020-10-31T17:50:09Z")

</div>

Hi, please, is there any implementation of RandomForest with Out of bag predictions? Seems to the reference RF package DecisionTree.jl has still this only as TODO (long time). Another package with OOB RandomForests.jl …

---

## [Regularization with Flux](https://discourse.julialang.org/t/regularization-with-flux/49232)

<div class="topic-metadata">

**Author:** [@Mariana](https://discourse.julialang.org/u/Mariana)\
**Replies:** 3\
**Last updated:** [October 29, 2020, 5:09pm UTC](https://discourse.julialang.org/t/regularization-with-flux/49232 "2020-10-29T17:09:21Z")

</div>

I’m currently training an ANN with 54 inputs and 12 outputs and I have already achieved good results by using the following model: model = Chain(Dense(54,54,sigmoid), Dense(54,54,sigmoid), Dense(54,12,leakyrelu)). Howe…

---

## [Extract model coefficients with FixedEffectModels.jl package](https://discourse.julialang.org/t/extract-model-coefficients-with-fixedeffectmodels-jl-package/49174)

<div class="topic-metadata">

**Author:** [@Antonina\_Klyuyeva](https://discourse.julialang.org/u/Antonina_Klyuyeva)\
**Replies:** 14\
**Last updated:** [October 28, 2020, 6:26pm UTC](https://discourse.julialang.org/t/extract-model-coefficients-with-fixedeffectmodels-jl-package/49174 "2020-10-28T18:26:55Z")

</div>

Hi everyone! In my regression model (Y ~ A:B), a numeric variable (A) interacts with a categorical variable (B). Since the categorical variable has a lot of unique levels, fitting the model using GLM.jl package consumes…

---

## [Flux with Julia master fails with IRCode error](https://discourse.julialang.org/t/flux-with-julia-master-fails-with-ircode-error/48865)

<div class="topic-metadata">

**Author:** [@Colin\_Beckingham](https://discourse.julialang.org/u/Colin_Beckingham)\
**Replies:** 0\
**Last updated:** [October 23, 2020, 6:14am UTC](https://discourse.julialang.org/t/flux-with-julia-master-fails-with-ircode-error/48865 "2020-10-23T06:14:57Z")

</div>

Using the sample code from the documentation for Flux Optimizers, with Julia 1.5.2 the code works (apart from a warning about slow compilation) but fails with master on julia\> grads = gradient(() -\> loss(x, y), θ) ERROR…

---

## [Locally connected layer in Flux](https://discourse.julialang.org/t/locally-connected-layer-in-flux/48624)

<div class="topic-metadata">

**Author:** [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Replies:** 0\
**Last updated:** [October 19, 2020, 12:00pm UTC](https://discourse.julialang.org/t/locally-connected-layer-in-flux/48624 "2020-10-19T12:00:12Z")

</div>

Has anyone made an implementation of a locally connected layer in Flux similar to the one(s) in Keras?

---

## [Quaternion-, and up to, sedenion-valued neural networks: Parallelizing Hamilton product on GPUs/CUDA](https://discourse.julialang.org/t/quaternion-and-up-to-sedenion-valued-neural-networks-parallelizing-hamilton-product-on-gpus-cuda/48442)

<div class="topic-metadata">

**Author:** [@Palli](https://discourse.julialang.org/u/Palli)\
**Replies:** 7\
**Last updated:** [October 17, 2020, 11:34pm UTC](https://discourse.julialang.org/t/quaternion-and-up-to-sedenion-valued-neural-networks-parallelizing-hamilton-product-on-gpus-cuda/48442 "2020-10-17T23:34:29Z")

</div>

\[Edit: See my next comment on August 2020 Sedenion-networks, and there and here a bit on Octonion-networks. I changed the title yet again replacing Octonion with Sedenion as to not make it too long.\] The specific fou…

---

## [Assistance with Transformers.jl](https://discourse.julialang.org/t/assistance-with-transformers-jl/48483)

<div class="topic-metadata">

**Author:** [@Stantin\_Siebritz](https://discourse.julialang.org/u/Stantin_Siebritz)\
**Replies:** 0\
**Last updated:** [October 16, 2020, 9:43am UTC](https://discourse.julialang.org/t/assistance-with-transformers-jl/48483 "2020-10-16T09:43:33Z")

</div>

Hi I would like to create an implementation of the Transformer NLP architecture, more specifically I am trying to implement a BERT Summarizer. I have been using the Transformer.jl (v 0.1.7) package and have followed the…

---

## [ODE-LSTM implementation](https://discourse.julialang.org/t/ode-lstm-implementation/48437)

<div class="topic-metadata">

**Author:** [@lungd](https://discourse.julialang.org/u/lungd)\
**Replies:** 1\
**Last updated:** [October 15, 2020, 4:26pm UTC](https://discourse.julialang.org/t/ode-lstm-implementation/48437 "2020-10-15T16:26:42Z")

</div>

I am working on implementing the ODELSTM cell https://github.com/SciML/DiffEqFlux.jl/issues/422 The cell expects hidden states and a tuple of a feature vector and the elapsed time between the observations (or a 2D featu…

---

## [Reading tfrecord files](https://discourse.julialang.org/t/reading-tfrecord-files/26316)

<div class="topic-metadata">

**Author:** [@jbrea](https://discourse.julialang.org/u/jbrea)\
**Replies:** 1\
**Last updated:** [October 15, 2020, 3:57pm UTC](https://discourse.julialang.org/t/reading-tfrecord-files/26316 "2020-10-15T15:57:43Z")

</div>

I want to play a bit with the Youtube 8M dataset. Did somebody already work on reading tfrecord files? I tried julia\> using TensorFlow julia\> it = TensorFlow.io.TFRecord.RecordIterator("train0111.tfrecord") julia\> firs…

---

## [Could one chain chains?](https://discourse.julialang.org/t/could-one-chain-chains/30642)

<div class="topic-metadata">

**Author:** [@zht](https://discourse.julialang.org/u/zht)\
**Replies:** 3\
**Last updated:** [October 7, 2020, 7:47pm UTC](https://discourse.julialang.org/t/could-one-chain-chains/30642 "2020-10-07T19:47:10Z")

</div>

Is it okay for the layers used in a chain to be chains themselves? Such as Chain(Chain(A, B), C)? All the examples I have seen so far do not use Chain recurrently so I am thinking this may not be kosher. Does this mean t…

---

## [ANN: Announcing Unitary.jl, a differentiable parametrisation of the group of Unitary Matrices](https://discourse.julialang.org/t/ann-announcing-unitary-jl-a-differentiable-parametrisation-of-the-group-of-unitary-matrices/47917)

<div class="topic-metadata">

**Author:** [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Replies:** 7\
**Last updated:** [October 7, 2020, 6:11pm UTC](https://discourse.julialang.org/t/ann-announcing-unitary-jl-a-differentiable-parametrisation-of-the-group-of-unitary-matrices/47917 "2020-10-07T18:11:36Z")

</div>

For our SumProductTransformation networks https://arxiv.org/abs/2005.01297, we have created an invertible “Dense” transformations usual in neural networks. Our version features efficient inversion and efficient calculati…

---

## [Using the noise parameters sol.W.W from an SDE solution throws an error when taking gradients](https://discourse.julialang.org/t/using-the-noise-parameters-sol-w-w-from-an-sde-solution-throws-an-error-when-taking-gradients/47640)

<div class="topic-metadata">

**Author:** [@MikeKlocCZ](https://discourse.julialang.org/u/MikeKlocCZ)\
**Replies:** 3\
**Last updated:** [October 7, 2020, 2:13pm UTC](https://discourse.julialang.org/t/using-the-noise-parameters-sol-w-w-from-an-sde-solution-throws-an-error-when-taking-gradients/47640 "2020-10-07T14:13:03Z")

</div>

I need to work with individual Wiener increments from an SDE solution in my loss function. Whenever I try to get these values from sol.W.W command, the ERROR: type Nothing has no field W pops up when computing the gradie…

---

## [How can I get the loss functions in a multi-class classification Flux.jl neural network to work with Flux.onehotbatch?](https://discourse.julialang.org/t/how-can-i-get-the-loss-functions-in-a-multi-class-classification-flux-jl-neural-network-to-work-with-flux-onehotbatch/47767)

<div class="topic-metadata">

**Author:** [@qingyanz](https://discourse.julialang.org/u/qingyanz)\
**Replies:** 2\
**Last updated:** [October 5, 2020, 8:45pm UTC](https://discourse.julialang.org/t/how-can-i-get-the-loss-functions-in-a-multi-class-classification-flux-jl-neural-network-to-work-with-flux-onehotbatch/47767 "2020-10-05T20:45:50Z")

</div>

I am trying to set up a model to predict a 8-label classification problem. Suppose the labels are ycat = \["a", "b", "c", "d", "e", "f", "g"\] and I have a sample set of labels to train & test on: ysample = \["a", "a", "g",…

---

## [Flux vs pytorch cpu performance](https://discourse.julialang.org/t/flux-vs-pytorch-cpu-performance/42667)

<div class="topic-metadata">

**Author:** [@d1cker](https://discourse.julialang.org/u/d1cker)\
**Replies:** 59\
**Last updated:** [October 2, 2020, 3:39pm UTC](https://discourse.julialang.org/t/flux-vs-pytorch-cpu-performance/42667 "2020-10-02T15:39:07Z")

</div>

Hi, I’m trying to make my colleagues use Julia, but I got this performance issue with some simple code. At first I though that the problem is use the AD but now I think it’s with the feed forward (and because of that th…

---

## [Two questions on Flux](https://discourse.julialang.org/t/two-questions-on-flux/22307)

<div class="topic-metadata">

**Author:** [@BLI](https://discourse.julialang.org/u/BLI)\
**Replies:** 23\
**Last updated:** [October 2, 2020, 3:28pm UTC](https://discourse.julialang.org/t/two-questions-on-flux/22307 "2020-10-02T15:28:35Z")

</div>

I’ve watched videos in JuliaAcademy (I recommend them), and continued playing around with Flux. To test Flux, I’ve tried to re-create simple results from: Bishop, Chris M. (1994). “Neural networks and their application…

---

## [GeometricFlux: GCNConv but with different graphs](https://discourse.julialang.org/t/geometricflux-gcnconv-but-with-different-graphs/36518)

<div class="topic-metadata">

**Author:** [@rkurchin](https://discourse.julialang.org/u/rkurchin)\
**Replies:** 12\
**Last updated:** [October 2, 2020, 5:08am UTC](https://discourse.julialang.org/t/geometricflux-gcnconv-but-with-different-graphs/36518 "2020-10-02T05:08:24Z")

</div>

I’m working with GeometricFlux and want to adapt the GCNConv layer to allow for different adjacency matrices corresponding to each piece of training data. Since this now requires two separate “streams” of input data (the…

---

## [No changes with Flux NN regression training](https://discourse.julialang.org/t/no-changes-with-flux-nn-regression-training/47490)

<div class="topic-metadata">

**Author:** [@dmoored4](https://discourse.julialang.org/u/dmoored4)\
**Replies:** 2\
**Last updated:** [October 2, 2020, 3:03am UTC](https://discourse.julialang.org/t/no-changes-with-flux-nn-regression-training/47490 "2020-10-02T03:03:40Z")

</div>

Problem Description Hi all, I’m having trouble using Flux to learn a non-linear function of two independent variables, x1 and x2. Everything is running, but I have a feeling that the parameters aren’t actually being upda…

---

## [Flux model with categorical and continuous covariates/features](https://discourse.julialang.org/t/flux-model-with-categorical-and-continuous-covariates-features/47556)

<div class="topic-metadata">

**Author:** [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Replies:** 0\
**Last updated:** [September 30, 2020, 10:49pm UTC](https://discourse.julialang.org/t/flux-model-with-categorical-and-continuous-covariates-features/47556 "2020-09-30T22:49:53Z")

</div>

Using Flux, I would like to train a regression model with both continuous and categorical features. Since the categorical variable(s) may have a many levels, I would prefer to map these into a few new continuous variable…

---

## [LASSO.jl, GLMNet.jl - hyperparameters](https://discourse.julialang.org/t/lasso-jl-glmnet-jl-hyperparameters/28799)

<div class="topic-metadata">

**Author:** [@versipellis](https://discourse.julialang.org/u/versipellis)\
**Replies:** 6\
**Last updated:** [September 30, 2020, 9:35am UTC](https://discourse.julialang.org/t/lasso-jl-glmnet-jl-hyperparameters/28799 "2020-09-30T09:35:34Z")

</div>

I’m not seeing any documentation for such, but in LASSO.jl and GLMNet.jl, I’m looking to do the following: Specify a validation set to train hyperparameters on, rather than just using cross-validation. I know CV is bet…

---

## [Implementing Crystal Graph CNN in Julia](https://discourse.julialang.org/t/implementing-crystal-graph-cnn-in-julia/33169)

<div class="topic-metadata">

**Author:** [@rkurchin](https://discourse.julialang.org/u/rkurchin)\
**Replies:** 4\
**Last updated:** [September 29, 2020, 6:56pm UTC](https://discourse.julialang.org/t/implementing-crystal-graph-cnn-in-julia/33169 "2020-09-29T18:56:11Z")

</div>

Hi everyone! I’d like to implement Crystal Graph Convolutional Neural Networks (CGCNNs) in Julia, in particular using the GeometricFlux package. CGCNN’s are a method in computational materials science for representing cr…

---

## [Flux: Hard to use train! and DataLoader for minibatched NamedTuple dataset with GPU](https://discourse.julialang.org/t/flux-hard-to-use-train-and-dataloader-for-minibatched-namedtuple-dataset-with-gpu/47189)

<div class="topic-metadata">

**Author:** [@paalon](https://discourse.julialang.org/u/paalon)\
**Replies:** 2\
**Last updated:** [September 24, 2020, 8:54am UTC](https://discourse.julialang.org/t/flux-hard-to-use-train-and-dataloader-for-minibatched-namedtuple-dataset-with-gpu/47189 "2020-09-24T08:54:55Z")

</div>

I want to write the following neural networks machine learning code on GPU but I need to write complicated and dirty something within dataloader’s for loop to pass data to train!: # This GPU based program works but very…

---

## [Is there an implementation of the attention mechanism in Flux.jl?](https://discourse.julialang.org/t/is-there-an-implementation-of-the-attention-mechanism-in-flux-jl/26959)

<div class="topic-metadata">

**Author:** [@Azamat](https://discourse.julialang.org/u/Azamat)\
**Replies:** 5\
**Last updated:** [September 23, 2020, 6:49am UTC](https://discourse.julialang.org/t/is-there-an-implementation-of-the-attention-mechanism-in-flux-jl/26959 "2020-09-23T06:49:40Z")

</div>

Specifically, I’m interested in the attention mechanism described in Listen, Attend and Spell work (where it’s referred to as AttentionContext), but more generally any other kind of attention’s implementation would be qu…

---

## [How to implement LSTM base meta relearning by Flux?](https://discourse.julialang.org/t/how-to-implement-lstm-base-meta-relearning-by-flux/47164)

<div class="topic-metadata">

**Author:** [@nuclear718](https://discourse.julialang.org/u/nuclear718)\
**Replies:** 0\
**Last updated:** [September 23, 2020, 11:38pm UTC](https://discourse.julialang.org/t/how-to-implement-lstm-base-meta-relearning-by-flux/47164 "2020-09-23T23:38:32Z")

</div>

In LSTM meta learning approach, we have a base leaner and update base learner’s parameters by another learner (single cell LSTM). The base learner’s parameters is LSTM’s cell state and will input LSTM one by one, the ou…

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=38)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=40)