# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=23

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 24

---

## [Pullback for both Flux model's inputs and parameters](https://discourse.julialang.org/t/pullback-for-both-flux-models-inputs-and-parameters/83316)

<div class="topic-metadata">

**Author:** [@luciano-drozda](https://discourse.julialang.org/u/luciano-drozda)\
**Replies:** 9\
**Last updated:** [June 26, 2022, 9:08am UTC](https://discourse.julialang.org/t/pullback-for-both-flux-models-inputs-and-parameters/83316 "2022-06-26T09:08:08Z")

</div>

How can I obtain pullback for a Flux model not only w.r.t. its params but also its inputs ? outputs, back = Zygote.pullback( () -\> model(inputs), Flux.params(model) ) returns the adjoint (i.e., back) for the model para…

---

## [Right way of applying \`inverse\_transform\`](https://discourse.julialang.org/t/right-way-of-applying-inverse-transform/83124)

<div class="topic-metadata">

**Author:** [@ctrebbau](https://discourse.julialang.org/u/ctrebbau)\
**Replies:** 3\
**Last updated:** [June 25, 2022, 6:27pm UTC](https://discourse.julialang.org/t/right-way-of-applying-inverse-transform/83124 "2022-06-25T18:27:18Z")

</div>

Hi, I have a setup like this: dftrain, dftest = partition(df, 0.7, shuffle=true, rng=123) datapipe = ContinuousEncoder() |\> Standardizer() datatrans\_mach = machine(datapipe, dftrain) |\> fit! normalized\_train = MLJ.tran…

---

## [(Flux) certain arbitrary model sizes and random seeds make gradients exactly zero](https://discourse.julialang.org/t/flux-certain-arbitrary-model-sizes-and-random-seeds-make-gradients-exactly-zero/83277)

<div class="topic-metadata">

**Author:** [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Replies:** 3\
**Last updated:** [June 25, 2022, 1:25pm UTC](https://discourse.julialang.org/t/flux-certain-arbitrary-model-sizes-and-random-seeds-make-gradients-exactly-zero/83277 "2022-06-25T13:25:59Z")

</div>

I’m building a small neural network to predict a one-dimensional output from 5 input features. Simplified reproducing example: function build\_model(mid\_layer\_size, seed) Random.seed!(seed) return Chain( …

---

## [Simultaneous training of multiple neural networks with Lux](https://discourse.julialang.org/t/simultaneous-training-of-multiple-neural-networks-with-lux/83255)

<div class="topic-metadata">

**Author:** [@bkuwahara](https://discourse.julialang.org/u/bkuwahara)\
**Replies:** 5\
**Last updated:** [June 25, 2022, 7:51am UTC](https://discourse.julialang.org/t/simultaneous-training-of-multiple-neural-networks-with-lux/83255 "2022-06-25T07:51:35Z")

</div>

Suppose I’m trying to train a neural ODE composed of two distinct neural networks, for instance because I want to preserve information about which model states influence which other states. This worked easily with the o…

---

## [Package for clustering data points](https://discourse.julialang.org/t/package-for-clustering-data-points/83154)

<div class="topic-metadata">

**Author:** [@math\_opt](https://discourse.julialang.org/u/math_opt)\
**Replies:** 8\
**Last updated:** [June 23, 2022, 5:22pm UTC](https://discourse.julialang.org/t/package-for-clustering-data-points/83154 "2022-06-23T17:22:55Z")

</div>

Any package recommendations for clustering data points? I see clustering.jl exist, however, I don’t see any recent activity on it. Is it still being maintained? Any other package that might be actively maintained?

---

## [Using a variable graph structure with NeuralODE and GCNConv](https://discourse.julialang.org/t/using-a-variable-graph-structure-with-neuralode-and-gcnconv/78881)

<div class="topic-metadata">

**Author:** [@Zapientus](https://discourse.julialang.org/u/Zapientus)\
**Replies:** 3\
**Last updated:** [June 22, 2022, 4:56am UTC](https://discourse.julialang.org/t/using-a-variable-graph-structure-with-neuralode-and-gcnconv/78881 "2022-06-22T04:56:20Z")

</div>

Hello everyone, I have recently been working with Graph Neural Ordinary Differential Equations for structure predictions of proteins. This has been working quite well as long as the graph structure is fixed and given to…

---

## [How do I can I selectively inspect and use learned parameters in an MLJ pipeline?](https://discourse.julialang.org/t/how-do-i-can-i-selectively-inspect-and-use-learned-parameters-in-an-mlj-pipeline/83097)

<div class="topic-metadata">

**Author:** [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Replies:** 1\
**Last updated:** [June 20, 2022, 10:57pm UTC](https://discourse.julialang.org/t/how-do-i-can-i-selectively-inspect-and-use-learned-parameters-in-an-mlj-pipeline/83097 "2022-06-20T22:57:25Z")

</div>

From a question on slack: If I have a pipeline like this @pipeline ContinuousEncoder(drop\_last=true) Standardizer() can I specify to inverse\_transform the standadizer part so that I won’t have to break up the pipeline? …

---

## [How to find jacobian of learnt neural network model](https://discourse.julialang.org/t/how-to-find-jacobian-of-learnt-neural-network-model/83010)

<div class="topic-metadata">

**Author:** [@umerzakheer](https://discourse.julialang.org/u/umerzakheer)\
**Replies:** 1\
**Last updated:** [June 19, 2022, 4:09pm UTC](https://discourse.julialang.org/t/how-to-find-jacobian-of-learnt-neural-network-model/83010 "2022-06-19T16:09:51Z")

</div>

I modelled a neural network with a vector as input and a vector as output (multi input multi output NN model). Now I want to find jacobian of the learnt function as: Would anyone please guide me how to do this in Jul…

---

## [Is it possible to train sparse neural networks in GPU?](https://discourse.julialang.org/t/is-it-possible-to-train-sparse-neural-networks-in-gpu/82680)

<div class="topic-metadata">

**Author:** [@paulobruno](https://discourse.julialang.org/u/paulobruno)\
**Replies:** 3\
**Last updated:** [June 17, 2022, 2:57am UTC](https://discourse.julialang.org/t/is-it-possible-to-train-sparse-neural-networks-in-gpu/82680 "2022-06-17T02:57:34Z")

</div>

I would like to train a sparse neural network using Flux. I can train it in CPU, but I would like to do it using GPU, profiting from CuSparseMatrixCSC. However, when I try to run the code below I get ERROR: LoadError: Th…

---

## [Re-using layers in Flux.jl: how to train a multi-layer model sharing a common LSTM layer and separate dense layers?](https://discourse.julialang.org/t/re-using-layers-in-flux-jl-how-to-train-a-multi-layer-model-sharing-a-common-lstm-layer-and-separate-dense-layers/82705)

<div class="topic-metadata">

**Author:** [@rio](https://discourse.julialang.org/u/rio)\
**Replies:** 1\
**Last updated:** [June 15, 2022, 2:16am UTC](https://discourse.julialang.org/t/re-using-layers-in-flux-jl-how-to-train-a-multi-layer-model-sharing-a-common-lstm-layer-and-separate-dense-layers/82705 "2022-06-15T02:16:19Z")

</div>

I would like to train a model that consists of two “submodels”, say model1 and model2. The two models share a common LSTM layer and are followed by two separate dense layers. The final output is the average of the two mo…

---

## [Way to write the integral with ModelingToolkit](https://discourse.julialang.org/t/way-to-write-the-integral-with-modelingtoolkit/82347)

<div class="topic-metadata">

**Author:** [@kaido975](https://discourse.julialang.org/u/kaido975)\
**Replies:** 7\
**Last updated:** [June 13, 2022, 3:22pm UTC](https://discourse.julialang.org/t/way-to-write-the-integral-with-modelingtoolkit/82347 "2022-06-13T15:22:45Z")

</div>

Is there a way to write the following integral with ModelingToolkit.jl, is binomial expansion the only option here? \\int\_{0}^{t} (t - x)^{\\lambda} u(x) d x \\quad \\text{, } \\lambda \< 0 \\quad \\text{, for } t\> 0 I want t…

---

## [A minimal CUDA-compatible and differentiable GP implementation](https://discourse.julialang.org/t/a-minimal-cuda-compatible-and-differentiable-gp-implementation/80099)

<div class="topic-metadata">

**Author:** [@Red-Portal](https://discourse.julialang.org/u/Red-Portal)\
**Replies:** 11\
**Last updated:** [June 9, 2022, 3:22pm UTC](https://discourse.julialang.org/t/a-minimal-cuda-compatible-and-differentiable-gp-implementation/80099 "2022-06-09T15:22:07Z")

</div>

Hi folks, I was really frustrated last week because it seems there isn’t a way to implement Gaussian processes with both CUDA acceleration and differentiation. Because of this, I had to roll my own CUDA kernels that did…

---

## [Dispatch on the last layer type in Flux](https://discourse.julialang.org/t/dispatch-on-the-last-layer-type-in-flux/82481)

<div class="topic-metadata">

**Author:** [@DoktorMike](https://discourse.julialang.org/u/DoktorMike)\
**Replies:** 3\
**Last updated:** [June 9, 2022, 11:07am UTC](https://discourse.julialang.org/t/dispatch-on-the-last-layer-type-in-flux/82481 "2022-06-09T11:07:01Z")

</div>

I’m making an implementation of a Evidential Deep Learning model in Flux and I would like to have a function predict(model, x) where x is input data. The model is a Chain which consists possibly of many layers. I want th…

---

## [How to retrieve gradient value in custom Flux training loop?](https://discourse.julialang.org/t/how-to-retrieve-gradient-value-in-custom-flux-training-loop/82435)

<div class="topic-metadata">

**Author:** [@BatyLeo](https://discourse.julialang.org/u/BatyLeo)\
**Replies:** 3\
**Last updated:** [June 8, 2022, 2:16pm UTC](https://discourse.julialang.org/t/how-to-retrieve-gradient-value-in-custom-flux-training-loop/82435 "2022-06-08T14:16:42Z")

</div>

Hello, I have a few questions about custom training loops in Flux. According to the documentation, I should write something like that : function my\_custom\_train!(loss, ps, data, opt) ps = Params(ps) for d in data …

---

## [Feature selection+classification pipeline](https://discourse.julialang.org/t/feature-selection-classification-pipeline/81365)

<div class="topic-metadata">

**Author:** [@Michele\_Avanzo](https://discourse.julialang.org/u/Michele_Avanzo)\
**Replies:** 3\
**Last updated:** [June 7, 2022, 11:04am UTC](https://discourse.julialang.org/t/feature-selection-classification-pipeline/81365 "2022-06-07T11:04:16Z")

</div>

Hi, I am relatively new to Julia. I find it very difficult to build a pipeline that selects optimal subset of features from a large number of features and then training a ML classifier. Currently, I first apply ShapML…

---

## [How do I regularise MLJFlux models?](https://discourse.julialang.org/t/how-do-i-regularise-mljflux-models/82349)

<div class="topic-metadata">

**Author:** [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Replies:** 1\
**Last updated:** [June 6, 2022, 11:18pm UTC](https://discourse.julialang.org/t/how-do-i-regularise-mljflux-models/82349 "2022-06-06T23:18:00Z")

</div>

Reposting this question from a Slack channel.

---

## [How to use \`HyperOpt.jl\` to optimise a vector of parameters](https://discourse.julialang.org/t/how-to-use-hyperopt-jl-to-optimise-a-vector-of-parameters/82232)

<div class="topic-metadata">

**Author:** [@this\_josh](https://discourse.julialang.org/u/this_josh)\
**Replies:** 10\
**Last updated:** [June 6, 2022, 3:26pm UTC](https://discourse.julialang.org/t/how-to-use-hyperopt-jl-to-optimise-a-vector-of-parameters/82232 "2022-06-06T15:26:25Z")

</div>

I’d like to use HyperOpt.jl to optimise a vector of parameters, but I’m having trouble using following the libraries documentation (a readme) As far as I can tell this should be sufficient using Hyperopt n = 10 d1 = ra…

---

## [PINNs for integro-differential equations with noise](https://discourse.julialang.org/t/pinns-for-integro-differential-equations-with-noise/82165)

<div class="topic-metadata">

**Author:** [@kaido975](https://discourse.julialang.org/u/kaido975)\
**Replies:** 1\
**Last updated:** [June 3, 2022, 10:34am UTC](https://discourse.julialang.org/t/pinns-for-integro-differential-equations-with-noise/82165 "2022-06-03T10:34:54Z")

</div>

Is there a way to solve an IDE with a noise term such as: \\frac{\\partial}{\\partial t} u(t) + u(t) + \\int\_{0}^{t} u(x) d x + w(t) = 0 with the help of NeuralPDE.jl, where w(t) is the noise term.

---

## [Does anybody wanna help me make a Hypernetworks package?](https://discourse.julialang.org/t/does-anybody-wanna-help-me-make-a-hypernetworks-package/82093)

<div class="topic-metadata">

**Author:** [@afishy](https://discourse.julialang.org/u/afishy)\
**Replies:** 0\
**Last updated:** [June 1, 2022, 11:06pm UTC](https://discourse.julialang.org/t/does-anybody-wanna-help-me-make-a-hypernetworks-package/82093 "2022-06-01T23:06:31Z")

</div>

Hypernetworks are an increasingly popular tool for several applications, including continual learning, generative modeling etc. Naively, I’d think it would be relatively easy to create a simple interface to create an arb…

---

## [Help me pin this bug in \`Flux\`…](https://discourse.julialang.org/t/help-me-pin-this-bug-in-flux/81901)

<div class="topic-metadata">

**Author:** [@Jollywatt](https://discourse.julialang.org/u/Jollywatt)\
**Replies:** 2\
**Last updated:** [May 31, 2022, 10:34pm UTC](https://discourse.julialang.org/t/help-me-pin-this-bug-in-flux/81901 "2022-05-31T22:34:15Z")

</div>

On Julia 1.6.5 and 1.7.2, the following code gives a method error: using Flux # v0.13.0 x = Flux.onehotbatch(\[1, 2\], 0:9) # x = float.(x) ### model = zeros() Flux.gradient(Flux.params(model)) do y = x y = reshape(…

---

## [\_generic\_matmatmul error in sciml\_train](https://discourse.julialang.org/t/generic-matmatmul-error-in-sciml-train/81976)

<div class="topic-metadata">

**Author:** [@linkz](https://discourse.julialang.org/u/linkz)\
**Replies:** 3\
**Last updated:** [May 31, 2022, 1:20pm UTC](https://discourse.julialang.org/t/generic-matmatmul-error-in-sciml-train/81976 "2022-05-31T13:20:35Z")

</div>

Hello all, I am trying to train a neural ODE model using DiffEqFlux.sciml\_train. I have reasons to believe that if I use the outer vector product of the current state u as the neural net input, I might get better perform…

---

## [Converting PyTorch to Flux while keeping performance](https://discourse.julialang.org/t/converting-pytorch-to-flux-while-keeping-performance/81877)

<div class="topic-metadata">

**Author:** [@Jarartur](https://discourse.julialang.org/u/Jarartur)\
**Replies:** 7\
**Last updated:** [May 29, 2022, 9:58pm UTC](https://discourse.julialang.org/t/converting-pytorch-to-flux-while-keeping-performance/81877 "2022-05-29T21:58:01Z")

</div>

Hey, I am currently converting some models from pytorch to flux and can’t figure out why I am having performance issues. My function in PyTorch: def stiffness\_reg(x, idx): x\_nn = knn\_gather(x, idx) # basically ind…

---

## [MethodError: objects of type Float64 are not callable](https://discourse.julialang.org/t/methoderror-objects-of-type-float64-are-not-callable/81876)

<div class="topic-metadata">

**Author:** [@Hupdup](https://discourse.julialang.org/u/Hupdup)\
**Replies:** 2\
**Last updated:** [May 29, 2022, 6:04pm UTC](https://discourse.julialang.org/t/methoderror-objects-of-type-float64-are-not-callable/81876 "2022-05-29T18:04:11Z")

</div>

Hi, when I want to update the weights of my main\_network, I always get this ERROR: MethodError: objects of type Float64 are not callable The Output is: -24.429102722601773 Float32\[-0.019576067, 0.08515469, -0.026…

---

## [Pipeline for different columns](https://discourse.julialang.org/t/pipeline-for-different-columns/81494)

<div class="topic-metadata">

**Author:** [@jfcal](https://discourse.julialang.org/u/jfcal)\
**Replies:** 11\
**Last updated:** [May 29, 2022, 9:08am UTC](https://discourse.julialang.org/t/pipeline-for-different-columns/81494 "2022-05-29T09:08:23Z")

</div>

Hi everyone, I have mixed type data and want to apply different functions to different types in a pipeline. Like imputation with mean for continuous and knn imputation for categorical and so on… Is it possible with MLJ…

---

## [Archival of discontinued packages in JuliaML](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375)

<div class="topic-metadata">

**Author:** [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Replies:** 11\
**Last updated:** [May 26, 2022, 9:27pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375 "2022-05-26T21:27:08Z")

</div>

I am cleaning the house in JuliaML and will proceed with a sequence of archival operations to facilitate the lives of newcomers who come to Julia and are completely lost in the middle of packages that don’t even load any…

---

## [Need Help to implement my loss function](https://discourse.julialang.org/t/need-help-to-implement-my-loss-function/81724)

<div class="topic-metadata">

**Author:** [@Hupdup](https://discourse.julialang.org/u/Hupdup)\
**Replies:** 0\
**Last updated:** [May 26, 2022, 2:13pm UTC](https://discourse.julialang.org/t/need-help-to-implement-my-loss-function/81724 "2022-05-26T14:13:40Z")

</div>

Hi I want to implement a loss function for the weight-update of the neural network in my Reinforcement learning (DQN) project. The Q Value of the next state should be estimated by my target network (yellow mark) wheras…

---

## [MLJ.fit! complain "UndefVarError: level not defined" in a sysimage](https://discourse.julialang.org/t/mlj-fit-complain-undefvarerror-level-not-defined-in-a-sysimage/81499)

<div class="topic-metadata">

**Author:** [@tiZ](https://discourse.julialang.org/u/tiZ)\
**Replies:** 1\
**Last updated:** [May 23, 2022, 9:48am UTC](https://discourse.julialang.org/t/mlj-fit-complain-undefvarerror-level-not-defined-in-a-sysimage/81499 "2022-05-23T09:48:45Z")

</div>

Hello, I try to fit a simple logistic regression in Julia, with a sysimage, which loaded these packages : “OhMyREPL” “Plots” “GLMakie” “DataFrames” “RCall” “PyCall” “CSV” “Flux” “Random” “TensorBoardLogger” Here is m…

---

## [Superexpressive Activation function (paper) and "Three Hidden Layers Are Enough" paper](https://discourse.julialang.org/t/superexpressive-activation-function-paper-and-three-hidden-layers-are-enough-paper/81336)

<div class="topic-metadata">

**Author:** [@Palli](https://discourse.julialang.org/u/Palli)\
**Replies:** 1\
**Last updated:** [May 22, 2022, 1:25am UTC](https://discourse.julialang.org/t/superexpressive-activation-function-paper-and-three-hidden-layers-are-enough-paper/81336 "2022-05-22T01:25:15Z")

</div>

I came across Superexpressive Activation functions, the elementary kind on arxiv, the paper on it is also at: A. Elementary Superexpressive Activations http://proceedings.mlr.press/v139/yarotsky21a/yarotsky21a.pdf we…

---

## [ReinforcementLearning.jl vs POMDPs.jl](https://discourse.julialang.org/t/reinforcementlearning-jl-vs-pomdps-jl/81355)

<div class="topic-metadata">

**Author:** [@filchristou](https://discourse.julialang.org/u/filchristou)\
**Replies:** 0\
**Last updated:** [May 20, 2022, 10:17am UTC](https://discourse.julialang.org/t/reinforcementlearning-jl-vs-pomdps-jl/81355 "2022-05-20T10:17:42Z")

</div>

Hello. Some weeks ago, there was a live session regarding the ReinforcementLearning.jl package, and some of the questions were to identify differences, overlaps, and cooperation opportunities with the JuliaPOMDP organiza…

---

## [Flux.destructure a Flux.Chain that takes an argument?](https://discourse.julialang.org/t/flux-destructure-a-flux-chain-that-takes-an-argument/81067)

<div class="topic-metadata">

**Author:** [@linkz](https://discourse.julialang.org/u/linkz)\
**Replies:** 6\
**Last updated:** [May 15, 2022, 9:47pm UTC](https://discourse.julialang.org/t/flux-destructure-a-flux-chain-that-takes-an-argument/81067 "2022-05-15T21:47:48Z")

</div>

Hello all, I have a use case where I need to use some custom layers in a Flux.Chain that depend on the initial vector passed to the chain (which changes). I can construct the chain easily as per below (with some dummy la…

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=22)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=24)