# Machine Learning

**URL:** https://discourse.julialang.org/c/domain/ml/24.md?page=10

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 11

---

## [\[English help\] An imputer that works with any supervised model, a "GeneralImputer" or a "UniversalImputer" (or other?)](https://discourse.julialang.org/t/english-help-an-imputer-that-works-with-any-supervised-model-a-generalimputer-or-a-universalimputer-or-other/109187)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 0\
**Last updated:** [January 24, 2024, 9:09am UTC](https://discourse.julialang.org/t/english-help-an-imputer-that-works-with-any-supervised-model-a-generalimputer-or-a-universalimputer-or-other/109187 "2024-01-24T09:09:48Z")

</div>

I am standardizing the names of the models on my Beta Machine Learning Toolkits (BetaML)… concerning the Imputers (of missing values), I have some that use a specific algorithm, e.g. Random Forest, or Gaussian Mixture mo…

---

## [Inverse problem with NeuralPDE and GPU support](https://discourse.julialang.org/t/inverse-problem-with-neuralpde-and-gpu-support/109051)

<div class="topic-metadata">

**Author:** [@Rene\_Schenkendorf](https://discourse.julialang.org/u/Rene_Schenkendorf)\
**Replies:** 5\
**Last updated:** [January 23, 2024, 8:29am UTC](https://discourse.julialang.org/t/inverse-problem-with-neuralpde-and-gpu-support/109051 "2024-01-23T08:29:00Z")

</div>

Hello everyone. I am new to Julia (at least that’s how it feels from time to time). Using the GPU according to tutorial https://docs.sciml.ai/NeuralPDE/stable/tutorials/gpu/ works; the inverse problem (first-principles m…

---

## [Issue with Zygote over ForwardDiff.derivative](https://discourse.julialang.org/t/issue-with-zygote-over-forwarddiff-derivative/70824)

<div class="topic-metadata">

**Author:** [@jlmaccal](https://discourse.julialang.org/u/jlmaccal)\
**Replies:** 10\
**Last updated:** [January 21, 2024, 4:18pm UTC](https://discourse.julialang.org/t/issue-with-zygote-over-forwarddiff-derivative/70824 "2024-01-21T16:18:25Z")

</div>

I’m having some trouble getting Zygote over ForwardDiff.derivative to work. I’m going to refer to this prior post. The following code used to fail with ERROR: setindex! not defined for ForwardDiff.Partials{1,Float64}. T…

---

## [Maliar, Maliar, and Winant using Flux.jl (I just want to write a custom objective)](https://discourse.julialang.org/t/maliar-maliar-and-winant-using-flux-jl-i-just-want-to-write-a-custom-objective/107993)

<div class="topic-metadata">

**Author:** [@JHall](https://discourse.julialang.org/u/JHall)\
**Replies:** 8\
**Last updated:** [January 19, 2024, 4:02pm UTC](https://discourse.julialang.org/t/maliar-maliar-and-winant-using-flux-jl-i-just-want-to-write-a-custom-objective/107993 "2024-01-19T16:02:24Z")

</div>

Hi all, I am trying to translate the code that you can find here from Python into Julia. Everything works swimmingly until I get to the end of the code where I have to do the training section. I am using Flux.jl, and …

---

## [Microsoft Phi model](https://discourse.julialang.org/t/microsoft-phi-model/108895)

<div class="topic-metadata">

**Author:** [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Replies:** 0\
**Last updated:** [January 17, 2024, 8:42am UTC](https://discourse.julialang.org/t/microsoft-phi-model/108895 "2024-01-17T08:42:50Z")

</div>

Is anyone up to porting (adding) microsoft phi model to Transformers.jl? I would like to use it for experiments (I am interested to make it a bert-like model). I have been using so far llama2-7b, but phi models seems sma…

---

## [Is there any julia package for multivariate polynomial regression?](https://discourse.julialang.org/t/is-there-any-julia-package-for-multivariate-polynomial-regression/108785)

<div class="topic-metadata">

**Author:** [@WuSiren](https://discourse.julialang.org/u/WuSiren)\
**Replies:** 18\
**Last updated:** [January 16, 2024, 2:34am UTC](https://discourse.julialang.org/t/is-there-any-julia-package-for-multivariate-polynomial-regression/108785 "2024-01-16T02:34:33Z")

</div>

For example, I want to build a multivariate polynomial model for data \\{x\_{1i},x\_{2i},\\cdots,x\_{mi},y\_i\\}\_{i=1}^N with given degree n. Thanks for your attention!

---

## [Learning rate decay in callback function](https://discourse.julialang.org/t/learning-rate-decay-in-callback-function/107859)

<div class="topic-metadata">

**Author:** [@KianH](https://discourse.julialang.org/u/KianH)\
**Replies:** 3\
**Last updated:** [January 11, 2024, 4:10pm UTC](https://discourse.julialang.org/t/learning-rate-decay-in-callback-function/107859 "2024-01-11T16:10:06Z")

</div>

I was wondering if there is a way to access the learning rate through a callback function when using Lux.jl and Optimization.jl packages. For instance in the line below: Optimization.solve(optprob, ADAM(1e-3), callback …

---

## [DiffEqGPU.jl with CUDA: Error computing gradients through SDE solver](https://discourse.julialang.org/t/diffeqgpu-jl-with-cuda-error-computing-gradients-through-sde-solver/108545)

<div class="topic-metadata">

**Author:** [@MatthieuDarcy](https://discourse.julialang.org/u/MatthieuDarcy)\
**Replies:** 12\
**Last updated:** [January 10, 2024, 7:32pm UTC](https://discourse.julialang.org/t/diffeqgpu-jl-with-cuda-error-computing-gradients-through-sde-solver/108545 "2024-01-10T19:32:12Z")

</div>

Hi Julia community! I’m trying to fit a custom, one-layer NN to an SDE by computing an MSE loss. Here is my code defining the model and computing the loss (noise and sample\_features are cuda matrices defined globally) d…

---

## [Differentiation (Zygote, but the issue is likely in ChainRules) return different types with \`Diagonal\`](https://discourse.julialang.org/t/differentiation-zygote-but-the-issue-is-likely-in-chainrules-return-different-types-with-diagonal/108608)

<div class="topic-metadata">

**Author:** [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Replies:** 2\
**Last updated:** [January 10, 2024, 7:09pm UTC](https://discourse.julialang.org/t/differentiation-zygote-but-the-issue-is-likely-in-chainrules-return-different-types-with-diagonal/108608 "2024-01-10T19:09:24Z")

</div>

Dear All, We are hacking a bit AbstractGP.jl and we have found an inconsistency in the returned type of Diagonal, which causes Zygote to complain. An MWE adapted from GPs is using LinearAlgebra, Zygote K = \[1.0009999…

---

## [Physics-enhanced deep surrogates for partial differential equations](https://discourse.julialang.org/t/physics-enhanced-deep-surrogates-for-partial-differential-equations/108596)

<div class="topic-metadata">

**Author:** [@Perrin\_Meyer](https://discourse.julialang.org/u/Perrin_Meyer)\
**Replies:** 0\
**Last updated:** [January 9, 2024, 11:03pm UTC](https://discourse.julialang.org/t/physics-enhanced-deep-surrogates-for-partial-differential-equations/108596 "2024-01-09T23:03:50Z")

</div>

Very Interesting new Nature ML paper from some of the core Julia developers (using Julia as well). (It’s nice the journal article is open access, it’s worth a read).

---

## [How to read model weights using FluxTraining (stateaccess issues)?](https://discourse.julialang.org/t/how-to-read-model-weights-using-fluxtraining-stateaccess-issues/106409)

<div class="topic-metadata">

**Author:** [@neuromancer](https://discourse.julialang.org/u/neuromancer)\
**Replies:** 3\
**Last updated:** [January 9, 2024, 9:35pm UTC](https://discourse.julialang.org/t/how-to-read-model-weights-using-fluxtraining-stateaccess-issues/106409 "2024-01-09T21:35:21Z")

</div>

Summary I am looking for some help with implementing a custom validation loop using FluxTraining. Briefly, I am using Flux to optimize a matrix to satisfy a data-driven cost-function. Since the cost function depends on t…

---

## [Does Lux work with FluxTraining?](https://discourse.julialang.org/t/does-lux-work-with-fluxtraining/108591)

<div class="topic-metadata">

**Author:** [@neuromancer](https://discourse.julialang.org/u/neuromancer)\
**Replies:** 0\
**Last updated:** [January 9, 2024, 9:33pm UTC](https://discourse.julialang.org/t/does-lux-work-with-fluxtraining/108591 "2024-01-09T21:33:38Z")

</div>

I’m thinking about switching from Flux to Lux, since I really need a WeightNorm layer and I am struggling to write one for myself. I am heavily entrenched into using FluxTraining. Does anyone know if Lux works with FluxT…

---

## [XGBoostClassifier on bigger than RAM database](https://discourse.julialang.org/t/xgboostclassifier-on-bigger-than-ram-database/108358)

<div class="topic-metadata">

**Author:** [@Paulo\_Refosco](https://discourse.julialang.org/u/Paulo_Refosco)\
**Replies:** 2\
**Last updated:** [January 8, 2024, 12:56pm UTC](https://discourse.julialang.org/t/xgboostclassifier-on-bigger-than-ram-database/108358 "2024-01-08T12:56:13Z")

</div>

Hello, I am trying to run XGBoostClassifier on a dataset that doesn’t fit into RAM, so getting OutOfMemoryError(). I’m not soo much proeficient on writing codes but I think solutions could range from mini-batches and/or…

---

## [ReversedDiff/Zygote with SDE DifferentialEquations fails to compute gradients after a certain number of parameters](https://discourse.julialang.org/t/reverseddiff-zygote-with-sde-differentialequations-fails-to-compute-gradients-after-a-certain-number-of-parameters/108451)

<div class="topic-metadata">

**Author:** [@MatthieuDarcy](https://discourse.julialang.org/u/MatthieuDarcy)\
**Replies:** 2\
**Last updated:** [January 7, 2024, 3:37am UTC](https://discourse.julialang.org/t/reverseddiff-zygote-with-sde-differentialequations-fails-to-compute-gradients-after-a-certain-number-of-parameters/108451 "2024-01-07T03:37:10Z")

</div>

Hello, I’m trying to solve a parameter estimation problem for an SDE. The parameters are a small, one-layer neural network (technically, it’s a random feature model). My parameters are contained in one matrix of size (4…

---

## [Lux Loss Not Decreasing](https://discourse.julialang.org/t/lux-loss-not-decreasing/108277)

<div class="topic-metadata">

**Author:** [@Dale\_James\_Black](https://discourse.julialang.org/u/Dale_James_Black)\
**Replies:** 1\
**Last updated:** [January 4, 2024, 12:18am UTC](https://discourse.julialang.org/t/lux-loss-not-decreasing/108277 "2024-01-04T00:18:42Z")

</div>

I am currently developing a comprehensive tutorial for my lab and the broader Julia community on training a Lux model for image segmentation. This tutorial is contained within a Pluto notebook that encompasses the entire…

---

## [Understanding and Overcoming Zygote's Functional Limitations for distributed](https://discourse.julialang.org/t/understanding-and-overcoming-zygotes-functional-limitations-for-distributed/108143)

<div class="topic-metadata">

**Author:** [@josemanuel22](https://discourse.julialang.org/u/josemanuel22)\
**Replies:** 5\
**Last updated:** [December 29, 2023, 8:57pm UTC](https://discourse.julialang.org/t/understanding-and-overcoming-zygotes-functional-limitations-for-distributed/108143 "2023-12-29T20:57:00Z")

</div>

Hello everyone, I am trying to parallelize a cost function. I am using Distributed and Zygote as an autodifferentiator for this. The example is as follows function sliced\_invariant\_statistical\_loss\_distributed(nn\_model,…

---

## [I am building a simple to use autoencoder model.. anyone interested?](https://discourse.julialang.org/t/i-am-building-a-simple-to-use-autoencoder-model-anyone-interested/108146)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 4\
**Last updated:** [December 29, 2023, 3:33pm UTC](https://discourse.julialang.org/t/i-am-building-a-simple-to-use-autoencoder-model-anyone-interested/108146 "2023-12-29T15:33:44Z")

</div>

Hello, I saw that all autoencoders presented in the various blogs require to actually implement the autoencoder from a DL framework. I am implementing instead one where the user just creates the model m =AutoEncoder(out…

---

## [Flux loss with contribution gradient is slow](https://discourse.julialang.org/t/flux-loss-with-contribution-gradient-is-slow/107956)

<div class="topic-metadata">

**Author:** [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)\
**Replies:** 5\
**Last updated:** [December 27, 2023, 5:20am UTC](https://discourse.julialang.org/t/flux-loss-with-contribution-gradient-is-slow/107956 "2023-12-27T05:20:29Z")

</div>

I am trying to replicate some computations on solving PDEs with neural networks (the committor problem, see \[1802.10275\] Solving for high dimensional committor functions using artificial neural networks). This involves …

---

## [Higher order derivatives/ automatic differentiation](https://discourse.julialang.org/t/higher-order-derivatives-automatic-differentiation/107936)

<div class="topic-metadata">

**Author:** [@ayushinav](https://discourse.julialang.org/u/ayushinav)\
**Replies:** 5\
**Last updated:** [December 26, 2023, 10:05am UTC](https://discourse.julialang.org/t/higher-order-derivatives-automatic-differentiation/107936 "2023-12-26T10:05:01Z")

</div>

Tldr: Getting gradients for training PINNs for higher order ODEs/PDEs. There’s a certain Diff Eq I want to solve using PINNs. Let’s say a damped harmonic oscillator: u''(t)+ \\mu u'(t)+ k u(t)= 0 with a Dirichlet BC (u…

---

## [PINN using Flux](https://discourse.julialang.org/t/pinn-using-flux/107782)

<div class="topic-metadata">

**Author:** [@andres](https://discourse.julialang.org/u/andres)\
**Replies:** 4\
**Last updated:** [December 24, 2023, 4:32pm UTC](https://discourse.julialang.org/t/pinn-using-flux/107782 "2023-12-24T16:32:05Z")

</div>

Hello, folks I’m trying to create a Physics-Informed Neural Network (PINN) to solve the harmonic oscillator using Flux. The idea was to make a Julia version based on the following link, whose implementation is in Pytho…

---

## [Issues with computing gradient with ForwardDiff.jl (Any fixes other than ND?)](https://discourse.julialang.org/t/issues-with-computing-gradient-with-forwarddiff-jl-any-fixes-other-than-nd/107989)

<div class="topic-metadata">

**Author:** [@ayushinav](https://discourse.julialang.org/u/ayushinav)\
**Replies:** 2\
**Last updated:** [December 24, 2023, 4:30am UTC](https://discourse.julialang.org/t/issues-with-computing-gradient-with-forwarddiff-jl-any-fixes-other-than-nd/107989 "2023-12-24T04:30:06Z")

</div>

When training a model using the boundary loss function, how do you all compute gradients? We can easily apply numerical differentiaion and get things up and running and I’ve seen a good example here. The Flux.gradient(.…

---

## [Learning rate scheduler with the new interface of Flux](https://discourse.julialang.org/t/learning-rate-scheduler-with-the-new-interface-of-flux/107142)

<div class="topic-metadata">

**Author:** [@iHany](https://discourse.julialang.org/u/iHany)\
**Replies:** 4\
**Last updated:** [December 23, 2023, 5:34am UTC](https://discourse.julialang.org/t/learning-rate-scheduler-with-the-new-interface-of-flux/107142 "2023-12-23T05:34:07Z")

</div>

Hi, I’ve not been actively using Flux.jl for a while and I found that the interface of Flux has changed a bit. Here is the docs of Flux, “Scheduling Optimisers”. It seems like that using ParameterSchedulers.jl is reco…

---

## [Smoothing probability distribution output of network](https://discourse.julialang.org/t/smoothing-probability-distribution-output-of-network/107934)

<div class="topic-metadata">

**Author:** [@taotree](https://discourse.julialang.org/u/taotree)\
**Replies:** 3\
**Last updated:** [December 22, 2023, 4:17pm UTC](https://discourse.julialang.org/t/smoothing-probability-distribution-output-of-network/107934 "2023-12-22T16:17:57Z")

</div>

I’m training a neural network with flux and outputting a probability distribution (ie. probability bins for the value of a continuous variable). The output layer is outputting the values for the bins which I then normali…

---

## [Introducing NNUE to the Julia community](https://discourse.julialang.org/t/introducing-nnue-to-the-julia-community/107722)

<div class="topic-metadata">

**Author:** [@Tarny\_GG\_Channie](https://discourse.julialang.org/u/Tarny_GG_Channie)\
**Replies:** 1\
**Last updated:** [December 19, 2023, 11:53am UTC](https://discourse.julialang.org/t/introducing-nnue-to-the-julia-community/107722 "2023-12-19T11:53:33Z")

</div>

Background: Local search has been a backbone of many optimization problems. When it comes to heuristic guide, however, machine learning is difficult to integrate into the field due to the time it takes to evaluate the n…

---

## [Are there guidelines or rules of thumb on how to stack hidden layers in a RNN?](https://discourse.julialang.org/t/are-there-guidelines-or-rules-of-thumb-on-how-to-stack-hidden-layers-in-a-rnn/107307)

<div class="topic-metadata">

**Author:** [@Hugo](https://discourse.julialang.org/u/Hugo)\
**Replies:** 5\
**Last updated:** [December 14, 2023, 10:25pm UTC](https://discourse.julialang.org/t/are-there-guidelines-or-rules-of-thumb-on-how-to-stack-hidden-layers-in-a-rnn/107307 "2023-12-14T22:25:03Z")

</div>

I’m currently working on the prediction of chaotic data and I have decided to see how well would an RNN, namely an LSTM, would do. I am fairly new to the topic of Neural Networks, but I have found a spate of helpful reso…

---

## [Any equivalent in Lux.jl to torch.nn.Parameter?](https://discourse.julialang.org/t/any-equivalent-in-lux-jl-to-torch-nn-parameter/107200)

<div class="topic-metadata">

**Author:** [@liuyxpp](https://discourse.julialang.org/u/liuyxpp)\
**Replies:** 1\
**Last updated:** [December 13, 2023, 7:53pm UTC](https://discourse.julialang.org/t/any-equivalent-in-lux-jl-to-torch-nn-parameter/107200 "2023-12-13T19:53:24Z")

</div>

I want to know if there is any equivalent to PyTorch’s torch.nn.Parameter in Lux.jl. Thanks!

---

## [JuliaSimModelOptimiser, UDEs, network and model connection](https://discourse.julialang.org/t/juliasimmodeloptimiser-udes-network-and-model-connection/107569)

<div class="topic-metadata">

**Author:** [@rpex](https://discourse.julialang.org/u/rpex)\
**Replies:** 1\
**Last updated:** [December 13, 2023, 3:24pm UTC](https://discourse.julialang.org/t/juliasimmodeloptimiser-udes-network-and-model-connection/107569 "2023-12-13T15:24:32Z")

</div>

When I use Neural Automated Model Discovery for Autocompleting Models with Prior Structural Information from JuliaSimModelOptimiser. How is the neural network connected with the incomplete model? Is it: \\frac{d states}…

---

## [Flux.update! not working with custom AbstractArray](https://discourse.julialang.org/t/flux-update-not-working-with-custom-abstractarray/107530)

<div class="topic-metadata">

**Author:** [@nelslind](https://discourse.julialang.org/u/nelslind)\
**Replies:** 1\
**Last updated:** [December 12, 2023, 11:13pm UTC](https://discourse.julialang.org/t/flux-update-not-working-with-custom-abstractarray/107530 "2023-12-12T23:13:37Z")

</div>

When building a model using custom structs that are subtypes of AbstractArray, Flux.update! cannot update the model. For my use case, it is particularly convenient to do computations using these custom structs. Is there…

---

## [Type unstable gradients in Zygote (@code\_warntype)](https://discourse.julialang.org/t/type-unstable-gradients-in-zygote-code-warntype/107279)

<div class="topic-metadata">

**Author:** [@Iris\_Allevi](https://discourse.julialang.org/u/Iris_Allevi)\
**Replies:** 5\
**Last updated:** [December 12, 2023, 4:31pm UTC](https://discourse.julialang.org/t/type-unstable-gradients-in-zygote-code-warntype/107279 "2023-12-12T16:31:04Z")

</div>

Hello there, I’m trying to write efficient code for custom automatic differentiation in Julia. I noticed that even for a simple case, Zygote gives type unstable gradients. Can you please help me understand why? Here’s…

---

## [LSTM Method Error - Time Series](https://discourse.julialang.org/t/lstm-method-error-time-series/73973)

<div class="topic-metadata">

**Author:** [@sherlock\_holmes](https://discourse.julialang.org/u/sherlock_holmes)\
**Replies:** 3\
**Last updated:** [December 7, 2023, 1:37am UTC](https://discourse.julialang.org/t/lstm-method-error-time-series/73973 "2023-12-07T01:37:28Z")

</div>

I’m trying to train an LSTM model to predict number of real roots of polynomials. x\_train and y\_train include array of arrays such as \[\[-204, 20, 13, 1, 0\]\] which are coefficients of polynomials. x\_test and y\_test includ…

[Previous page](https://discourse.julialang.org/c/domain/ml/24.md?page=9)

[Next page](https://discourse.julialang.org/c/domain/ml/24.md?page=11)