Hi, I am doing a machine learning using gradient descend # I have a set of data, called samples # I have two models: model1 and model2 global par = Flux.params(model1, model2) #wrap them in params gs = gradient(par) do global l1 = mean(Flux.mse(model(sample),true1)) for sample in sa…

In your do block, the returned value is the value of the last expression. In this case, you actually return the output of a call to println, which is nothing (and it has no gradient). The last line of the do block needs to be return allLoss, and it should work?

Tangentially, I don’t think there is the need for all these globals in your snippet. For performance reason you should consider removing them because globals are desastrous for performance. See related performance tip: Performance Tips · The Julia Language Documentation for The Julia La…

Note also that using Flux.params is discouraged, and it will be removed soon. The recommended style is something like this: samples, label, true1, opt, model1, model2 = ... opt_stats = Flux.setup(opt, (model1, model2)) # neccesary for explicit mode gs = gradient(model1, model2) do m1, m2 # expl…

Yes. The problem is just the “println”. Thanks for helping!

Yes. You are right. I delete them and it works well. Thanks for helping!

Thanks for helping! I tried your method and succeed. But I do have another problem. If I execute code like this: using DelimitedFiles using InferOpt using ProgressMeter using Flux using Gurobi using JuMP using Distributions using Statistics using JSON using Dates using Parameters model1 = Chain( …

You might be in a region of the parameter space where all relu activation functions are on the left side of zero, which means the gradients vanish. I assume this effect would disappear with another activation function like tanh (although this is not necessarily a wise switch). More generally, you’re…

Two issues: You’re again using Chain(x -> model1(x), ...) instead of Chain(model1, ...) in your definitions. Therefore the parameters of model1 are not visible in model2/model3. You are taking the gradient gradient(model2, model3) wrt model2, model3, but update Flux.update!(opt_stats, (model1,mode…

Oh yes! I made a primary mistake. Thanks for helping!! But the framework seems have some problems when using GPU? using CUDA using Flux using Random Random.seed!(333) model1 = Chain( Dense(3, 3) |> gpu ) model2 = Chain( model1, Dense(ones(3,3),true,relu) |> gpu ) model3 = Chain( …

See also [image] Problem relating to use gradient descends two models using GPU General Usage Sorry, My fault. I mistakenly used trianloader…though I still didn’t know how to use it now. Anyway, it will run smoothly without using trainloader: using CUDA using Flux usin…

Yes. That’s what I realised before. I forgot to state in here

Problems relating to "gradient is nothing"

General Usage

gdalle October 29, 2024, 2:14pm 11

Topic		Replies	Views
Flux.gradient returns dict of param and Nothing Machine Learning flux	3	816	September 22, 2021
Gradient not being computed when training NN using Flux Machine Learning question , flux , sciml	3	157	January 30, 2025
Computing Flux.gradient change the model Machine Learning	10	1448	January 31, 2021
Params not getting updated during training New to Julia flux	25	1920	October 11, 2020
Problem on model and gradient descend in Flux General Usage	18	323	October 27, 2024

Problems relating to "gradient is nothing"

Related topics