Is there a function in Flux to estimate the best learning rate for a good gradient descent before training a neural network?

@xiaodai You are right about Adam modifying the learning rate adaptively. Yet, the code at this URL: Function in Flux to estimate learning rate - #5 by samq uses Adam as the optimizer, but changes the learning rate anyway here: # If we haven't seen improvement in 5 epochs, drop our learning rate:…

Function in Flux to estimate learning rate

Specific Domains Machine Learning

samq March 11, 2019, 10:37pm 5

@ oxinabox I humbly stand corrected!
Thank you for the example!

Topic		Replies	Views
What package allowed setting the learning rate for a minimization problem? General Usage optimization	6	288	April 16, 2023
Learning rate decay in callback function Machine Learning question , lux	3	414	January 11, 2024
How to update learning rate during Flux training in a better manner? New to Julia flux	7	2263	December 23, 2023
Regression in Flux.jl Machine Learning question , flux	5	2590	May 4, 2020
Learning rate scheduler with the new interface of Flux Machine Learning flux	4	876	December 23, 2023

Function in Flux to estimate learning rate

Related topics