Is there a function in Flux to estimate the best learning rate for a good gradient descent before training a neural network?

@xiaodai You are right about Adam modifying the learning rate adaptively. Yet, the code at this URL: Function in Flux to estimate learning rate - #5 by samq uses Adam as the optimizer, but changes the learning rate anyway here: # If we haven't seen improvement in 5 epochs, drop our learning rate:…

Function in Flux to estimate learning rate

Specific Domains Machine Learning

samq March 11, 2019, 10:37pm 5

@ oxinabox I humbly stand corrected!
Thank you for the example!

Topic		Replies	Views
How to update learning rate during Flux training in a better manner? New to Julia flux	7	2408	December 23, 2023
What package allowed setting the learning rate for a minimization problem? General Usage optimization	6	321	April 16, 2023
Learning rate decay in callback function Machine Learning question , lux	3	493	January 11, 2024
Learning rate scheduler with the new interface of Flux Machine Learning flux	4	1090	December 23, 2023
Implementing the Learn rate scheduling in the NeuralPDE julia package New to Julia question	5	257	November 21, 2023

Function in Flux to estimate learning rate

Related topics