# Meaning of Δ in different \`apply!\` methods is different in different Flux optimizers?

**URL:** <https://discourse.julialang.org/t/meaning-of-in-different-apply-methods-is-different-in-different-flux-optimizers/52811>\
**Category:** Statistics\
**Tags:** flux, optimization\
**Created:** [January 4, 2021, 7:57am UTC](https://discourse.julialang.org/t/meaning-of-in-different-apply-methods-is-different-in-different-flux-optimizers/52811 "2021-01-04T07:57:08Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![gxyd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gxyd/32/19986_2.png) [@gxyd](https://discourse.julialang.org/u/gxyd)\
**Post date:** [January 4, 2021, 7:57am UTC](https://discourse.julialang.org/t/meaning-of-in-different-apply-methods-is-different-in-different-flux-optimizers/52811/1 "2021-01-04T07:57:08Z")

</div>

Hi,

I want to understand the meaning of `Δ` used in different methods (i.e. for different optimizers) in Flux.

For example: in [Descent](https://github.com/FluxML/Flux.jl/blob/master/src/optimise/optimisers.jl#L39), the meaning of `Δ` seems to be:  
[∂f/∂θᵢ] i=1,2,3,…

While looking at the meaning of `Δ` in [AdaGrad](https://github.com/FluxML/Flux.jl/blob/master/src/optimise/optimisers.jl#L346), the meaning of AdaGrad takes time into account as well (if I understood AdaGrad correctly), then `Δ` mathematically means a diagonal matrix where `ith` element is:

![Screen Shot 2021-01-04 at 1.19.54 PM](https://global.discourse-cdn.com/julialang/original/3X/3/3/33ed2feb100b1e47cf72b5e8dad7b67b67e752e9.png)

, considering that my understanding is correct. How is `Δ` given different (types) of values? Since its a part of the API, which is usually called by `update!` function, which eventually calls `x -= Δ`. I’m not able to wrap my head around as to how that operation (even when vectorised) is going to be a valid one?