# Problem defining a Straight Through Estimator gradient

**URL:** <https://discourse.julialang.org/t/problem-defining-a-straight-through-estimator-gradient/20698>\
**Category:** Machine Learning\
**Created:** [February 12, 2019, 11:31am UTC](https://discourse.julialang.org/t/problem-defining-a-straight-through-estimator-gradient/20698 "2019-02-12T11:31:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![velix](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/velix/32/7067_2.png) [@velix](https://discourse.julialang.org/u/velix)\
**Post date:** [February 12, 2019, 11:31am UTC](https://discourse.julialang.org/t/problem-defining-a-straight-through-estimator-gradient/20698/1 "2019-02-12T11:31:00Z")

</div>

Hi all,

I am trying to define a custom STE gradient in Flux. The activation function is the sign function, and its gradient should just be the incoming gradient as is, iff its absolute value is \<= 1, or cancelled otherwise. The code I have is below

```julia
binarize(x) = x>=0 ? true : false

binarize(x::Flux.Tracker.TrackedReal) = Flux.Tracker.track(binarize, x)

@grad function binarize(x)
    return binarize(Flux.Tracker.data(x)), Δ -> (abs(x) <= 1 ? Δ : 0, )
end

```

For a random 5x1 array `a`, i do the following

```julia
>> a= param(randn(5))
>> Tracked 5-element Array{Float64,1}:
 -0.3605564089879154
 -0.7853512499733902
  0.8102988051980005
 -0.9715952052917924
 -1.276343849200165 

>> c=binarize.(a)
>> 5-element BitArray{1}:
 false
 false
  true
 false
 false

>> Tracker.back!(c, [1,1,1,1,1])
>> a.grad
>> 5-element Array{Float64,1}:
 0.0
 0.0
 0.0
 0.0
 0.0

```

There are a couple problems here. First of all I’d expect the gradient to be a TrackedArray.  
But more importantly, the returned gradient should not be all zeroes, but ones except for the last element right?

What am I doing wrong here?

Thanks,