# Zygote push forward unable to differentiate through generic broadcast

**URL:** <https://discourse.julialang.org/t/zygote-push-forward-unable-to-differentiate-through-generic-broadcast/64364>\
**Category:** Machine Learning\
**Created:** [July 9, 2021, 5:07pm UTC](https://discourse.julialang.org/t/zygote-push-forward-unable-to-differentiate-through-generic-broadcast/64364 "2021-07-09T17:07:03Z")\
**Posts on this page:** 1\
**Showing post:** 10

<div class="post-metadata">

**Author:** ![RynoLaubscher](https://avatars.discourse-cdn.com/v4/letter/r/8baadc/32.png) [@RynoLaubscher](https://discourse.julialang.org/u/RynoLaubscher)\
**Post date:** [July 14, 2021, 7:14am UTC](https://discourse.julialang.org/t/zygote-push-forward-unable-to-differentiate-through-generic-broadcast/64364/10 "2021-07-14T07:14:48Z")

</div>

Hi @ChrisRackauckas, I tried ForwardDiff over Zygote on a simpler net. I noted that reverse (Zygote) over forward (Zygote) is significantly faster than ForwardDiff over Zygote (reverse), this is most likely due to the fact that the forward diff now has to compute a one-to-many set of gradients. Please see the MWE below along with execution time. Maybe I misunderstood your message…most likely 🙂

```julia
using Zygote
using ForwardDiff
using Statistics
using Flux

X = randn(2, 100)
data_ones = ones(2,100)

fc1 = Dense(2, 10, tanh)
fc2 = Dense(10, 10, tanh)
fc3 = Dense(10, 2, identity)

model = Chain(fc1, fc2, fc3)
θ, re = Flux.destructure(model)

function loss_FD_over_Z(x, θ)
    network = re(θ)
    ∂o = Zygote.gradient(p -> sum(network(p)), x)[1]
    mean((cos.(x) .- ∂o).^2)
end
loss_FD_over_Z(X, θ)

@time ∇θ = ForwardDiff.gradient(p -> loss_FD_over_Z(X, p), θ)
  4.893996 seconds (21.12 M allocations: 1.006 GiB, 4.20% gc time, 99.46% compilation time)

function loss_Z_over_Z(x)
    ∂o = Zygote.pushforward(x -> model(x), x)(data_ones)
    mean((cos.(x) .- ∂o).^2)
end
loss_Z_over_Z(X, θ)

@time ∇θ = Zygote.gradient(() -> loss_Z_over_Z(X), Zygote.Params(model))
  0.896909 seconds (43.72 k allocations: 3.084 MiB, 99.60% compilation time)

```

---

_[View the full topic](https://discourse.julialang.org/t/zygote-push-forward-unable-to-differentiate-through-generic-broadcast/64364)._