# Understanding the performance of Zygote

**URL:** https://discourse.julialang.org/t/understanding-the-performance-of-zygote/55447
**Category:** Machine Learning
**Tags:** question, differentiation, zygote
**Created:** [February 17, 2021, 9:33am UTC](https://discourse.julialang.org/t/understanding-the-performance-of-zygote/55447 "2021-02-17T09:33:57Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Himanshu\_Chaudhary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/himanshu_chaudhary/32/14903_2.png) [@Himanshu\_Chaudhary](https://discourse.julialang.org/u/Himanshu_Chaudhary)
#### Post date: [February 17, 2021, 9:33am UTC](https://discourse.julialang.org/t/understanding-the-performance-of-zygote/55447/1 "2021-02-17T09:33:57Z")

</div>

I remember reading somewhere that the cost of automatic differentiation is usually less than 2 times the cost of evaluating the function itself.

I tried to benchmark a small code and the results that I got were a little confusing.

```nohighlight
using BenchmarkTools
using Zygote

function simple(x)
    return sin(x)
end

function complicated(x)
    return sin(x)^2*exp(x)*x^-2 + cos(x)^3
end

x = 1:0.01:10

@btime simple.($x);
@btime simple'.($x);

@btime complicated.($x);
@btime complicated'.($x);

```

```julia
14.942 μs (1 allocation: 7.19 KiB)
  21.299 μs (1 allocation: 7.19 KiB)
  47.380 μs (1 allocation: 7.19 KiB)
  672.962 μs (14417 allocations: 1006.73 KiB)

```

Why is the AD of simple function so much faster than that of the complicated function?  
Why are there so many allocations while taking the derivative of the complicated function?

Thanks

---

<div class="post-metadata">

### Author: ![wulpuqu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wulpuqu/32/17640_2.png) [@wulpuqu](https://discourse.julialang.org/u/wulpuqu)
#### Post date: [February 18, 2021, 10:37am UTC](https://discourse.julialang.org/t/understanding-the-performance-of-zygote/55447/2 "2021-02-18T10:37:27Z")

</div>

It might have to do with the broadcasting. The pullback is faster when the function `complicated` operates on vectors directly:

```julia
complicated_wb(x) = sin.(x) .^2 .* exp.(x) .* x.^(-2) .+ cos.(x) .^ 3
x = 1:0.01:10

```

```julia
 >@btime complicated_wb(x)
104.899 μs (1 allocation: 7.19 KiB)
> @btime pullback(complicated_wb,x)[2](ones(length(x)))[1]
331.299 μs (8230 allocations: 380.52 KiB)
> @btime complicated'.(x)
759.201 μs (14420 allocations: 1006.92 KiB)

```