# Automatic Differentiation Slow (Slower than Finite-Differences)

**URL:** <https://discourse.julialang.org/t/automatic-differentiation-slow-slower-than-finite-differences/11209>\
**Category:** Optimization (Mathematical)\
**Created:** [May 29, 2018, 1:21am UTC](https://discourse.julialang.org/t/automatic-differentiation-slow-slower-than-finite-differences/11209 "2018-05-29T01:21:14Z")\
**Posts on this page:** 1\
**Showing post:** 14

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [May 31, 2018, 2:34am UTC](https://discourse.julialang.org/t/automatic-differentiation-slow-slower-than-finite-differences/11209/14 "2018-05-31T02:34:22Z")

</div>

Yeah, there have been a bunch of changes in `reinterpret`. This works on both 0.6 and master:

```julia
using Compat
using ForwardDiff
const LinearAlgebra = Compat.LinearAlgebra

function MTv(M::Matrix{Float64}, v::Vector{ForwardDiff.Dual{T,V,N}}) where {T, V <: LinearAlgebra.BlasFloat, N}
    num_v = length(v)
    vmat = transpose(reshape(reinterpret(V, v), N + 1, num_v)) # first column: values, other columns: partials
    result = transpose(M * vmat) # first row: values, other rows: partials
    reinterpret(ForwardDiff.Dual{T, V, N}, reshape(result, length(result)))
end

using Compat.Test

@testset "MTv" begin
    m = 3
    n = 4
    p = 5

    M = rand(m, n)
    v = [ForwardDiff.Dual(rand(), ForwardDiff.Partials(tuple(rand(p)...))) for _ = 1 : n]
    ret1 = MTv(M, v)
    ret2 = M * v
    @test ret1 ≈ ret2
end

nparams = 10;
v_len = 100_000;
out_len =30;
v = ForwardDiff.Dual.(randn(v_len),[tuple(rand(nparams)...) for dd=1:v_len]);
M=rand(out_len, v_len);

using BenchmarkTools
@btime MTv($M, $v);
@btime $M * $v;

```

Results on latest master:

```julia
  6.503 ms (6 allocations: 2.88 KiB)
  14.319 ms (1 allocation: 2.75 KiB)

```

Results on 0.6.2:

```julia
  9.500 ms (12 allocations: 8.40 MiB)
  17.520 ms (1 allocation: 2.75 KiB)

```

Edit: I noticed that the difference is much smaller (but still significant) with `-O3`.

---

_[View the full topic](https://discourse.julialang.org/t/automatic-differentiation-slow-slower-than-finite-differences/11209)._