How to test differentiability efficiently


As the tile suggest I am interested in testing (using Test) the “AD” capabilities of functions I write. My idea is to simply compare if Zygote.gradient() and finite differences give the same answer.

Many times I have functions which have multiple inputs say my_func(a,b,c,d,e). So far what I do is generate f = x -> my_func(x, b, c, d, e) and test and so on.

Is there a better way of doing this? By better I mean less verbose than generating 5 functions for each function I make and then testing each. Perhaps to do it in fewer lines and more automatically.

One concern I have is that I want to know which input the test fails. Which splitting it this way makes it easier.

Any suggestions.

Thank you!

1 Like

Change it to take a single vector argument (e.g. x -> my_func(x...)) and compute the Jacobian?

1 Like

woudn’t that make it hard to see which input is cousing troubles?

Depending on which components of the Jacobian are wrong, you will know immediately which input screwed up. It’s exactly the same information, just all at once


Maybe, but if you split it you can see when the test fails which part did. If you just see Jacobian[2, 12] was miscalculated that is hard? Or is there a way of making that meaningful in tests?

This makes the output a bit verbose, but you could wrap the tests in their own @testset with an appropriate name. Perhaps group the tests by the corresponding column of the jacobian and attach the name of the variable only to the corresponding @testset.

I had something like this in mind, but I’m not sure if it suits your needs

using Test

f(x) = x * x'

@testset "Differentiability" begin
    variableNames = [:x, :y]

    testInputs = [[1, 2], [3, 4]]

    for xVal in testInputs
        jac = f(xVal)
        @testset "point = $(xVal)" begin
            for (i, (x, c)) in enumerate(zip(variableNames, eachcol(jac)))
                @testset "$x = $(xVal[i])" begin @test c == [1, 2] end

Using NamedTuples instead of putting the variables names in a separate vector might even be more convenient.

I think it is very cool. Thank you!

1 Like