How to test differentiability efficiently

General Usage
,
1

Hello.

As the tile suggest I am interested in testing (using Test) the “AD” capabilities of functions I write. My idea is to simply compare if Zygote.gradient() and finite differences give the same answer.

Many times I have functions which have multiple inputs say my_func(a,b,c,d,e). So far what I do is generate f = x -> my_func(x, b, c, d, e) and test and so on.

Is there a better way of doing this? By better I mean less verbose than generating 5 functions for each function I make and then testing each. Perhaps to do it in fewer lines and more automatically.

One concern I have is that I want to know which input the test fails. Which splitting it this way makes it easier.

Any suggestions.

Thank you!

1 Like
2

Change it to take a single vector argument (e.g. x -> my_func(x...)) and compute the Jacobian?

1 Like
3

woudn’t that make it hard to see which input is cousing troubles?

4

Depending on which components of the Jacobian are wrong, you will know immediately which input screwed up. It’s exactly the same information, just all at once

2 Likes
5

Maybe, but if you split it you can see when the test fails which part did. If you just see Jacobian[2, 12] was miscalculated that is hard? Or is there a way of making that meaningful in tests?

6

This makes the output a bit verbose, but you could wrap the tests in their own @testset with an appropriate name. Perhaps group the tests by the corresponding column of the jacobian and attach the name of the variable only to the corresponding @testset.

I had something like this in mind, but I’m not sure if it suits your needs

using Test

f(x) = x * x'

@testset "Differentiability" begin
    variableNames = [:x, :y]

    testInputs = [[1, 2], [3, 4]]

    for xVal in testInputs
        jac = f(xVal)
        @testset "point = $(xVal)" begin
            for (i, (x, c)) in enumerate(zip(variableNames, eachcol(jac)))
                @testset "$x = $(xVal[i])" begin @test c == [1, 2] end
            end
        end
    end
end

Using NamedTuples instead of putting the variables names in a separate vector might even be more convenient.

7

I think it is very cool. Thank you!

1 Like