# Zygote @adjoint usage for covariance matrix calculation

**URL:** <https://discourse.julialang.org/t/zygote-adjoint-usage-for-covariance-matrix-calculation/87352>\
**Category:** General Usage\
**Tags:** question, statistics, zygote\
**Created:** [September 16, 2022, 8:25am UTC](https://discourse.julialang.org/t/zygote-adjoint-usage-for-covariance-matrix-calculation/87352 "2022-09-16T08:25:14Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![kadir-gunel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kadir-gunel/32/35790_2.png) [@kadir-gunel](https://discourse.julialang.org/u/kadir-gunel)\
**Post date:** [September 16, 2022, 8:25am UTC](https://discourse.julialang.org/t/zygote-adjoint-usage-for-covariance-matrix-calculation/87352/1 "2022-09-16T08:25:14Z")

</div>

Hello,

I implemented `Fréchet distance` for using it as a loss function in Flux. But Zygote is not happy because it does not recognize the covariance calculation.

The documentation of Zygote package has a `custom adjoint` section but I don’t understand how we extend Zygote to calculate the covariance method. Could someone explain me how we implement it ?

The function (`freschet distance`) is described as below:

```julia
function freschet_distance(X::T, Y::T) where {T}    
    μ1 = mean(X, dims=2)
    μ2 = mean(Y, dims=2)
    
    μ = sum((μ1 - μ2).^2)
    
    σ1 = cov(X, dims=2)
    σ2 = cov(Y, dims=2)
    
    σ_mean = sqrt.(σ1 .* σ2)
    
    σ_mean = isequal(σ_mean |> typeof, ComplexF32) ? real(σ_mean) : σ_mean
    
    return μ + tr(σ1 + σ2 - 2σ_mean)
end

```

By the way, 2 years ago, I had a very similar problem with `CUDA.zeros` and `CUDA.fill` methods : [this intrinsic must be compiled to call](https://discourse.julialang.org/t/error-this-intrinsic-must-be-compiled-to-be-called/52497). And the problem was fixed just by writing one line of code for each. At that time, I just wanted to solve the problem and didn’t/couldn’t understand why they are needed.  
This time, in addition to covariance definition for Zygote, my other question is : The exact same distance function is being used by PyTorch. And there is no “re-definition” of covariance for autograd in PyTorch. Then why Zygote needs this type of things ? What makes it special ?

B.R.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [September 16, 2022, 11:14am UTC](https://discourse.julialang.org/t/zygote-adjoint-usage-for-covariance-matrix-calculation/87352/2 "2022-09-16T11:14:16Z")

</div>

I think the issue is the [lack of a rule for `cov`](https://github.com/JuliaDiff/ChainRules.jl/issues/634). Someone should write one, either from [the formula](https://en.wikipedia.org/wiki/Estimation_of_covariance_matrices#Estimation_in_a_general_context) or by finding another implementation to base it on.

Alternatively, for now you could use a simpler `cov` implementation which will be differentiable, something like `mycov(x::AbstractVector; corrected::Bool=true) = sum(abs2, x .- mean(x)) / (length(x) - corrected)`.

> [@kadir-gunel](#):
>
> The exact same distance function is being used by PyTorch. And there is no “re-definition” of covariance for autograd in PyTorch.

Are you sure? [This](https://github.com/pytorch/pytorch/issues/19037) looks like it certainly needed some definitions.

---

<div class="post-metadata">

**Author:** ![kadir-gunel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kadir-gunel/32/35790_2.png) [@kadir-gunel](https://discourse.julialang.org/u/kadir-gunel)\
**Post date:** [September 16, 2022, 6:00pm UTC](https://discourse.julialang.org/t/zygote-adjoint-usage-for-covariance-matrix-calculation/87352/3 "2022-09-16T18:00:58Z")

</div>

About `mycov` function. What is the purpose of using a boolean value and subtracting from the length? I just tried it for a 20 element array, and it gives 19 when Boole variable is true, and 20 otherwise.  
Didn’t know that subtracting boolean from int is possible.

> Are you sure? [This](https://github.com/pytorch/pytorch/issues/19037) looks like it certainly needed some definitions.

The implementation that I referred to is [this](https://github.com/mseitzer/pytorch-fid) which does not have anything like you have mentioned. Since I was just looking for some reference code, I didn’t check if the code is used for training or testing. And the code seems to be used for testing. So it is being used as a distance function and not as a loss.

So, you are right - as always :).

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [September 17, 2022, 1:07am UTC](https://discourse.julialang.org/t/zygote-adjoint-usage-for-covariance-matrix-calculation/87352/4 "2022-09-17T01:07:14Z")

</div>

> [@kadir-gunel](#):
>
> What is the purpose of using a boolean value and subtracting from the length?

This is just one of the stats functions like variance where it’s standard to divide by `n-1` to account for only having a sample. And since Julia’s booleans are integers, `true == 1`, subtracting the flag is a quick way to do it.

The linked issue now has a sketch of how to write the gradient, BTW.

---

<div class="post-metadata">

**Author:** ![kadir-gunel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kadir-gunel/32/35790_2.png) [@kadir-gunel](https://discourse.julialang.org/u/kadir-gunel)\
**Post date:** [September 22, 2022, 10:01am UTC](https://discourse.julialang.org/t/zygote-adjoint-usage-for-covariance-matrix-calculation/87352/5 "2022-09-22T10:01:46Z")

</div>

Hello @mcabbott , sorry for the late reply.

Thank you for the link. I will try to use it.

B.R.