Using ReverseSourceDiff for a function with matrices

differentiation
#1

I was wondering if anyone could help me with why my code will not work:

using ReverseDiffSource
X_=randn(2,2)
Y_=randn(2,2)
DerivativeE_c=m.rdiff(E_c, (Matrix{Float64}, Matrix{Float64}))
DerivativeE_c(X_,Y_)

Where E_c is my function. I’ve modeled my code after the third example on this page:


But I cannot get the package to work at all. I get that m is not defined or rdiff is not. Have I modeled this function correctly?

#2

To clarify, there were no errors in adding the package:

Pkg.add("ReverseDiffSource")
#3

AFAICT this package is abandonned, and was not even updated for 0.6.

#4

I think XGrad.jl is in some ways the successor of ReverseDiffSource.

#5 
m.rdiff

This looks like a typo in the original README, function for differentiation in ReverseDiffSource is just rdiff as it’s shown in all other examples.

Anyway, as mentioned above, ReverseDiffSource.jl haven’t seen any updates for 2 years, but I maintain a similar package - XGrad.jl. You can use it as follows:

using XGrad

E_c(x, y) = sum(x * y)

DerivativeE_c = xdiff(E_c; x=rand(2,2), y=rand(2,2))

X_ = randn(2,2)
Y_ = randn(2,2)

DerivativeE_c(X_,Y_)

Note that latest version already targets Julia 0.7, so if you are on Julia 0.6 you need to checkout branch julia-0.6 of XGrad AND a branch with the same name in Espresso.

#6

Thank you so much that looks like it will work nicely for me. I am new to julia, would you be able to explain how to install just a branch? I learned how to install a package from a website but I cannot find the exact .jl webpage for the branch, just this one

It looks like the correct Espresso is


I believe

#7

In Julia 0.6 (the current stable version that you most likely are using), you can add registered package such as XGrad just as:

Pkg.add("XGrad")

or clone the master branch using (also works for unregistered packages):

Pkg.clone("https://github.com/dfdx/XGrad.jl")

This puts the whole git repository of the package somewhere on your disk, e.g. on Linux the default location is ~/.julia/v0.6/XGrad.

Espresso.jl is a dependency of XGrad.jl and is installed automatically into similar location, e.g. ~/.julia/v0.6/Espresso.

Once you get both of the packages, you can enter the directory of the project and, since it’s still a git repo, switch to the branch you want. Again, in Linux you can do it as:

cd ~/.julia/v0.6/XGrad
git checkout julia-0.6

cd ~/.julia/v0.6/Espresso
git checkout julia-0.6

Alternatively, you can switch a branch right from julia console as follows:

Pkg.checkout("XGrad", "julia-0.6")
Pkg.checkout("Espresso", "julia-0.6")

which should also work on Windows.

In Julia 0.7 things are a bit different, but I won’t describe it for now to not overload you with too much information.

#8

Okay, thank you I understand now and I’ve gotten it working for me!

#9

Does XGrad support linear algebra functions such as kron, trace, conj, and exponentials of matrices? My function seems too complicated for XGrad as I get the error:
Can't understand module of Main
But I also may be wrongly setting it up.

#10

XGrad uses simple chain rule over a set of functions with known derivatives. The default list of such functions and derivatives is defined in rules.jl and can be extended using @diffrule macro. However, there’s no derivatives for functions you mentioned, nor I can find them in DiffRules.jl (another repository with similar goal). So if you find derivatives for them, I’ll add these to the list.

Can’t understand module of Main

This seems unrelated, can you please post the whole example as an issue on GitHub?

#11

Okay, for the moment I need to clear up and refine my code, I’ll post again when I have done this. Thank you