Gradient check by zygote

New to Julia
,
1

Hi I am using Julia Gridap library to calculate gradients. I wrote the rules for gradients but when i check the gradients with running finite difference manually. they are different. I check equations multiple time and i couldn’t find the issue. is there any suggestion to debug and find the issue?
thank you

2

Hi,
Could you provide some reproducible code? It’s a bit hard to debug abstractly

3

Hi, thank you for you response. I really tried to make it simple and post it here but i couldnt it is huge. in general i do not get any errors in code but i checked the gradients of zygote with finite difference and i found that gradients of zygote are 10e4 times larger than finite difference. is there any method to check the gradient calculation and find the issue?

4

When you say you wrote the rules, do you mean the ChainRules?
There is a package called ChainRulesTestUtils that can help you check those rules against finite differences (and perform other verifications), but presumably it will alert you to the same issue you already found.

Other than that, I fear the advice I can give without some code or math will be very generic program debugging advice. Check on very very small and easy instances first. Insert print statements or use the debugger to follow the execution of your code. Break big functions down into smaller functions.

Since the gradients are much larger than they should be, maybe you’re dividing by a very small quantity which should not be small?

5

yes i am using chainrule. I will implement your suggestions thank you. ChainRulesTestUtils is new for me i hope it helps

6

Hi Mary,

so I have written many rules manually, and what i typically do is start by easy. That mean I start with a very simple code you want to manually AD and gradually increase complexity towards the final form I want. After each change, I test the gradient by comparing my rule to finite differences. Doing this step by step and starting with easy allow me to find the spot, where I have a bug.

Hope it helps.

1 Like
7

Hi Tomas,
Thank you for your suggestion. I also did that. my model has phase field modelling so at first i did that without considering phase field but as i added phase field parameter i got discrepancies.