How to tell if invalidations are worth fixing?

I can use SnoopCompile to detect invalidations.
and the manual talks about how to fix them by modifying the code that gets invalidated.
But it is also possible to fix it sometimes by modifying the code that causes the invalidation.
Namely, you can delete it.
Normally don’t want to do this, but if it is an obscure and largely unused overload in a highly used package I guess it can be worth it.

To determine if it is worth it, I want to know how much removing it would speed “things” up.
But I don’t exactly know what things I should measure.

Example is that convert in ChainRulesCore causes 126 invalidations.
https://github.com/JuliaDiff/ChainRulesCore.jl/issues/359
We absolutely could remove that (and might have other reasons to beyond invalidations) but I don’t know if removing it would save 1 millisecond per user per year, or 10ms per user per day.

What should i measure before and after the change of removing it?

7 Likes