As @dfdx had mentioned, there’s some trouble with differentiating expressions like logpdf(Normal(μ,σ),x) - “differentiating a data type” doesn’t really make sense.
I looked a bit more into the implementation. For functions like logpdf, Distributions just calls StatsFuns, which leads to a nice algebraic expression after a couple of steps. The following are all equivalent, because they are successive simplifications of the first line:
Main> logpdf(Normal(μ,σ),x)
-2.548376445638773
Main> normlogpdf(μ,σ,x)
-2.548376445638773
Main> normlogpdf((x-μ)/σ) - log(σ)
-2.548376445638773
Main> -(abs2((x-μ)/σ) + log2π)/2 - log(σ)
-2.548376445638773
Distributions manages this with a macro _delegate_statsfuns.
What’s really needed, I think, is a way to easily get from the first of these to the last. I could code this in Soss for this specific case, but it seems like a generally useful thing anyway. Most generally, I could imagine a function that takes an expression and returns an iterator over successive evaluation steps. I’m guessing the most efficient path might be somewhere between a quick hack and the “best possible world” evaluation iterator.
Any thoughts/suggestions?