I want to calculate the Kullback-Leibler divergence of data I collected in a vector
x, which I interpret as samples from an unknown distribution, and the standard normal distribution. The maths behind the KL divergence are straightforward. My naive approach would be to
- choose a number of bins
- make a histogram of
- discretize the density of the normal distribution according to the bins
- calculate the KL divergence of two vectors using for example
I wonder how good of an approach that is (conceptually and implementation wise). Is there a Julia package with more refined methods? What about the sensitivity with respect to the number of bins?
Thanks for all the answers!