# Kernel density estimation status
**URL:** https://discourse.julialang.org/t/kernel-density-estimation-status/5928
**Category:** Statistics
**Tags:** question, package
**Created:** [September 17, 2017, 3:30am UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928 "2017-09-17T03:30:03Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [September 17, 2017, 3:30am UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/1 "2017-09-17T03:30:03Z")
What is the de facto standard package for KDE in Julia?
- [KernelDensity.jl](https://github.com/JuliaStats/KernelDensity.jl)
- [KernelDensityEstimate.jl](https://github.com/dehann/KernelDensityEstimate.jl)
- [KernelEstimator.jl](https://github.com/panlanfeng/KernelEstimator.jl)
My attempts with the packages listed above were not very successful.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [September 17, 2017, 8:01am UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/2 "2017-09-17T08:01:33Z")
I have used `KernelDensity.jl` recently. The following runs with the current version just fine:
```julia
using KernelDensity
using Plots; gr()
x = randn(100)
y = kde(x)
plot(linspace(extrema(x)..., 100), z->pdf(y, z))
```
Being more specific than “not successful” would make it easier to help you. What did you try? What happened? How does it not conform to your expectations?
---
### Author:  [@mcreel](https://discourse.julialang.org/u/mcreel)
#### Post date: [September 17, 2017, 9:52am UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/3 "2017-09-17T09:52:26Z")
I have a function that does basic multivariate local constant, linear and quadratic nonparametric fitting in [https://github.com/mcreel/Econometrics.jl](https://github.com/mcreel/Econometrics.jl), just call npreg() to see an example. I use this code for teaching, and will try to make it reasonable accessible, but maybe not fully featured.
In the package [https://github.com/mcreel/QuantileIV.jl](https://github.com/mcreel/QuantileIV.jl) there is an extended version which does nonpararmetric quantile regression, using code from [https://github.com/pkofod/QuantileRegression.jl](https://github.com/pkofod/QuantileRegression.jl). The stuff in my QuantileIV.jl package needs to be cleaned up a bit for public use, but it works well for me.
---
### Author:  [@joshday](https://discourse.julialang.org/u/joshday)
#### Post date: [September 17, 2017, 1:34pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/4 "2017-09-17T13:34:39Z")
The first Julia package I wrote was [GitHub - joshday/AverageShiftedHistograms.jl: ⚡ Lightning fast density estimation in Julia ⚡](https://github.com/joshday/AverageShiftedHistograms.jl), which essentially does KDE over a fine-partition histogram. It’s cool if you’re working with large datasets, as it uses constant memory and can be estimated on-line.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [September 17, 2017, 3:20pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/5 "2017-09-17T15:20:35Z")
> [@Tamas\_Papp](#):
>
> Being more specific than “not successful” would make it easier to help you. What did you try? What happened? How does it not conform to your expectations?
For KernelDensity.jl, I was concerned about the theoretical aspect of it. Is it common practice to use spline interpolation for evaluating the density at new locations? It currently uses Interpolations.jl, but I am not sure this is the correct thing to do? Plus, I need to evaluate the KDE object thousands of times, my experience with the package is that it is somehow slow? I will double check my code to make sure the slowness is coming from the KDE part and not somewhere else.
For KernelEstimator.jl, it currently only supports 1D KDEs. My use case requires 2D.
For KernelDensityEstimate.jl, the repository only has 2 stars on GitHub, I wanted to double check if it is being used.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [September 17, 2017, 3:32pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/6 "2017-09-17T15:32:06Z")
Thanks @joshday! It looks like a very nice package, both in terms of performance and API! I will definitely give it a try.
Do you recommend a paper on average shifted histograms? Or a publication on which you based your implementation?
---
### Author:  [@joshday](https://discourse.julialang.org/u/joshday)
#### Post date: [September 17, 2017, 3:43pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/7 "2017-09-17T15:43:59Z")
I used some course notes from David Scott (who invented ASH) which I can no longer find online. He has a few papers on the subject.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [September 17, 2017, 4:08pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/8 "2017-09-17T16:08:50Z")
Thanks, I’ve downloaded a bunch of papers by him for my dessert 🙂
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [September 17, 2017, 4:24pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/9 "2017-09-17T16:24:53Z")
> [@juliohm](#):
>
> I was concerned about the theoretical aspect of it. Is it common practice to use spline interpolation for evaluating the density at new locations? It currently uses Interpolations.jl, but I am not sure this is the correct thing to do?
If you are using kernel densities for data visualization, there is no “correct” approach. Do what you like. If interpolations give a speedup, use them.
If you are using it for nonparametric estimation, then most results are asymptotic anyway. So the spline-interpolated density it may not coincide with some theoretical definitions, but this should not matter.
It is hard to say more without some context.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [September 17, 2017, 4:31pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/10 "2017-09-17T16:31:22Z")
> [@Tamas\_Papp](#):
>
> If you are using it for nonparametric estimation, then most results are asymptotic anyway. So the spline-interpolated density it may not coincide with some theoretical definitions, but this should not matter.
I agree that results are always dependent on the number of samples, but this doesn’t mean we can play with arbitrary algorithms. When I think kernel density _estimation_, I don’t think of visualization. If I was thinking of visualization, I would use splines.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [September 17, 2017, 4:48pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/11 "2017-09-17T16:48:27Z")
> [@juliohm](#):
>
> but this doesn’t mean we can play with arbitrary algorithms
I am under the impression that it pretty much does. As long as we have asymptotic results, and the differences vanish asymptotically. People do this all the time, derive new algorithms with different small-sample properties, and argue about their merits, while preserving the same asymptotics.
The more you know (or are willing to assume) about your estimated density, the more guidance the literature gives you about nonparametric estimation. But again, most of those results are based on heuristics, MC or theoretical results for a particular distribution, or asymptotics.
> [@juliohm](#):
>
> If I was thinking of visualization, I would use splines.
Kernel densities are commonly used for data visualization and exploration. Probably one of the best tools for that.
---
### Author:  [@Lanfeng](https://discourse.julialang.org/u/Lanfeng)
#### Post date: [September 17, 2017, 5:05pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/12 "2017-09-17T17:05:17Z")
> For KernelEstimator.jl, it currently only supports 1D KDEs. My use case requires 2D.
`KernelEstimator.jl` does support multivariate kernel density estimate and multivariate local constants regression.
The only issue is the bandwidth selection for multivariate kernel density. The user are responsible to choose the right bandwidth. The default bandwidth is chosen via likelihood cross validation which has some known disadvantages. I don’t have good solution yet. Comments and pull requests are welcome.
Regression and univariate kde do not suffer this problem. Their bandwidth are chosen by least squares cross validation.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [September 17, 2017, 7:11pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/13 "2017-09-17T19:11:04Z")
Thank you @Lanfeng! I think the README is outdated:
> The Julia package for nonparametric kernel density estimate and regression. This package currently includes univariate kernel density estimate, local constant regression (Nadaraya-Watson regression) and local linear regression. It can also compute the Bootstrap confidence band [4].
I will give it a try as well, thanks for the package!
---
### Author:  [@Elrod](https://discourse.julialang.org/u/Elrod)
#### Post date: [September 18, 2017, 12:14am UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/14 "2017-09-18T00:14:05Z")
> [@juliohm](#):
>
> Plus, I need to evaluate the KDE object thousands of times, my experience with the package is that it is somehow slow?
```julia
julia> using KernelDensity, BenchmarkTools
julia> x = randn(100_000);
julia> d = kde(x);
julia> @benchmark pdf($d, 0.1)
BenchmarkTools.Trial:
memory estimate: 265.28 KiB
allocs estimate: 164
--------------
minimum time: 612.651 µs (0.00% GC)
median time: 652.279 µs (0.00% GC)
mean time: 700.002 µs (4.56% GC)
maximum time: 3.875 ms (78.17% GC)
--------------
samples: 7117
evals/sample: 1
julia> itp_d = InterpKDE(d);
julia> @benchmark pdf($itp_d, 0.1)
BenchmarkTools.Trial:
memory estimate: 16 bytes
allocs estimate: 1
--------------
minimum time: 40.197 ns (0.00% GC)
median time: 41.052 ns (0.00% GC)
mean time: 42.637 ns (1.10% GC)
maximum time: 769.449 ns (90.14% GC)
--------------
samples: 10000
evals/sample: 1000
```
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [September 18, 2017, 12:49am UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/15 "2017-09-18T00:49:33Z")
Thanks @Elrod, I am using 2D KDEs though. In any case, I am looking forward for the new release of AverageShiftedHistograms.jl that @joshday is working on.
The papers by David Scott are really great. ASH is very powerful, it has comparable statistical properties to that of kernel-based estimators and is as fast as simple frequency histograms. The best of both worlds. It has become my favorite density estimation technique today.
---
### Author:  [@Nectarineimp](https://discourse.julialang.org/u/Nectarineimp)
#### Post date: [September 18, 2017, 8:04pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/16 "2017-09-18T20:04:41Z")
I will have to check that out for myself! I found the other packages slow for my taste. Using 10,000 observations something like 10 or 30 seconds? I forgot what it was, but I had millions of observations. I ended up doing random sampling of 10,000 samples so it wouldn’t kill my entire process.
Thanks for sharing this.
---
### Author:  [@Nectarineimp](https://discourse.julialang.org/u/Nectarineimp)
#### Post date: [September 20, 2017, 6:47pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/17 "2017-09-20T18:47:33Z")
OK looks like AverageShiftedHistorgrams isn’t working well under Julia 0.6. At least the example code I tried is broken. It also gives quite a few deprecation warnings. I will try and find some time in the next two weeks to do some PRs to help make is 0.6 (and hopefully 1.0) compatible. fyi:@joshday
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [September 20, 2017, 6:49pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/18 "2017-09-20T18:49:06Z")
@Nectarineimp, you need to checkout the master branch, @joshday is still working on a new release.
---
### Author:  [@Nectarineimp](https://discourse.julialang.org/u/Nectarineimp)
#### Post date: [September 20, 2017, 6:54pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/19 "2017-09-20T18:54:14Z")
OK that sounds great. Maybe he has some particular tasks he wants help with. I’m all for volunteering for this. I used KDE quite a bit in the past.
---
### Author:  [@joshday](https://discourse.julialang.org/u/joshday)
#### Post date: [September 20, 2017, 8:14pm UTC](https://discourse.julialang.org/t/kernel-density-estimation-status/5928/20 "2017-09-20T20:14:03Z")
If there’s any missing functionality you want added, PRs are gladly welcome! There is an open issue for a pdf method for bivariate Ash if you’re up for tackling that.
[Next page](https://discourse.julialang.org/t/kernel-density-estimation-status/5928.md?page=2)