# How to understand GP extrapolation shape

**URL:** <https://discourse.julialang.org/t/how-to-understand-gp-extrapolation-shape/62765>\
**Category:** Probabilistic Programming\
**Tags:** gaussian-process\
**Created:** [June 11, 2021, 10:04pm UTC](https://discourse.julialang.org/t/how-to-understand-gp-extrapolation-shape/62765 "2021-06-11T22:04:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [June 11, 2021, 10:04pm UTC](https://discourse.julialang.org/t/how-to-understand-gp-extrapolation-shape/62765/1 "2021-06-11T22:04:42Z")

</div>

```julia
using CairoMakie
using KernelFunctions
using AbstractGPs

n = 30
x_train = rand(n) .+ 1
y_train = log.(x_train)

f = GP(Matern52Kernel())

fx = f(x_train, 0.1)

p_fx = posterior(fx, y_train)

fig = Figure()
Axis(fig[1,1])

xplot = collect(minimum(x_train):0.001:maximum(x_train))
ms = marginals(p_fx(xplot))
means = mean.(ms)

lines!(xplot, means, linewidth=10)
scatter!(x_train, y_train;color=:red)

xplot2 = collect(1:.001:5)
ms2 = marginals(p_fx(xplot2))
means2 = mean.(ms2)

lines!(xplot2, means2)
fig

```

 ![image](https://global.discourse-cdn.com/julialang/original/3X/6/c/6cbf05b7731a42ad7d8f18f9e746f0b87e583e3d.png)

I am extrapolating this Matern kernel GP outside the range of the data. I want to understand why the extrapolation looks the way it does.

- Why does it go back down after x=2?
- Why does it asymptote at y=0?

---

<div class="post-metadata">

**Author:** ![Red-Portal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/red-portal/32/9102_2.png) [@Red-Portal](https://discourse.julialang.org/u/Red-Portal)\
**Post date:** [June 20, 2021, 2:38pm UTC](https://discourse.julialang.org/t/how-to-understand-gp-extrapolation-shape/62765/2 "2021-06-20T14:38:47Z")

</div>

By not passing an explicit mean to `GP` like you did in:

```julia
f = GP(Matern52Kernel())

```

you’re declaring “zero-mean” Gaussian process prior. See [this line](https://github.com/JuliaGaussianProcesses/AbstractGPs.jl/blob/0e223c8cd57d625c9476f9e4365beaf80b7fad14/src/gp/gp.jl#L63). Since your GP prior assumes zero-mean, the GP will converge towards 0 where there are not enough data points.

---

<div class="post-metadata">

**Author:** ![cgeoga](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cgeoga/32/216186_2.png) [@cgeoga](https://discourse.julialang.org/u/cgeoga)\
**Post date:** [June 20, 2021, 3:22pm UTC](https://discourse.julialang.org/t/how-to-understand-gp-extrapolation-shape/62765/3 "2021-06-20T15:22:17Z")

</div>

For any mean zero process it will eventually asymptote at zero, but if you wanted the extrapolation to “last” a bit longer before smoothly heading to zero, you might consider using a covariance function corresponding to a long memory process, which I think is most cleanly defined as a covariance function whose corresponding spectral density has a (integrable) singularity at the origin. Fractional ARIMA processes are an example, and I bet there is at least one time series package that does ARFIMA models.

---

<div class="post-metadata">

**Author:** ![Red-Portal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/red-portal/32/9102_2.png) [@Red-Portal](https://discourse.julialang.org/u/Red-Portal)\
**Post date:** [June 21, 2021, 7:15am UTC](https://discourse.julialang.org/t/how-to-understand-gp-extrapolation-shape/62765/4 "2021-06-21T07:15:38Z")

</div>

Extrapolation in GPs is a sophisticated matter that need some special treatments. One usually need to use covariance kernels that are able to extrapolate as pointed out by @cgeoga . See for example [this paper](http://proceedings.mlr.press/v28/wilson13.html) and [this paper](https://proceedings.neurips.cc/paper/2014/hash/77369e37b2aa1404f416275183ab055f-Abstract.html) by Andrew Wilson which discusses a combination of kernel that works well for extrapolation in a lot of cases. Additional resources can be found in [this webpage](https://people.orie.cornell.edu/andrew/pattern/).