Gaussian process with noisy heteroscedastic correlated targets (outputs)

Specific Domains Statistics
1

Hi there,
I have 2 kinds of datasets to which I would like to fit a Gaussian process:
(1) a dataset with N observations: (x_i, y_i, \sigma_i) , i=1, 2, ..., N, where x_i are the input (covariate) variables, y_i the output (target) variables, and \sigma_i are the varying noises, so that it is a heteroscedastic process. I was able to use, in Python, the sci-kit learn’s package GaussianProcessRegressor, with the parameter alpha equal to the vector \sigma and everything went ok. I guess there should be an equally easy procedure in one of the packages of the GaussianProcesses echosystem of Julia. Could you give me the exact reference?
(2) another dataset, constituted by two files:

  • the first one is the observations (x_i, y_i) , i=1, 2,...N
  • the second one contains the noise for the output variables (targets) in the form of a N \times N covariance matrix (array) \Sigma , no longer diagonal. This is what I have called noisy heteroscedastic correlated targets in the title of my topic. Does it make sense to fit a Gaussian process to such a dataset? If so, which Julia package allows implementing this and exactly how? Pondering a bit deeper, I think (is this correct?) what this second file amounts to is a sample or estimate for the covariance function k(x_i, x_j) , i,j =1, 2, ..., N ; as such I should use some approximation to it to define a new kernel for my call to the Gaussian process routine…

Thank you very much in advance!

2

I can point out one possible approach, using Turing.jl. For example, your model could look like below?

using Turing, LinearAlgebra
@model function GPR(x, Σ)
    N = length(x)
    η ~ Exponential(2)
    ρ ~ Exponential(1)
    d² = (x .- x').^2
    Κ = η * (exp.(-ρ * d²))
    y ~ MvNormal(zeros(N), Κ + Σ)
end

No guarantees about anything (don’t know anything about GPs), just to note that it lets you try almost anything. (Turing Gaussian Process tutorial)

3

Thanks @eteppo,
I will ponder about your suggestion.