# Gaussian Process Model with Turing

**URL:** <https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453>\
**Category:** Probabilistic Programming\
**Tags:** regression, turing, gaussian-process\
**Created:** [July 2, 2020, 9:10pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453 "2020-07-02T21:10:36Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![theogf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/theogf/32/1987_2.png) [@theogf](https://discourse.julialang.org/u/theogf)\
**Post date:** [November 18, 2020, 1:22pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/62 "2020-11-18T13:22:51Z")

</div>

> Note that I don’t really understand the distinction 😂

What you really care about is how many unique samples you get from your sampler. For example in Metropolis Hastings, if you would reject all proposal you would get stuck with an effective sample size of one (although you might have done calculations for 10 millions tries).

> I did some googling and didn’t find any julia-specific information about this - any ideas where I should look?

I don’t think there is any Laplace method implemented at the moment, also because Julia AD is pretty bad with Hessians right now…

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [November 18, 2020, 2:31pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/63 "2020-11-18T14:31:38Z")

</div>

> [@kevbonham](#):
>
> Note that I don’t really understand the distinction 😂

This is a fun topic! Here’s a very hand-wavy explanation, just to build on @theogf’s point. 🙂

It’s common for “number of samples” n to show up in computations statistical computations, usually with the assumption that samples are independent. Think of the computation for standard error of the mean, for example.

In an MCMC context, we can often compute things more directly than depending on formulas like this, which often only hold asymptotically and with some assumptions (iid, etc).

That’s great, but when you plug everything in, the equations often don’t quite work out, because some assumptions are broken. It turns out we can quantify “how broken” they are (again, hand-wavily).

So we pretend we don’t know n, and solve for it, calling it n\_{\text{eff}}, the _effective number of samples_. However many samples we actually took, the result behaves as if we had taken n\_{\text{eff}}.

Though we often sweep it under the rug, it’s important to remember that there are different ways of computing this. So when you dig into the details, you always need to consider the asymptotic result used, and which assumptions from that result are broken.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [November 18, 2020, 7:03pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/64 "2020-11-18T19:03:06Z")

</div>

> [@mschauer](#):
>
> @kevbonham Can you restate the actual model you are interested in?

Sure! Given that I’m not totally certain that the code I’m writing is what I want, I’ll state it again in words. What I am most interested in is comparing two models, one that contains one variable (in this case, diet) and the other that doesn’t. The components of the model are:

- `bug` - the dependent variable (continuous)
- `diet` - I want to compare models with and without this variable (continuous), which should have linear relationship with `bug`
- x\_1 \dots x\_n some number of covariates with a linear relationship to `bug`
- `subject` (categorical) and `time` - I’ve got multiple samples from each subject over time.

`bug` varies over time, with samples that are closer to one another in time more similar to one another. But this relationship only holds for the same `subject`. That is, `time=1` for `subject=`1 and `time=1` for `subject=2` have no relationship. Also, the number `subject` has no numerical meaning, it’s just a UID.

So here’s the model I came up with\[1\] in code. Here, I’m ignoring the x\_n stuff, just trying to do `bug`, `diet`, `subject` and `time`:

```julia
sekernel(alpha, rho) = 
  alpha^2 * KernelFunctions.transform(SEKernel(), sqrt(0.5)/rho)

@model function GPmodel1(bug, diet, subj, tp, jitter=1e-6)
    ns = length(unique(subj))
    
    # subj coefficients (random effects) priors
    a ~ filldist(Normal(), ns)
    
    # diet coefficient prior
    b ~ Normal()

    # gp priors
    sig2 ~ LogNormal(0, 1)
    alpha ~ LogNormal(0, 0.1)
    rho ~ LogNormal(0, 1)

    # lm specs
    mu = a[subj] + b * bug
    
    # gp specs
    kernel = sekernel(alpha, rho) # covariance function
    K = kernelpdmat(kernel, reshape(tp, length(tp), 1), obsdim=1) # cov matrix
    K += LinearAlgebra.I * (sig2 + jitter)
    
    bug .~ MvNormal(mu, K)
end

```

* * *

1. When I say “I came up with,” I really mean “that I mostly copied from examples earlier in this thread from @luiarthur”

---

<div class="post-metadata">

**Author:** ![luiarthur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/luiarthur/32/16405_2.png) [@luiarthur](https://discourse.julialang.org/u/luiarthur)\
**Post date:** [November 18, 2020, 9:28pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/65 "2020-11-18T21:28:51Z")

</div>

I think this line:

`mu = a[subj] + b * bug`

should be

`mu = a[subj] + b * diet`

* * *

Also, I think it might help to think about the model more abstractly. Something like

y\_{s,j} = \beta\_0 + d\_{s,j} \cdot \beta\_d + f\_s(t\_{s,j}) + \epsilon\_{s,j}, for j=1,\dots,J\_s, and s=1,\dots,S,

where y\_{s,j} represents the dependent variable (bug) for the j-th measurement for subject s, d\_{s,j} is the diet for subject s at measurement j, \beta\_d is the diet effect, \beta\_0 is the intercept, t\_{s,j} is the time at which measurement j was taken for subject s, f\_s(\cdot) is a (possibly non-linear) function of time for subject s, and \epsilon\_{s,j} is the residuals. (f could be a number of things. e.g., a linear function, a polynomial, some other non-linear function, etc.)

I would think that the relationship between the dependent variable and time might be different for each subject. So, I included a subscript on f for each subject. Though, this was not accounted for in the Turing model.

@kevbonham Does this look like the model you intend to fit? If not, what is the high level description of your intended model? For example, do you think f should be indexed by subject? Or is one diet prescribed for all time for each subject and, thus, d does not need to be indexed by j?

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [November 18, 2020, 10:01pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/66 "2020-11-18T22:01:39Z")

</div>

Can you show a couple of trajectory of persons? Perhaps one can see what is the actual correlation structure in time.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [November 19, 2020, 4:49pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/67 "2020-11-19T16:49:35Z")

</div>

> [@luiarthur](#):
>
> I think this line:
> 
> `mu = a[subj] + b * bug`
> 
> should be
> 
> `mu = a[subj] + b * diet`

Totally right - I used to have this with x and y but changed them for this post and inserted the wrong thing there. It was correct when I was running it

> [@luiarthur](#):
>
> @kevbonham Does this look like the model you intend to fit? If not, what is the high level description of your intended model? For example, do you think f should be indexed by subject? Or is one diet prescribed for all time for each subject and, thus, d does not need to be indexed by j?

Yes, or at least that’s definitely close, thanks for writing it out! In terms of model complexity, I was planning to assume that diet effects are the same for all subjects (though that’s certainly wrong biologically). I’m not 100% sure what you mean by “do you think f should be indexed by subject?” - in the GP context, we certainly expect to use the same covariance function (SE). As with diet, I was thinking that we’d just be modeling the same hyperparameters though this is certainly wrong biologically.

If it’s possible to fit a model where everything is subject-specific in an efficient way, that’s more plausible biologically, but I think it’s safe to assume that diet effects, and the length-scale of the GP are the approximately the same if that speeds things up substantially.

And to restate, the thing we’re primarily interested in is whether including the diet term improves the model fit. Beyond that, it would be nice to get an estimate of the magnitude of the diet effect, though that’s not essential. The only other parameter that might be of interest is the GP length-scale, but that’s even less necessary.

> [@mschauer](#):
>
> Can you show a couple of trajectory of persons? Perhaps one can see what is the actual correlation structure in time.

Yeah, for sure, this is something I need to do anyway. Were you thinking a 3D axis with time, diet, and bug, and separate lines for different subjects?

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [November 19, 2020, 5:37pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/68 "2020-11-19T17:37:21Z")

</div>

Or just 2d. Important thing is response over time for different subjects, perhaps with explanatories plotted along. That’s where my visual brain kicks in.

---

<div class="post-metadata">

**Author:** ![jkbest2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jkbest2/32/7350_2.png) [@jkbest2](https://discourse.julialang.org/u/jkbest2)\
**Post date:** [November 19, 2020, 5:39pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/69 "2020-11-19T17:39:56Z")

</div>

> [@theogf](#):
>
> I don’t think there is any Laplace method implemented at the moment, also because Julia AD is pretty bad with Hessians right now…

I [started a package](https://github.com/jkbest2/LaplaceApproximations.jl), but it’s definitely not ready yet. It sounds like higher-order autodiff is being addressed by Keno’s Diffractor.jl, so I’m hopeful that that piece will improve.

> [@kevbonham](#):
>
> `K = kernelpdmat(kernel, reshape(tp, length(tp), 1), obsdim=1)`

This looks like you’re creating a single covariance matrix for all patients at all times? For `P` patients each with `T` observations, this scales like O((PT)^3), which is not great. If you instead form one covariance matrix per patient, this scales like O(P T^3). There is probably also a package that will do this as a block-diagonal matrix.

---

<div class="post-metadata">

**Author:** ![luiarthur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/luiarthur/32/16405_2.png) [@luiarthur](https://discourse.julialang.org/u/luiarthur)\
**Post date:** [November 19, 2020, 5:46pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/70 "2020-11-19T17:46:05Z")

</div>

@kevbonham Let me clarify my questions.

Is it the case that for each subject s, that the diet is the same over time? Or is it the case that over time, the diet changes for a subject. (If the diet is the same over time for a given subject, then d does not need to be indexed by j. In terms of modeling, it really doesn’t matter, but it might help in thinking about the problem.) It also seems that you want to model one global diet effect. So, \beta\_d doesn’t need to be indexed by subject.

f is a function. If it is indexed by subject, then the relationship between the dependent variable and time is possibly **different** for each subject. In an extreme case, it is possible that for one subject, “bug” increases with time; and for another subject, “bug” decreases with time. Do you believe this to be the case? If so, each subject should get their own f (in other words, f should be indexed by s).

Also, what is the interpretation of “bug” and “subject”?

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [November 19, 2020, 6:19pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/71 "2020-11-19T18:19:29Z")

</div>

> [@luiarthur](#):
>
> Is it the case that for each subject ss , that the diet is the same over time? Or is it the case that over time, the diet changes for a subject.

Ahh, I see. Yes, we get a different diet observation at each time point.

> [@luiarthur](#):
>
> ff is a function. If it is indexed by subject, then the relationship between the dependent variable and time is possibly **different** for each subject. In an extreme case, it is possible that for one subject, “bug” increases with time; and for another subject, “bug” decreases with time. Do you believe this to be the case? If so, each subject should get their own ff (in other words, ff should be indexed by ss ).

It can definitely be the case that bug increases over time for one subject and decreases over time for another. But in terms of the **effect** of time on bug, we expect it to be random with a mean of zero for _all_ subjects. I’m not totally sure if this is a complete answer to your question.

> [@luiarthur](#):
>
> Also, what is the interpretation of “bug” and “subject”?

“subject” is an individual person in the study. “bug” is a measurement of the relative abundance of some species of bacteria in the gut (well, really the poop) of a subject.

---

<div class="post-metadata">

**Author:** ![luiarthur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/luiarthur/32/16405_2.png) [@luiarthur](https://discourse.julialang.org/u/luiarthur)\
**Post date:** [November 19, 2020, 7:32pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/72 "2020-11-19T19:32:26Z")

</div>

Thanks for the clarification.

> It can definitely be the case that **bug** increases over time for one subject and decreases over time for another. But in terms of the effect of **time** on  
> **bug** , we expect it to be random with a mean of zero for all subjects. I’m not totally sure if this is a complete answer to your question.

This is great. To be more specific, do you expect the effect of time on bug to be random with mean zero **across all** subjects, or **within each** subject?

Also, what is the range for **bug**? (e.g. is **bug** some number between 0 and 1? Is it a positive number? This may affect your choice of likelihood.)

I think I’ve said this before, but I think it might help to see what a simple mixed effects model gives you.

Something like this:

y\_{s,j} = \beta\_0 + d\_{s,j} \cdot \beta\_d + \gamma\_s + \epsilon\_{s,j}

where \gamma\_s is a random intercept for each subject, and \epsilon\_{s,j} is zero-mean Gaussian error. If you want to go Bayesian, then in Turing, you would do something like this (you should consider better priors):

```julia
# NOTE: This code has not been tested.
"""
y: dependent variable for each subject and time point (length: N)
diet: diet of subject and time point (length: N)
subject_indicator: a vector containing the subject id (a number between 1 and S, where S is the number of subjects). Length of `subject_indicator` should be N. 
"""
@model bugmodel(y, diet, subject_indicator)
  num_subjects = length(unique(subject_indicator)) # number of subjects
  
  intercept ~ Normal(0, 1)
  diet_effect ~ Normal(0, 1)
  subject_effects ~ filldist(Normal(0, 1), num_subjects)
  noise_sd ~ LogNormal(0, 1) # eps_{s,j} ~ Normal(0, noise_sd)  

  m = intercept .+ diet * diet_effect + subject_effects[subject_indicator]
  y .~ Normal.(m, noise_sd) # you may need a different likelihood depending on the range of "bug".
end

```

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [November 19, 2020, 7:57pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/73 "2020-11-19T19:57:03Z")

</div>

> [@jkbest2](#):
>
> I [started a package](https://github.com/jkbest2/LaplaceApproximations.jl), but it’s definitely not ready yet. It sounds like higher-order autodiff is being addressed by Keno’s Diffractor.jl, so I’m hopeful that that piece will improve.

Nice! Looking forward to it (and if you need a guinea pig…)

> [@jkbest2](#):
>
> This looks like you’re creating a single covariance matrix for all patients at all times? For `P` patients each with `T` observations, this scales like O((PT)3)O((PT)^3) , which is not great. If you instead form one covariance matrix per patient, this scales like O(PT3)O(P T^3) . There is probably also a package that will do this as a block-diagonal matrix.

Ah, excellent point. So is changing that as simple as doing

```julia
 K = kernelpdmat(kernel, reshape(tp[subj], length(tp[subj]), 1), obsdim=1) # cov matrix

```

?

---

<div class="post-metadata">

**Author:** ![jkbest2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jkbest2/32/7350_2.png) [@jkbest2](https://discourse.julialang.org/u/jkbest2)\
**Post date:** [November 19, 2020, 9:53pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/74 "2020-11-19T21:53:04Z")

</div>

> [@kevbonham](#):
>
> Ah, excellent point. So is changing that as simple as doing
> 
> ```julia
> K = kernelpdmat(kernel, reshape(tp[subj], length(tp[subj]), 1), obsdim=1) # cov matrix
> 
> ```
> 
> ?

I’m not super familiar with `kernelpdmat`, but I think this will give you a `P`-dimensional Gaussian process for `T` time points, rather than `P` separate GPs. I think the most straightforward solution would be to loop over `subj` and construct a separate `K` for each.

---

<div class="post-metadata">

**Author:** ![theogf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/theogf/32/1987_2.png) [@theogf](https://discourse.julialang.org/u/theogf)\
**Post date:** [November 20, 2020, 11:12am UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/75 "2020-11-20T11:12:51Z")

</div>

`kernelpdmat` only guarantees that the output is `PDMat` matrix (that you can pass directly to a `MvNormal`, it automatically add noise on the diagonal until positive definiteness is achieved. I think what @jkbest2 is saying is that you should have a different GP for each patient (but using the same hyperparameters). The scaling would linear in the number of patients instead of cubic but you would lose direct correlation between the patients.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [November 20, 2020, 8:13pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/76 "2020-11-20T20:13:32Z")

</div>

2 posts incoming, this one is responding to a bunch of previous stuff, and the next one is a new GP specification that I think is closer to what I want. I managed have a meeting with the guy that started this whole thing, and he cleared up some misconceptions I had.

At a higher level, I feel like maybe I should wind this thread down, and kick off other ones that are a bit less meandering. Though I **hugely** appreciate everyone that’s put in the effort to knock down some of my ignorance, this has been enormously helpful!

> [@mschauer](#):
>
> Or just 2d

As requested, some plots:

![image](https://global.discourse-cdn.com/julialang/original/3X/0/a/0a30d141e9d9e11d64de4e55b96603d9e7e59395.png)

![image](https://global.discourse-cdn.com/julialang/original/3X/f/6/f621ee249e92c28f793b295debf6a343c45a2344.png)

![image](https://global.discourse-cdn.com/julialang/original/3X/b/e/bec159ba6595ab9869bd0c170a51919fb9cd2929.png)

You may notice that a weird result of sample collection is that we have 4 timepoints per subject, but in pairs that are quite close together.

> [@luiarthur](#):
>
> This is great. To be more specific, do you expect the effect of time on bug to be random with mean zero **across all** subjects, or **within each** subject?

I think both? If it’s true within each subject, it would be true across all subjects as well, right? I suppose we would expect to observe bug appearing to increase or decrease over time for some subjects, but that this wouldn’t be because of time per say.

> [@luiarthur](#):
>
> Also, what is the range for **bug**? (e.g. is **bug** some number between 0 and 1? Is it a positive number? This may affect your choice of likelihood.)

When we measure, it’s between 0 and 1, but in this case, it’s been normalized using inverse normal transformation (which I’ve been assured is appropriate in this case). So in principle, the values I’m feeding in should be zero-centered, although you can see above that they’re not actually.

> [@luiarthur](#):
>
> I think I’ve said this before, but I think it might help to see what a simple mixed effects model gives you.

Thanks for the example - that seems to work, though the output seems off

```julia
Summary Statistics
           parameters mean std naive_se mcse ess rhat  
               Symbol Float64 Float64 Float64 Missing Float64 Float64  
                                                                                    
          diet_effect -0.1687 0.0000 0.0000 missing NaN NaN  
            intercept -1.8538 0.0000 0.0000 missing 2.0408 0.9899  
             noise_sd 1.1454 0.0000 0.0000 missing NaN NaN  
   subject_effects[1] -0.6571 0.0000 0.0000 missing NaN NaN  
   subject_effects[2] -1.1052 0.0000 0.0000 missing 2.0408 0.9899  
   subject_effects[3] 0.3499 0.0000 0.0000 missing NaN NaN  
# ... etc

```

And all the quantiles for all parameters are the same

```julia
Quantiles
           parameters 2.5% 25.0% 50.0% 75.0% 97.5%  
               Symbol Float64 Float64 Float64 Float64 Float64  
                                                                         
          diet_effect -0.1687 -0.1687 -0.1687 -0.1687 -0.1687  
            intercept -1.8538 -1.8538 -1.8538 -1.8538 -1.8538  
             noise_sd 1.1454 1.1454 1.1454 1.1454 1.1454  
   subject_effects[1] -0.6571 -0.6571 -0.6571 -0.6571 -0.6571  
   subject_effects[2] -1.1052 -1.1052 -1.1052 -1.1052 -1.1052 
# ... etc

```

I tried to plot these chains and I got a bunch of warnings and no actual plots (though I’m able to run tutorials with plots of chains just fine), eg

```julia
GKS: Rectangle definition is invalid in routine SET_VIEWPORT
┌ Warning: No strict ticks found
└ @ PlotUtils ~/.julia/packages/PlotUtils/3Ttrk/src/ticks.jl:283
GKS: Rectangle definition is invalid in routine SET_VIEWPORT
GKS: Possible loss of precision in routine SET_WINDOW
GKS: Rectangle definition is invalid in routine SET_VIEWPORT
GKS: Rectangle definition is invalid in routine SET_VIEWPORT
GKS: Rectangle definition is invalid in routine SET_VIEWPORT
GKS: Rectangle definition is invalid in routine SET_VIEWPORT
GKS: Rectangle definition is invalid in routine SET_VIEWPORT
┌ Warning: No strict ticks found
└ @ PlotUtils ~/.julia/packages/PlotUtils/3Ttrk/src/ticks.jl:283
GKS: Rectangle definition is invalid in routine SET_VIEWPORT

```

Anyway, debugging this code is not necessarily top priority. It’s nice to see how that’s done, and it was super fast.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [November 20, 2020, 8:19pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/77 "2020-11-20T20:19:51Z")

</div>

So here’s a new GP model specification. As I said, I had a meeting with someone that hand-coded the GPs that I’m trying to reproduce in Turing. He cleared up some misconceptions I had, including things that I’m pretty sure others in this thread have explained, but that I didn’t quite get.

Here, I put everything into the GP covariance matrix. Starting with the SE kernel for time, I multiply by a matrix that removes all time-based correlation with different subjects. Then I add linear covariance associated with diet data.

At least, that’s the intention:

```julia
function subjectcorrmat(subjects, t=T) where T
    n = length(subjects)
    scov = zeros(t, n, n)
    # if same person, 1.0, otherwise, 0.0
    for i in 1:n
        for j in 1:n
            subjects[i] == subjects[j] && (scov[i,j] = one(t))
        end
    end
    return scov
end

@model function GPmodel1(bug, diet, subj, tp, jitter=1e-6, T=Float64)
    nobs = length(bug)
    # assume zero mean
    mu = zeros(T, nobs)

    # subj-specific cov
    scov = subjectcorrmat(subj, T)

    # diet covariance
    dietcov = diet * diet'
    
    # variance contributed by diet slope
    b ~ Normal()
    dietcov *= b

    # gp priors
    sig2 ~ LogNormal(0, 1)
    alpha ~ LogNormal(0, 0.1)
    rho ~ LogNormal(0, 1)
    
    # gp specs
    kernel = sekernel(alpha, rho) # covariance function
    K = kernelpdmat(kernel, reshape(tp, length(tp), 1), obsdim=1) # cov matrix for time
    
    # remove cross-person covariance
    K = K .* scov
    # add outter product of diet * "slope" (cov caused by slope)
    K = K .+ dietcov
    
    # jitter for numerical stability
    K += LinearAlgebra.I * (sig2 + jitter)
    
    bug ~ MvNormal(mu, K)
end

```

Unfortunately, I’m back to getting PosDefException 🤦‍♀️, so I can’t benchmark, but I’m guessing the vastly reduced number of parameters should help some, and I think this is closer to what I’m going for anyway.

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [November 20, 2020, 8:37pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/78 "2020-11-20T20:37:21Z")

</div>

These are the explanatory variables over time, not the response?

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [November 21, 2020, 10:48pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/79 "2020-11-21T22:48:41Z")

</div>

`bug` is the response.

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [November 21, 2020, 10:57pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/80 "2020-11-21T22:57:53Z")

</div>

Thank you, now I get it! What are the funny large last minute changes in bug before terminal time?

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [November 25, 2020, 8:16pm UTC](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453/81 "2020-11-25T20:16:52Z")

</div>

That’s just the nature of this stuff, sometimes there can be wild swings. But I think it mostly looks weird because of the long interval between time 2 and time 3. The sampling is like

```julia
1--2-----------------------------------3---4

```

[Previous page](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453.md?page=3)

[Next page](https://discourse.julialang.org/t/gaussian-process-model-with-turing/42453.md?page=5)
