Hi! This is more of a theoretical tangent, so forgive me if this is a bit off topic. I’ve recently finished my undergrad in Statistics and have been trying to navigate the US job market, to fairly dismal results. Anyway, I’ve tried to pick up some cool new statistical nuggets in the meantime, especially those I didn’t see in school, like Gaussian Processes.
I’ve been trying to work through Rasmussen and Williams, but I feel like I’m not getting a great conceptual idea of how Gaussian Processes fit in with the rest of Bayesian hierarchical models. I understand that by using a kernel function to generate a covariance matrix, we don’t need to specify the form of our model beforehand for more modeling flexibility. In a previous Bayesian course, I saw that we can compare the effectiveness of multiple models by creating a large hierarchical model with an extra level for “model”. Is there a connection with GP’s here or am I off base?
I suppose I (perhaps incorrectly) always thought of Bayesian linear regression models and the like as a distribution over possible functions, so the emphasis on this in Bayesian Processes is confusing to me. Is this a property unique to GP’s and I’ve been misinterpreting other models? Or is this just a connection to the larger Bayesian methods?
Last one; A good number of blogs online seem to define the functions over which the GP describes the distribution on, as smooth and continuous. Rasmussen and Williams seems to make note that this is not the case, and that when generating functions from the prior we simply generate a random vector from the GP. I suppose to me it seems GP’s place a distribution over a discrete collection of points, with a (countably) finite domain and I don’t understand how we can make predictions over points not listed in the domain.
I hope to utilize Steno + Soss in an NLP project I’m working on, but I figure I should get a decent understanding before jumping in.
Thanks in advance for reading through my super newbie questions!
P.S. All the PPL and GP presentations from JuliaCon were great this year!