I’m working on some changes in Soss, and they’re different than the usual way of going about PPL. Thought I’d share them here in case other are curious or have any thoughts blah blah (the usual stuff on that front)
To this point, my approach in
Soss.jl has been in line with previous probabilistic programming systems I’ve used. For example, here’s a very simple model:
m₀ = @model σ,x begin μ ~ Cauchy() N = length(x) x ~ Normal(μ,σ) |> iid(N) end
There’s a lot going on in these few lines:
- σ is a free variable: everything else in the join distribution is conditional on it.
- x is observed: its is assumed to come from the joint distribution described in the body of the model, but its value is known.
- μ is a parameter; our typical goal in PPL is to find its distribution, conditional on σ and x.
- N is deterministic, and is a function of x
Model manipulation is one goal of Soss that sets it apart from most PPLs. We’d like to turn a generative model into a conditional one, or a model with data into one with only a prior. But the way we write models makes this very weird.
What are the dependencies in the above model? Does x come first, or last? It’s a mess.
As one small step forward, I’m looking into an alternate approach for observed values. Rather than being passed as parameters, they are attached to the model struct.
In this approach, the above model is written
m = @model σ begin μ ~ Cauchy() N = length(x) x ~ Normal(μ,σ) |> iid(N) end
Data are then observed like this:
julia> m_obs = m(x=randn(10)) @model σ begin μ ~ Cauchy() N = length(x) x ⩪ Normal(μ, σ) |> iid(N) end
This creates a new model with everything the same, except that m_obs.data is now a
NamedTuple with field
Some other aspects:
- No inference code has been generated yet (that’s next)
- The type of the data is known statically
⩪is a hint to the user that data is attached. Data is expected to be too large to easily view inline.
- “Statements” (the body of the function) are unordered, and acquire an ordering at printing or inference time. There’s still some of this to work out.
- For running a model on different data, one option is to encode is an a
What do you think? See any big problems or opportunities?