Hi @00krishna . There is a little bit of overview about ReactiveMP (which uses GraphPPL for model specification).
In our cases we are mostly interested in creating the factor graph representation of a probabilistic model. The main purpose then is to run message passing-based Bayesian inference, we do not use sampling.
not DAG
Unlike other PPL (as far as I know) we are not concerned neither with loops the factor graph nor with direction. Our models are not DAGs. For example in ReactiveMP you can do something like:
@model function my_model()
x ~ Normal(0.0, 1.0)
x ~ Normal(0.0, 1.0) # This is a perfectly valid factor graph and has a proper interpretation
...
end
Message passing-based inference can run inference in this case and this form has a proper interpretation. ~ expression does not really mean sampled from (but tries to look like it), but simply creates a functional relationship between random variables in the form of an edge in a factor graph. We do not use this feature much though.
On the other hand, it applies some extra limitations to the model specification, e.g. we cannot use random variables within square brackets expressions as we treat random variables as … well… random variables and not numbers, as how for example Turing does. For example this is not possible:
@model function my_model()
x ~ Normal(0.0, 1.0)
y ~ Normal(0.0, 1.0)
# `x` and `y` cannot be used within indexing expression
# this expression has no proper interpretation in a form of a factor graph
z ~ MvNormal([ x, y ], diageye(2))
end
datavars
We also use a different way of passing observation in the model:
@model function my_model()
y = datavar(Float64, n) # `y` accepts an array of length `n` with type `Float64`
end
The reason is that we treat data variables as reactive inputs, model accepts an infinite data stream and continously updates posterior marginals of random variables in the model as soon as data arrives (and if it has time to do that). The whole API is designed about reactivity. Posterior marginals are exposed as streams, Bethe Free Energy computation is a stream of floats, etc. So in our framework it is possible, for example, to subscribe on posterior marginal and redirect it into a stream of prior for the next time step and perform online filtering procedure with potentially infinite data stream.
reactivity
Most of the other PPL packages are concerned with just evaluating p(x|y) given some static dataset y. ReactiveMP API (and PPL) is designed for different purposes. We treat y as an infinite data stream in a form of the observable (from Rocket.jl). We may update our data with new observations and, as a result, posterior marginals will also react and update themselves.
for loops
We are mostly interested in state-space models, so in ReactiveMP it is very common to see for loops:
@model function my_model(n)
x = randomvar(n)
y = datavar(Float64, n)
...
for i in 1:n
y[i] ~ Normal(x[i], 1.0)
end
...
end
form and factorisation constraints
Another feature of our language is that it supports extra inference constraints specification for random variables (e.g. functional form constraints, on strategy to compute posterior) and factorisation constraints specification for factor nodes, which to my knowledge does not do any of the packages. We use where { options... } syntax for that to inject some extra inference specific context.
For example:
@model function my_model()
# `x` will be approximated as a point mass, which resembles EM procedure
x = randomvar() where { marginal_form_constraint = PointMassFormConstraint() }
# or
y ~ Normal(x, v) where { q = MeanField() } # or q = q(x, y)q(v) for structured VMP
end
We plan to split though form and factorisation constraints language from model specification language in the next big release. It should look smth like:
constraints = @constraints begin
q(x) :: PointMass
q(x, y, v) = q(x, y)q(v)
end
Don’t hesitate to ask more questions! This description is rather small and probably does not cover all aspects of our specification language. Somewhat more information can be found here: Model Specification · ReactiveMP.jl
Best,
Dmitry.