Packages for Variational Bayes inference? Turing?

What is the most versatile and/or fasest Julia package for doing “Variational Bayes inference”?
I’ve only found two packages: VarBayes.jl and TopicModelsVB.jl, the former seems deprecated.

How can I do Variational Bayes inference with Turing.jl? it’s “universal”, though maybe only useful for MCMC. I can’t find it on its documentation.

Stan can do it (some methods) but I prefer to use a 100% Julia solution.

1 Like

Did you find what you were looking for?

I’ve also been looking for this and have not found it. As far as I can tell there is no robust package for variational bayesian inference in pure Julia.

I asked at Turing.jl and they ( xukai92) replied:

VI is not-supported at the moment. But this is something we will look at in the future.

I guess this we will need to wait for a long time or a new package.

At the moment, your best bet is via Stan. I am not sure if Stan.jl can do VI, but it’s mostly an interface thing so it should be easy to incorporate.

My understanding is that this can be done in Stan but it’s not a fully developed feature of the language. So it’s possible using the tools but still an experimental application.

1 Like

PyMC3 has good VI support:
https://docs.pymc.io/notebooks/variational_api_quickstart.html

If you want to do more elaborate things, Pyro focuses on stochastic VI:
http://pyro.ai/examples/

I’m hoping to get this kind of thing going in Soss.jl, but that will be summer at the earliest.

4 Likes

I looked into this a little more and Stan has experimental implementations of both fullrank and meanfield advi accessible through CmdStan.jl.

1 Like

While it doesn’t appear to officially released yet, you can try Gen.jl < https://github.com/probcomp/Gen > Looks like it’s aiming to be a pretty full-featured and sophisticated probabilistic programming language.

1 Like

How do the packages Gen.jl and Turing.jl compare?

Here is something related to this discussion: https://github.com/StanJulia/StanVariational.jl

FYI: Variational Inference is now supported in Turing.jl. At the moment you will need to use the master branch, but this will change soon.

4 Likes

You can find an example of the current ADVI implementation in Turing here: https://github.com/TuringLang/Turing.jl/blob/master/test/variational/advi.jl

Note that there are some open PRs (https://github.com/TuringLang/Turing.jl/pull/903, https://github.com/TuringLang/Turing.jl/pull/902) which will extend/improve the current state but it is already possible to use ADVI for most Turing models.

3 Likes