Packages for Variational Bayes inference? Turing?

Juan · November 23, 2018, 1:55pm

What is the most versatile and/or fasest Julia package for doing “Variational Bayes inference”?
I’ve only found two packages: VarBayes.jl and TopicModelsVB.jl, the former seems deprecated.

How can I do Variational Bayes inference with Turing.jl? it’s “universal”, though maybe only useful for MCMC. I can’t find it on its documentation.

Stan can do it (some methods) but I prefer to use a 100% Julia solution.

oliver · December 19, 2018, 2:39am

Did you find what you were looking for?

outlace · January 7, 2019, 5:32am

I’ve also been looking for this and have not found it. As far as I can tell there is no robust package for variational bayesian inference in pure Julia.

Juan · January 7, 2019, 10:51am

I asked at Turing.jl and they ( xukai92) replied:

VI is not-supported at the moment. But this is something we will look at in the future.

I guess this we will need to wait for a long time or a new package.

Tamas_Papp · January 7, 2019, 12:29pm

At the moment, your best bet is via Stan. I am not sure if Stan.jl can do VI, but it’s mostly an interface thing so it should be easy to incorporate.

oliver · January 17, 2019, 1:41pm

My understanding is that this can be done in Stan but it’s not a fully developed feature of the language. So it’s possible using the tools but still an experimental application.

cscherrer · January 17, 2019, 5:27pm

PyMC3 has good VI support:
https://docs.pymc.io/notebooks/variational_api_quickstart.html

If you want to do more elaborate things, Pyro focuses on stochastic VI:
http://pyro.ai/examples/

I’m hoping to get this kind of thing going in Soss.jl, but that will be summer at the earliest.

oliver · February 7, 2019, 3:43am

I looked into this a little more and Stan has experimental implementations of both fullrank and meanfield advi accessible through CmdStan.jl.

outlace · February 7, 2019, 4:43am

While it doesn’t appear to officially released yet, you can try Gen.jl < GitHub - probcomp/Gen.jl: A general-purpose probabilistic programming system with programmable inference > Looks like it’s aiming to be a pretty full-featured and sophisticated probabilistic programming language.

Juan · September 6, 2019, 11:14pm

How do the packages Gen.jl and Turing.jl compare?

oliver · September 13, 2019, 2:21pm

Here is something related to this discussion: https://github.com/StanJulia/StanVariational.jl

trappmartin · September 17, 2019, 1:58pm

FYI: Variational Inference is now supported in Turing.jl. At the moment you will need to use the master branch, but this will change soon.

trappmartin · September 17, 2019, 2:01pm

You can find an example of the current ADVI implementation in Turing here: https://github.com/TuringLang/Turing.jl/blob/master/test/variational/advi.jl

Note that there are some open PRs (https://github.com/TuringLang/Turing.jl/pull/903, https://github.com/TuringLang/Turing.jl/pull/902) which will extend/improve the current state but it is already possible to use ADVI for most Turing models.