# Packages for Variational Bayes inference? Turing?

**URL:** <https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904>\
**Category:** Probabilistic Programming\
**Tags:** package\
**Created:** [November 23, 2018, 1:55pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904 "2018-11-23T13:55:13Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [November 23, 2018, 1:55pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/1 "2018-11-23T13:55:13Z")

</div>

What is the most versatile and/or fasest Julia package for doing “Variational Bayes inference”?  
I’ve only found two packages: VarBayes.jl and TopicModelsVB.jl, the former seems deprecated.

How can I do Variational Bayes inference with Turing.jl? it’s “universal”, though maybe only useful for MCMC. I can’t find it on its documentation.

Stan can do it (some methods) but I prefer to use a 100% Julia solution.

---

<div class="post-metadata">

**Author:** ![oliver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oliver/32/19264_2.png) [@oliver](https://discourse.julialang.org/u/oliver)\
**Post date:** [December 19, 2018, 2:39am UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/2 "2018-12-19T02:39:46Z")

</div>

Did you find what you were looking for?

---

<div class="post-metadata">

**Author:** ![outlace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/outlace/32/2216_2.png) [@outlace](https://discourse.julialang.org/u/outlace)\
**Post date:** [January 7, 2019, 5:32am UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/3 "2019-01-07T05:32:28Z")

</div>

I’ve also been looking for this and have not found it. As far as I can tell there is no robust package for variational bayesian inference in pure Julia.

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [January 7, 2019, 10:51am UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/4 "2019-01-07T10:51:48Z")

</div>

I asked at Turing.jl and they ( **[xukai92](https://github.com/xukai92)**) replied:

> VI is not-supported at the moment. But this is something we will look at in the future.

> <https://github.com/TuringLang/Turing.jl/issues/606>
>
> Hello.
> 
> Can Turing.jl be use to do Variational Bayes inference? 
> it’s “univer…sal”, though maybe only useful for MCMC. I can’t find it on its documentation.
> Will this option be added?

I guess this we will need to wait for a long time or a new package.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 7, 2019, 12:29pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/5 "2019-01-07T12:29:37Z")

</div>

> [@Juan](#):
>
> I guess this we will need to wait for a long time or a new package.

At the moment, your best bet is via Stan. I am not sure if Stan.jl can do VI, but it’s mostly an interface thing so it should be easy to incorporate.

---

<div class="post-metadata">

**Author:** ![oliver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oliver/32/19264_2.png) [@oliver](https://discourse.julialang.org/u/oliver)\
**Post date:** [January 17, 2019, 1:41pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/6 "2019-01-17T13:41:27Z")

</div>

My understanding is that this can be done in Stan but it’s not a fully developed feature of the language. So it’s possible using the tools but still an experimental application.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [January 17, 2019, 5:27pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/7 "2019-01-17T17:27:21Z")

</div>

PyMC3 has good VI support:  
[https://docs.pymc.io/notebooks/variational\_api\_quickstart.html](https://docs.pymc.io/notebooks/variational_api_quickstart.html)

If you want to do more elaborate things, Pyro focuses on stochastic VI:  
[http://pyro.ai/examples/](http://pyro.ai/examples/)

I’m hoping to get this kind of thing going in [Soss.jl](https://github.com/cscherrer/Soss.jl), but that will be summer at the earliest.

---

<div class="post-metadata">

**Author:** ![oliver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oliver/32/19264_2.png) [@oliver](https://discourse.julialang.org/u/oliver)\
**Post date:** [February 7, 2019, 3:43am UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/8 "2019-02-07T03:43:20Z")

</div>

I looked into this a little more and Stan has experimental implementations of both `fullrank` and `meanfield` advi accessible through `CmdStan.jl`.

---

<div class="post-metadata">

**Author:** ![outlace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/outlace/32/2216_2.png) [@outlace](https://discourse.julialang.org/u/outlace)\
**Post date:** [February 7, 2019, 4:43am UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/9 "2019-02-07T04:43:12Z")

</div>

While it doesn’t appear to officially released yet, you can try Gen.jl \< [GitHub - probcomp/Gen.jl: A general-purpose probabilistic programming system with programmable inference](https://github.com/probcomp/Gen) \> Looks like it’s aiming to be a pretty full-featured and sophisticated probabilistic programming language.

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [September 6, 2019, 11:14pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/11 "2019-09-06T23:14:19Z")

</div>

How do the packages Gen.jl and Turing.jl compare?

---

<div class="post-metadata">

**Author:** ![oliver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oliver/32/19264_2.png) [@oliver](https://discourse.julialang.org/u/oliver)\
**Post date:** [September 13, 2019, 2:21pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/12 "2019-09-13T14:21:17Z")

</div>

Here is something related to this discussion: [https://github.com/StanJulia/StanVariational.jl](https://github.com/StanJulia/StanVariational.jl)

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [September 17, 2019, 1:58pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/13 "2019-09-17T13:58:52Z")

</div>

FYI: Variational Inference is now supported in Turing.jl. At the moment you will need to use the master branch, but this will change soon.

---

<div class="post-metadata">

**Author:** ![trappmartin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trappmartin/32/1165_2.png) [@trappmartin](https://discourse.julialang.org/u/trappmartin)\
**Post date:** [September 17, 2019, 2:01pm UTC](https://discourse.julialang.org/t/packages-for-variational-bayes-inference-turing/17904/14 "2019-09-17T14:01:16Z")

</div>

You can find an example of the current ADVI implementation in Turing here: [https://github.com/TuringLang/Turing.jl/blob/master/test/variational/advi.jl](https://github.com/TuringLang/Turing.jl/blob/master/test/variational/advi.jl)

Note that there are some open PRs ([https://github.com/TuringLang/Turing.jl/pull/903](https://github.com/TuringLang/Turing.jl/pull/903), [https://github.com/TuringLang/Turing.jl/pull/902](https://github.com/TuringLang/Turing.jl/pull/902)) which will extend/improve the current state but it is already possible to use ADVI for most Turing models.