How to obtain maximized ELBO value from Turing to use as approximation to model evidence?

Specific Domains Modelling & Simulations
,
1

Hi all! I’m running variational inference in Turing. However, I can’t seem to find a way to extract the maximized ELBO from the Turing ADVI. If anyone can point me to relevant information that would be wonderful!

1 Like
2

If there isn’t an easy way to extract this value, is there a way straightforward way to calculate ELBO with given parameter values and observed data?

3

I looked at some info in the tutorials etc and didn’t see an obvious way. I’m on my phone so I can’t look now, but have you tried looking at the code for ADVI and see where it calculates the ELBO? my guess is it should be quite straightforward to use but it’s not in the docs

1 Like
4

Thanks for the tip! I looked into it and turns out Turing interfaces with AdvancedVI.jl, which has a built in function for extracting the ELBO, detailed in its doc.

The doc example only passes in a custom log joint probability function to get the maximized ELBO, but it seems like passing in a Turing model works too, fortunately.

using Turing, AdvancedVI, Turing: Variational
d = 2; n = 100;
observations = randn((d, n));
advi = ADVI(10, 10000)
## turing model
@model function model(y, d)
    μ ~ MvNormal(ones(d))
    y ~ MvNormal(μ, ones(d))
end;

q = vi(model(observations, d), advi)
#I'm not sure yet what the last number does in elbo()
AdvancedVI.elbo(advi, q, model(observations, d), 10000)

The contributors seem to be interested in redesigning it at some point, so the solution above might not be applicable to future readers.

1 Like
5

The ELBO is calculated by drawing a sample and calculating a sum, I’d guess this number might be setting the size of the sample

1 Like
6

I see, that makes sense. Thanks!