Hi everyone, first-time user of Turing.jl here. It is really nice, thanks!
I am trying to implement a simple Bayesian logistic regression model, but I keep getting this warning from AdvancedHMC.jl:
┌ Warning: The current proposal will be rejected due to numerical error(s). │ isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false) └ @ AdvancedHMC ~/.julia/packages/AdvancedHMC/MIxdK/src/hamiltonian.jl:47
Is there a good method to debug such situations? The posterior samples in the chain are not updated at all as you can see in the attached image:
Here is the baseline model, which consumes 3D data arrays for conditioning:
@model function logistic(Δᵦ, Δₛ, Δᵣ, I) # Our prior belief θₒ ~ Normal(0.0, 5.0) θᵦ ~ Normal(0.0, 5.0) θₛ ~ Normal(0.0, 5.0) θᵣ ~ Normal(0.0, 5.0) # Data conditioning nx, ny, nt = size(I) for x in 1:nx, y in 1:ny, t in 1:nt z = θₒ + θᵦ*log(Δᵦ[x,y,t] + 1) + θₛ*log(Δₛ[x,y,t] + 1) + θᵣ*log(Δᵣ[x,y,t] + 1) p = 1 / (1 + exp(-z)) I[x,y,t] ~ Bernoulli(p) end end
When I feed the actual data to the model, I make sure that the values in the data arrays are in the range
[0,1] (feature normalization) to avoid vanishing gradients:
samples = sample(logistic(Δᵦ./κ, Δₛ./κ, Δᵣ./κ, I), HMC(0.1, 5), MCMCThreads(), 1000, 3)
The normalization constant
κ above takes care of the normalization.
Do you see any conceptual problem with this model implementation? I can provide examples of 3D data arrays in a Google Drive if you need them to debug this further. I asked the same question in the Turing’s Zulip channel and people suggested using
I[x,y,t] ~ BernoulliLogit(z) directly as this distribution type from Turing.jl avoids corner cases with the logarithm function. I tried the suggestion, but it didn’t change the results.
Appreciate any help.