# EM algorithm for HMM-GLM models in Julia?

**URL:** <https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136>\
**Category:** Probabilistic Programming\
**Tags:** glm, hmm, markov-decision\
**Created:** [September 6, 2024, 5:15pm UTC](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136 "2024-09-06T17:15:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![PeX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pex/32/49986_2.png) [@PeX](https://discourse.julialang.org/u/PeX)\
**Post date:** [September 6, 2024, 5:15pm UTC](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136/1 "2024-09-06T17:15:47Z")

</div>

Hi all,

Is there anyone here who tried to use Julia for HMM-GLM models using Expectation Maximization?  
I’m looking for good references to try it out. I looked at `HiddenMarkovModels.jl` but I’m not sure how to use it to learn the GLM parameters.

Would be great to hear if someone tried it already.

Thank you!

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [September 6, 2024, 5:27pm UTC](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136/2 "2024-09-06T17:27:29Z")

</div>

Hi, I’m the developer of HiddenMarkovModels.jl. Did you see the tutorial on [Markov-switching regression](https://gdalle.github.io/HiddenMarkovModels.jl/stable/examples/controlled/)? There’s a lot of different terminology but it seems to be close to what you need.  
There’s also a dedicated package called MarSwitching.jl, whose interface might be easier to work with for you (but perhaps less flexible).

---

<div class="post-metadata">

**Author:** ![PeX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pex/32/49986_2.png) [@PeX](https://discourse.julialang.org/u/PeX)\
**Post date:** [September 6, 2024, 6:04pm UTC](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136/3 "2024-09-06T18:04:00Z")

</div>

Wow! Interesting! Thank you for the reply!  
I have some questions just to make sure I’m following:

1. Can I use the control vector as a GLM of the inputs (features)? Is this the idea? Is it possible to use logistic regression GLM too?
2. Can I decide if the features affect both the transition matrix and the weights of the control vector or just one of them?

I like your package as it is very flexible and intuitive, so it would be great to use it for HMM-GLM.

Thank you!

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [September 6, 2024, 7:11pm UTC](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136/4 "2024-09-06T19:11:26Z")

</div>

> [@PeX](#):
>
> Can I use the control vector as a GLM of the inputs (features)? Is this the idea?

Yes, if you have a regression y\_t = f(x\_t, \theta[i\_t]) whose parameters \theta depend on the hidden state i\_t of a Markov model, then in my package docs the “controls” are the regression inputs x\_t and the “observations” are the regression outputs y\_t.

> [@PeX](#):
>
> Is it possible to use logistic regression GLM too?

Yes, but unfortunately for every new controlled model you have to code the estimation procedure yourself. The [“learning” section of the tutorial](https://gdalle.github.io/HiddenMarkovModels.jl/stable/examples/controlled/#Learning) demonstrates this:

- the first part of the `fit!` function is standard transition estimation for HMMs,
- the second part of the `fit!` function estimates \theta[i] for each possible value of i.

The key here is that to recover \theta[i], you use every pair (x\_t, y\_t) but you weigh them individually by the posterior probability of being in state i, which is stored in \gamma\_{i,t}. In the case of linear regression there is an explicit formula (which I hope I got right), for logistic regression you have to use numerical optimization with a slightly adjusted loss function. Does that make sense?

> [@PeX](#):
>
> Can I decide if the features affect both the transition matrix and the weights of the control vector or just one of them?

Yes, this is alluded to in the [“model” section of the tutorial](https://gdalle.github.io/HiddenMarkovModels.jl/stable/examples/controlled/#Model). By overloading `HMMs.transition_matrix(hmm, control)`, you can impose this additional dependency. But then the transition estimation in the `fit!` function must also be adapted manually.

---

<div class="post-metadata">

**Author:** ![PeX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pex/32/49986_2.png) [@PeX](https://discourse.julialang.org/u/PeX)\
**Post date:** [September 6, 2024, 7:31pm UTC](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136/5 "2024-09-06T19:31:03Z")

</div>

Perfect! This is a great answer that got me the info to get started!  
I will read how to optimize the logistic parameters and will try to use your package as it seems like a great ecosystem for it. Thank you for the informative answer!

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [September 6, 2024, 7:42pm UTC](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136/6 "2024-09-06T19:42:29Z")

</div>

Awesome! Ping me again if you struggle.  
The logistic regression alone should be straightforward to code, you just need to find a package that allows you to estimate it with weighted samples (you can do it in MJL.jl for example, see [Weights · MLJ](https://juliaai.github.io/MLJ.jl/stable/weights/)).  
However, if you let your transition matrix depend on the controls, you need to make sure that the resulting model allows easy estimation, because gradient descent is [much harder for stochastic matrices](https://gdalle.github.io/HiddenMarkovModels.jl/stable/examples/autodiff/#Gradient-methods).