# \#markov-decision

**URL:** https://discourse.julialang.org/tag/markov-decision/1313.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [EM algorithm for HMM-GLM models in Julia?](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136)

<div class="topic-metadata">

**Author:** [@PeX](https://discourse.julialang.org/u/PeX)\
**Replies:** 5\
**Last updated:** [September 6, 2024, 7:42pm UTC](https://discourse.julialang.org/t/em-algorithm-for-hmm-glm-models-in-julia/119136 "2024-09-06T19:42:29Z")

</div>

Hi all, Is there anyone here who tried to use Julia for HMM-GLM models using Expectation Maximization? I’m looking for good references to try it out. I looked at HiddenMarkovModels.jl but I’m not sure how to use it to …

---

## [Best library for particle filtering from a generative model?](https://discourse.julialang.org/t/best-library-for-particle-filtering-from-a-generative-model/115070)

<div class="topic-metadata">

**Author:** [@JasonPekos](https://discourse.julialang.org/u/JasonPekos)\
**Replies:** 2\
**Last updated:** [June 2, 2024, 6:06pm UTC](https://discourse.julialang.org/t/best-library-for-particle-filtering-from-a-generative-model/115070 "2024-06-02T18:06:03Z")

</div>

Hi! I’m looking for a library that will let me perform particle filtering for a generative model of a markov process, similar to the incredible pomp in R: or particles in Python. Basically, I want to specify some sor…

---

## [\[pre-ANN\] CompressedBeliefMDPs.jl: Solve large POMDPs in POMDPs.jl!](https://discourse.julialang.org/t/pre-ann-compressedbeliefmdps-jl-solve-large-pomdps-in-pomdps-jl/112625)

<div class="topic-metadata">

**Author:** [@FlyingWorkshop](https://discourse.julialang.org/u/FlyingWorkshop)\
**Replies:** 0\
**Last updated:** [April 7, 2024, 4:28am UTC](https://discourse.julialang.org/t/pre-ann-compressedbeliefmdps-jl-solve-large-pomdps-in-pomdps-jl/112625 "2024-04-07T04:28:35Z")

</div>

https://github.com/JuliaPOMDP/CompressedBeliefMDPs.jl Hello everyone! I’m excited to announce CompressedBeliefMDPs.jl (docs on repo), an upcoming addition to the POMDPs.jl ecosystem. CompressedBeliefMDPs.jl provides a …

---

## [How to solve a Markov decision process with randomness](https://discourse.julialang.org/t/how-to-solve-a-markov-decision-process-with-randomness/96426)

<div class="topic-metadata">

**Author:** [@Jian\_ZUO](https://discourse.julialang.org/u/Jian_ZUO)\
**Replies:** 0\
**Last updated:** [March 22, 2023, 12:24am UTC](https://discourse.julialang.org/t/how-to-solve-a-markov-decision-process-with-randomness/96426 "2023-03-22T00:24:21Z")

</div>

Hi, I have a Markov decision process as follows: states: S1, S2; the increment of S1 follows a Gamma law, i.e. S1(t+d)-S1(t) ~ gamma(alpha, beta), S1 has a fixed range of zero to a failure threshold FT (thus the transi…