# What is the difference between ReinforcementLearning.jl and pomdps.jl

**URL:** <https://discourse.julialang.org/t/what-is-the-difference-between-reinforcementlearning-jl-and-pomdps-jl/82640>\
**Category:** Specific Domains\
**Tags:** question, package, machine-learning\
**Created:** [June 12, 2022, 2:33pm UTC](https://discourse.julialang.org/t/what-is-the-difference-between-reinforcementlearning-jl-and-pomdps-jl/82640 "2022-06-12T14:33:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vamp](https://avatars.discourse-cdn.com/v4/letter/v/f4b2a3/32.png) [@vamp](https://discourse.julialang.org/u/vamp)\
**Post date:** [June 12, 2022, 2:33pm UTC](https://discourse.julialang.org/t/what-is-the-difference-between-reinforcementlearning-jl-and-pomdps-jl/82640/1 "2022-06-12T14:33:16Z")

</div>

Hello,

I have been reading about both packages and also found [this link](https://discourse.julialang.org/t/reinforcementlearning-jl-vs-pomdps-jl/81355), but I don`t see the difference between them. Do you?

---

<div class="post-metadata">

**Author:** ![jbrea](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbrea/32/3879_2.png) [@jbrea](https://discourse.julialang.org/u/jbrea)\
**Post date:** [June 13, 2022, 7:30am UTC](https://discourse.julialang.org/t/what-is-the-difference-between-reinforcementlearning-jl-and-pomdps-jl/82640/2 "2022-06-13T07:30:01Z")

</div>

These packages implement different algorithms. Maybe there is some overlap, but generally the focus of POMDPs.jl is defining and solving POMDPs, i.e. finding optimal policies for a known (PO)MDPs, whereas the focus of ReinforcementLearning.jl is - well - reinforcement learning, i.e. finding an optimal policy for (PO)MDPs with unknown transition and reward probabilities.

Do you need some guidance in picking the right package for a specific problem you would like to solve?

---

<div class="post-metadata">

**Author:** ![vamp](https://avatars.discourse-cdn.com/v4/letter/v/f4b2a3/32.png) [@vamp](https://discourse.julialang.org/u/vamp)\
**Post date:** [June 13, 2022, 7:40am UTC](https://discourse.julialang.org/t/what-is-the-difference-between-reinforcementlearning-jl-and-pomdps-jl/82640/3 "2022-06-13T07:40:28Z")

</div>

Thanks for your answer!

So in case I have a problem I can define as an MDP, it is better to use POMDPs.jl rather than ReinforcementLearning.jl right?

Can I define an MDP using ReinforcementLearning.jl?

---

<div class="post-metadata">

**Author:** ![jbrea](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbrea/32/3879_2.png) [@jbrea](https://discourse.julialang.org/u/jbrea)\
**Post date:** [June 13, 2022, 9:10am UTC](https://discourse.julialang.org/t/what-is-the-difference-between-reinforcementlearning-jl-and-pomdps-jl/82640/4 "2022-06-13T09:10:55Z")

</div>

> So in case I have a problem I can define as an MDP, it is better to use POMDPs.jl rather than ReinforcementLearning.jl right?

Yes. I would start with POMDPs.jl for a known MDP.

> Can I define an MDP using ReinforcementLearning.jl?

Yes. You can use the environment interface to implement an MDP and use the [basic implementation of policy and value iteration in ReinforcementLearning.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/blob/639717388fb41199c98b90406bea76232bc6294d/src/ReinforcementLearningZoo/src/algorithms/tabular/policy_iteration.jl) (see e.g. [the car rental problem](https://juliareinforcementlearning.org/ReinforcementLearningAnIntroduction.jl/notebooks/Chapter04_Car_Rental.html)). This is one place where there is currently a bit of redundancy between those packages 🙂 . I think it would be straightforward to implement the car rental problem in POMDPs.jl.

---

<div class="post-metadata">

**Author:** ![vamp](https://avatars.discourse-cdn.com/v4/letter/v/f4b2a3/32.png) [@vamp](https://discourse.julialang.org/u/vamp)\
**Post date:** [June 13, 2022, 11:20am UTC](https://discourse.julialang.org/t/what-is-the-difference-between-reinforcementlearning-jl-and-pomdps-jl/82640/5 "2022-06-13T11:20:34Z")

</div>

Perfect! I will try POMDPs.jl because I would like to use tabular methods and I do not see these methods on ReinforcementLearning.jl (maybe I am wrong).

Thank you so much for your help.

---

<div class="post-metadata">

**Author:** ![jbrea](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbrea/32/3879_2.png) [@jbrea](https://discourse.julialang.org/u/jbrea)\
**Post date:** [June 13, 2022, 12:31pm UTC](https://discourse.julialang.org/t/what-is-the-difference-between-reinforcementlearning-jl-and-pomdps-jl/82640/6 "2022-06-13T12:31:38Z")

</div>

Perfect, welcome.

In fact, the policy and value iteration methods in ReinforcementLearning.jl _are_ tabular methods. Also the car rental example is tabular (with 21^2 states and 11 actions). But I think the POMDPs interface is better for these kinds of problems and it has more solvers available.