# Tabular MDP/Q-learning solver: Reward Matrix R(s, a, s') vs. R(s, a)

**URL:** <https://discourse.julialang.org/t/tabular-mdp-q-learning-solver-reward-matrix-r-s-a-s-vs-r-s-a/87457>\
**Category:** Machine Learning\
**Tags:** machine-learning\
**Created:** [September 19, 2022, 4:52am UTC](https://discourse.julialang.org/t/tabular-mdp-q-learning-solver-reward-matrix-r-s-a-s-vs-r-s-a/87457 "2022-09-19T04:52:50Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Richard\_Wright](https://avatars.discourse-cdn.com/v4/letter/r/ee7513/32.png) [@Richard\_Wright](https://discourse.julialang.org/u/Richard_Wright)\
**Post date:** [September 19, 2022, 4:52am UTC](https://discourse.julialang.org/t/tabular-mdp-q-learning-solver-reward-matrix-r-s-a-s-vs-r-s-a/87457/1 "2022-09-19T04:52:51Z")

</div>

Hello Folks,

I am using the POMDP package along with the TabularTDLearning package

In the first part of the problem I worked out a 3D transition matrix T(s’, a, s) and a 2D reward matrix R(s, a). Then formulated it as a tabular MDP and solved it using the QLearningSolver function which is part of the TabularTDLearning package.

Next, suppose the R-matrix changes as R(s, a, s’), can the the TabularMDP function formulate it? I have my doubts because the documentation states that the R-matrix must be 2D.

How should I handle this kind of R-matrix? Please advice. Thank you.