Print the MDP q-matrix

New to Julia
1

Hello,

Need some help please, with printing out the Q-matrix when using MDP and Tabular TD learning.

I have been able to formulate the MDP and the solver and solve the policy. The code is

mdp = TabularMDP(T, R, Discount_factor);
print("\nStep-1: MDP object created")
# use Q-Learning
exppolicy = EpsGreedyPolicy(mdp, 0.02)
q_learning_solver = QLearningSolver(exploration_policy=exppolicy, 
                                n_episodes=1000, 
                                max_episode_length=100,
                                learning_rate=0.5,
                                eval_every=10000, 
                                n_eval_traj=20, 
                                verbose=true);
print("\nStep-2: Solver  working")
policy = solve(q_learning_solver, mdp)
print("\nStep-3: Policy solved \n")

When I run this in debug mode, I am able to see the value table

image

How do I print this to console? I tried using

showpolicy(stdout,"text/plain", mdp, policy)

But this shows the following error.

Exception has occurred: MethodError
MethodError: no method matching showpolicy(::Base.TTY, ::String, ::TabularMDP, ::ValuePolicy{TabularMDP, Matrix{Float64}, Int64})
Closest candidates are:
showpolicy(::IO, !Matched::MIME{Symbol(“text/plain”)}, ::MDP, ::Policy; pre, kwargs…)

What would be the correct way to print the q-matrix to console? Thank you

2

You can check the fields of the policy object yourself. In the REPL, you can write policy. and press tab twice to show the field names, one of which is likely the value table. Then you can print that directly to the console.

Also, could you say which package you are using to do this, to be more familiar with the functions?

3

@jmair Thank you.

I was able to print the value table in REPL by using policy.value_table.

How would I do the same when coding in a file (using VS Code)?

The packages I am using are

using POMDPs, QMDP, POMDPModels, POMDPTools, QuickPOMDPs
using TabularTDLearning
using POMDPModels: TabularPOMDP
4

@jmair

Used the same in VS Code and it’s working. Thanks

1 Like
5

The first two arguments are optional, so you only need showpolicy(mdp, policy) here. (The specific error was because "text/plain" should be passed in as MIME"text/plain"(), but you can just skip the argument instead since that’s the default anyway.)