How can I change specific values in array?

Solution:

solver.Q_vals = zeros(length(states(mdp)), length(actions(mdp)))

Before your code.

Thank you so much.