Plotting for Julia Reinforcement Learning policies

I have not done much work with reinforcement learning before, so I was looking over some books on RL algorithms etc., The JuliaReinforcmentLearning organization has put together a really nice package with lots of algorithms with which to experiment. My only question was whether there were any good packages or examples for plotting the policies that come out of a reinforcement learning problem. Hence I figured I would ask the community if they have found any good tools or packages to visualize the policies that come out of an RL problem.

I included a simple example of a gridworld below, but there are more complicated examples like cart poles and mountain cars that are on continuous spaces. In the example the space is divided up into squares with arrows and values in each square. I am not sure if there is a package in julia that can handle these types of plots? Like perhaps Javis.jl or something I have not heard of? In the case of gridworld, there are squares with numbers and arrows.

Of course real world RL problems are much more complicated, so it would be nice to know of a package that can handle visualizations like cart poles and mountain cars, etc. I also have some ideas for plotting geographic polygons.

Does anyone know what plotting packages in Julia might work for visualizations like these?