When doing reinforcement learning tasks, OpenAI Gym is commonly used for experimentation, for example, solving classic problems such as Cart Pole and Pendulum. While ReinforcementLearning.jl also has built-in environments for these problems, I would like to ask whether ReinforcementLearning.jl supports displaying animation effects like the one shown below, and how it can be implemented in Julia?
You can do animations with Makie
https://docs.makie.org/stable/documentation/animation/index.html#animations
Thank you, this is a good thing worth learning! However, what I’m more interested in is whether there is a visualization experimental environment in Julia for testing the effectiveness of reinforcement learning at present?
There is default plotting for a few environments, you can see them here.
So you would probably have to call this yourself each step to get an animation, though I think that should be easy to do with a callback.
Thank you! I will study it. By the way, I would like to ask if there are any detailed explanations for beginners about the built-in environments in ReinforcementLearning.jl?
Me too, thanks!
Just a reminder that this might be very slow since the whole plot has to be regenerated for every step. The strength of Makie animations is that you can choose to only animate certain things, which is much cheaper computationally
That is probably true.
Though I wouldn’t recommend running this for every step of the training anyway, would generate too much data. And just running it for evaluation every now and then it might not matter too much about the speed.
Though we don’t really know what environment OP was interested in, and if they plan on creating their own animation for one. So maybe it could be nice to try in Makie to open up for the higher speed if it should be used in the background.
The pole will shorten when it falls…
Hmm, unclear to me. Never really used the plotting, just remembered having seen it existed.
Looking at the code it is not obvious to me what the problem is, assuming theta=0 means up the plotting of the arm seems sensible to me, so can’t really say why it would behave as described.
Maybe easiest to just roll your own plotting since this seems to not really be well maintained things, no tutorial using them and so on.
Thank you very much! Your reminder is very helpful to me. 