Recently I reproduced most figures on the book Reinforcement Learning An Introduction in Julia. I hope it is helpful to those who are interested in both Julia and RL.
If you also have a background in Python, I’d suggest you to take a rough view of the Python implementation here. Then you’ll find that by using multiple dispatch, it is very easy to make abstractions of different levels in Julia.