Does Julia provide enough support for reinforcement learning now? Is Julia the recommended language for reinforcement learning? Does it have any advantage over Python in this regard? If yes, how to get started to learn and use?
1 Like
There are a few packages like ReinforcementLearning.jl which I think are quite comprehensive now. But I haven’t used it much, so hopefully someone else can comment on this.
For doing RL, or particularly deep RL, you usually just need a neural network library (Flux.jl, Lux.jl or SimpleChains.jl) with an autodiff library. The other side of the equation is creating the environments. This is where Julia really shines, as this is something that isn’t easily done (in a performant way) in Python (if you are writing your own environment).
I have recently been redoing an old project (which used first Python, then C++) in Julia for RL and was able to replicate it in about a day, and it was about an order of magnitude faster to train (thanks to zero allocation training with SimpleChains.jl). I was very impressed.
Which area of RL are you interested in? Deep RL or more traditional methods? Do you want to apply RL to a specific problem, or create a new algorithm to benchmark many environments against?
6 Likes
Frankly speaking, I am a novice in reinforcement learning. I only know some of the most basic principles, and have not really used and practiced them. Now I hope to learn RL and deep RL as quickly as possible, and I hope to practice them with Julia (if it is really suitable, convenient and efficient). Maybe I can use one of the most classic examples as the task. Could you please provide some suggestions? thank you!
That’s great! Best time to learn is now.
If you want a textbook then Sutton and Barto (Sutton & Barto Book: Reinforcement Learning: An Introduction) is the typical textbook. The original version was written a while ago, so most of the book goes over the very fundamentals, but the 2nd edition does cover some deep learning, but it’s not the focys. It’s really good for learning the language of RL and how to think about it. You can jump around the book to see what interests you, or just get a feel for the fundamentals.
I personally think Julia is more suitable than Python or C++ for RL, especially as a beginner.
If you want to start experimenting, jump into the ReinforcementLearning.jl docs (Home · ReinforcementLearning.jl) and just start running examples and experiment with it. I always find doing things practically helps me learn more. The docs have a few tutorials and guides (start from Tutorial · ReinforcementLearning.jl). You can try and create your own environment or game and apply the different algorithms and experiment with different rewards. It’s all good practice.
3 Likes
there is this nice episode of the podcast TalkJulia
that touches upon the subject.
They go over some of the available libraries.
Maybe a good starting point for someone new to the subject?
Enjoy your time learning Julia!
2 Likes
Thank you very very much! You are so kind!
1 Like
Thanks a lot! It’s helpful!
1 Like
The best* use-case for Julia is writing your own code. The best way to learn a (computational) subject is implementing the methods yourself. The choice is obvious 
*source (contains swear, sorry mods)
6 Likes
1 Like