# \[ANN\] ReinforcementLearning.jl v0.4.0

**URL:** <https://discourse.julialang.org/t/ann-reinforcementlearning-jl-v0-4-0/35348>\
**Category:** Package Announcements\
**Created:** [March 1, 2020, 4:05am UTC](https://discourse.julialang.org/t/ann-reinforcementlearning-jl-v0-4-0/35348 "2020-03-01T04:05:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![findmyway](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/findmyway/32/4946_2.png) [@findmyway](https://discourse.julialang.org/u/findmyway)\
**Post date:** [March 1, 2020, 4:05am UTC](https://discourse.julialang.org/t/ann-reinforcementlearning-jl-v0-4-0/35348/1 "2020-03-01T04:05:19Z")

</div>

Hi all,

I’d like to announce a new release of [ReinforcementLearning.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl). It’s been a really long time since our last release (almost a year and a half). Especially thank @jbrea for his encouragement, I reorganized almost all the code/project structure. Hoping that it will make things easier for people to learn and try different algorithms in Julia.

During the last two years, both the RL related fields and the DL ecosystem in Julia changed a lot. And we witnessed a lot of new RL related packages (both in Julia and Python) created in this period. Today ReinforcementLearning.jl is still at a very early stage. But I’m quite confident that it will have a much broader usage in the future.

# Changes since the last release

- Added a new RL environment [OpenSpiel.jl](https://github.com/JuliaReinforcementLearning/OpenSpiel.jl)
- Examples in [ReinforcementLearningAnIntroduction.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearningAnIntroduction.jl) can be run on [mybinder](https://mybinder.org/v2/gh/JuliaReinforcementLearning/ReinforcementLearningAnIntroduction.jl/master) interactively now. (Thanks to the efforts in [this post](https://discourse.julialang.org/t/ann-mybinder-org-support-for-julia-1-x-and-project-toml/22522))
- Reusability is highly valued and most components can accept a keyword argument named `seed`.
- The performance of some core components are greatly improved.
- Projects in the JuliaReinforcementLearning org is reorganized as bellow:

```
+-------------------------------------------------------------------------------------------+
| |
|[ReinforcementLearning.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl)|
| |
| +------------------------------+ |
| |[ReinforcementLearningBase.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearningBase.jl)| |
| +--------|---------------------+ |
| | |
| | +--------------------------------------+ |
| | |[ReinforcementLearningEnvironments.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearningEnvironments.jl)| |
| | | | |
| | | (Conditionally depends on) | |
| | | | |
| | |[ArcadeLearningEnvironment.jl](https://github.com/JuliaReinforcementLearning/ArcadeLearningEnvironment.jl)| |
| +-------->+[OpenSpiel.jl](https://github.com/JuliaReinforcementLearning/OpenSpiel.jl)| |
| | |[POMDPs.jl](https://github.com/JuliaPOMDP/POMDPs.jl)| |
| | |[PyCall.jl](https://github.com/JuliaPy/PyCall.jl)| |
| | |[ViZDoom.jl](https://github.com/JuliaReinforcementLearning/ViZDoom.jl)| |
| | | Maze.jl(WIP) | |
| | +--------------------------------------+ |
| | |
| | +------------------------------+ |
| +-------->+[ReinforcementLearningCore.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearningCore.jl)| |
| +--------|---------------------+ |
| | |
| | +-----------------------------+ |
| |--------->+[ReinforcementLearningZoo.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearningZoo.jl)| |
| | +-----------------------------+ |
| | |
| | +----------------------------------------+ |
| +--------->+[ReinforcementLearningAnIntroduction.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearningAnIntroduction.jl)| |
| +----------------------------------------+ |
+-------------------------------------------------------------------------------------------+
```

# Next step

- [Alexander Terenin](https://github.com/aterenin) is working on some model-based algorithms 💪 .
- Many policy gradient methods will be added to RLZoo.
- Some basic multi-agent reinforcement learning algorithms will be implemented and may be organized into a seperate project in the next release.

# Call for contributions

I just work on this project in my spare time (And only have plenty of time recently due to COVID-19 in my hometown 😷). So the time and efforts are not guaranteed. Really wish that there be more contributors. I’m really glad to see that some students who are applying for the J(G)SoC contacted me for advice. My suggestion is to take an overview of the [doc](https://juliareinforcementlearning.org/ReinforcementLearning.jl/latest/) first. I think the ideas in this project will still be useful even if you end up without relying on components in it.

Stay safe!

Jun

---

<div class="post-metadata">

**Author:** ![Jon\_Norberg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jon_norberg/32/3218_2.png) [@Jon\_Norberg](https://discourse.julialang.org/u/Jon_Norberg)\
**Post date:** [April 12, 2020, 6:21pm UTC](https://discourse.julialang.org/t/ann-reinforcementlearning-jl-v0-4-0/35348/2 "2020-04-12T18:21:48Z")

</div>

This looks really well structured. I am not knowledgeable enough to contribute, but using it seems very straightforward now! Will try it out!