# Deep Reinforcement Learning Example that will build on Julia 1.0.1?

**URL:** <https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656>\
**Category:** Machine Learning\
**Created:** [October 22, 2018, 10:41pm UTC](https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656 "2018-10-22T22:41:18Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jack\_Vice](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jack_vice/32/6955_2.png) [@Jack\_Vice](https://discourse.julialang.org/u/Jack_Vice)\
**Post date:** [October 22, 2018, 10:41pm UTC](https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656/1 "2018-10-22T22:41:18Z")

</div>

Hi, Can someone recommend a good Deep Reinforcement Learning example code that will build on Julia 1.0.1. I am looking for Actor-Critic, policy gradient or even deep q learning example that works so I can then start doing fun stuff knowing everything builds. I have tried the following without luck.  
[https://github.com/CarloLucibello/DeepRLexamples.jl](https://github.com/CarloLucibello/DeepRLexamples.jl)  
(Gym won’t build)  
[https://github.com/JuliaML/Reinforce.jl](https://github.com/JuliaML/Reinforce.jl)  
(mountain\_car.jl example fails)

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [October 23, 2018, 2:12am UTC](https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656/2 "2018-10-23T02:12:04Z")

</div>

You could try running them with Julia 0.7 to see what deprecations need fixing, then fix those so they run on 1.0. Then make a Pull Request!

---

<div class="post-metadata">

**Author:** ![CarloLucibello](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carlolucibello/32/3278_2.png) [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)\
**Post date:** [October 23, 2018, 9:11am UTC](https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656/3 "2018-10-23T09:11:29Z")

</div>

I just updated [CarloLucibello/DeepRLexamples.jl](https://github.com/CarloLucibello/DeepRLexamples.jl)  
for julia 1.0, please try it out.

You should also check out [https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl),  
I’ve never used it but seems a really nice package.

---

<div class="post-metadata">

**Author:** ![Jack\_Vice](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jack_vice/32/6955_2.png) [@Jack\_Vice](https://discourse.julialang.org/u/Jack_Vice)\
**Post date:** [October 23, 2018, 10:50am UTC](https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656/4 "2018-10-23T10:50:18Z")

</div>

Carlo, Thanks. I have actor\_critic\_pong.jp working! And, thank you for helping to make the Julia community great!

---

<div class="post-metadata">

**Author:** ![CarloLucibello](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carlolucibello/32/3278_2.png) [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)\
**Post date:** [October 23, 2018, 11:07am UTC](https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656/5 "2018-10-23T11:07:17Z")

</div>

thanks! wish I could do much more for the community. Fortunately is plenty of brilliant people around here.

little caveat: the cpu implementation of convolutions in Knet is rather slow, you should experiment much better performances running the code on a gpu

---

<div class="post-metadata">

**Author:** ![Jack\_Vice](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jack_vice/32/6955_2.png) [@Jack\_Vice](https://discourse.julialang.org/u/Jack_Vice)\
**Post date:** [October 23, 2018, 9:52pm UTC](https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656/6 "2018-10-23T21:52:33Z")

</div>

Thanks Carlo. I just use the “Knet Array” type to get GPU to work right? I do have a fast GPU set up with NVIDIA.

---

<div class="post-metadata">

**Author:** ![CarloLucibello](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carlolucibello/32/3278_2.png) [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)\
**Post date:** [October 23, 2018, 10:17pm UTC](https://discourse.julialang.org/t/deep-reinforcement-learning-example-that-will-build-on-julia-1-0-1/16656/7 "2018-10-23T22:17:54Z")

</div>

yep, if you are using KnetArray the deep net is running on the gpu. Have fun!