DDPG using Flux

Ronis_BR · May 26, 2020, 8:30pm

Hi!

I need to implement DDPG (and TD3) algorithm for Julia using Flux. I saw that we have an initial implementation here:

tejank10/Flux-baselines/blob/master/baselines/ddpg/ddpg.jl

using Flux, CuArrays, Flux.Tracker
using OpenAIGym
import Reinforce.action
import Flux.params
using Distributions: Uniform

#Define custom policy for choosing action
mutable struct PendulumPolicy <: Reinforce.AbstractPolicy
  train::Bool

  function PendulumPolicy(train = true)
    new(train)
  end
end

#Load game environment
env = GymEnv("Pendulum-v0")

# ----------------------------- Parameters -------------------------------------

This file has been truncated. show original

However, the algorithm is very old and does not work with the newest version of Flux. I am wondering if anyone has already implemented a version of DDPG that works with newest Flux. If so, is it possible to provide me some tips? It seems that the Flux API changed a lot.

CarloLucibello · May 26, 2020, 8:49pm

You could try to ask here Add DDPG by findmyway · Pull Request #16 · JuliaReinforcementLearning/ReinforcementLearningZoo.jl · GitHub
what’s missing in the implementation

Ronis_BR · May 26, 2020, 8:51pm

Thanks! I was not find this pull request. This should help a lot