Changing RL Experiment Arguments

Specific Domains Machine Learning
1

This is kind of embarrassing, but, I can’t figure out how to change the seed=123 argument on the DQN_Cartpole experiment.

I threw the code into a Jupyter notebook, with the function in one big cell, then the ex=E'JuliaRL_BasicDQN_CartPole' and run(ex) each in their own cell. Everything runs fine but I want to add more arguments to the function and I can’t figure out how the function is called…

Among other things, I’ve tried:

  • run(ex, seed=42)
  • run(ex(seed=42)
  • run(run(ex), seed=42)
  • ex=EJuliaRL_BasicDQN_CartPole, seed=42

I hate being back to a newbie with a language!

Thanks!

2

I think the way to call this with the additional arguments is either

ex = E`JuliaRL_DQN_CartPole(seed=37)`

if you want to use the string macros (personally I think they mostly confuse things) or

ex = Experiment(Val(:JuliaRL), Val(:DQN), Val(:CartPole), nothing; seed=37)

if you want to use the method call.

If you want to change more things though, I would copy the env into my own function so I can mess with it best I want. So starting from the base file here

you can set up a function to generate your experiment something like this

function my_experiment(;
    seed=123,
    n=1,
    γ=0.99f0,
    is_enable_double_DQN=true
)
    rng = StableRNG(seed)
    env = CartPoleEnv(; T=Float32, rng=rng)
    ns, na = length(state(env)), length(action_space(env))

    agent = Agent(
        ...
    )

    stop_condition = StopAfterStep(10_000, is_show_progress=!haskey(ENV, "CI"))
    hook = TotalRewardPerEpisode()
    Experiment(agent, env, stop_condition, hook)
end

ex = my_experiment(; seed=37)
run(ex)