I’m trying to use the above-named ReinforcementLearning.jl experiment to model training a DQN that calls out to another program and introduces some lag time. I can, of course, run the experiment but I can’t quite figure out where to insert a small sleep() call to simulate the external call. My first…

To add multiple hooks you can combine them into a ComposedHook, e.g. ... hooks = ComposedHook( DoEveryNEpisode((t, agent, env) -> sleep(1)), TotalRewardPerEpisode(), ) Experiment(policy, env, stop_condition, hooks, "# BasicDQN <-> CartPole") ... though with that said, I don’t really unders…

Where in the loop do you want the lag time? You can insert hooks into pretty much all locations in the run loop, where you could then have a hook that simply adds a delay.

I would want to put the lag time in each step. In the real project environment, the call out to the external program is the step… So, add the hook as a parameter to the learner? I’m still really hazy on how all these parts work. Guess hooks are tomorrow’s topic of study.

Ahh, I think I understand what you want now. So adding a sleep will really just make the problem run slower since the environment and the agent are both executed at discrete steps. So I think what you are asking for is that there should be an emulated delay somewhere between the environment (observ…

TYVM for your help! I’ll have to think this through a little more, maybe. The “harder to control” aspect is more a side-effect than a desired outcome. The more complete story is, we are investigating using a DQN in our RL problem but a very rough back-of-an-envelope calculation shows that training w…

Thanks for the reply. I found ComposedHook() today and I think it does what I need. The last part of the story is my boss has been running a ton of Python (PyTorch) RL experiments and he’s apparently found some odd bottlenecks that is causing our current panic. I thought maybe Julia could be an ans…

How to add sleep to "JuliaRL_BasicDQN_CartPole"?

Specific Domains Machine Learning

albheim February 1, 2023, 9:08am 8

Okay, glad you found what you needed and welcome to the Julia community

Topic		Replies	Views
DQN Reinforcement Learning Agent is Not Learning New to Julia question , neural-network	16	1800	April 27, 2022
Python RL environments from Julia Machine Learning question , python	2	933	October 21, 2019
Changing RL Experiment Arguments Machine Learning	1	251	February 1, 2023
I need help with getting a simple Deep QNetwork to learn New to Julia question , flux , neural-network	2	505	January 20, 2023
Reinforcement Learning Package Machine Learning package , announcement	2	2038	October 15, 2018

How to add sleep to "JuliaRL_BasicDQN_CartPole"?

Related topics