Hello. I am trying to use reinforcement learning to solve a simple problem as a proof of concept. I am using DeepQLearning and defining the AbstractEnv(an interface from CommonRLInterface). I am getting the following error and cannot figure out how to make it happy. I believe commonRLInterface and D…

Just tried this in the REPL: julia> Vector{DQExperience{Int32, Float32, Vector{Float64}}}(undef, 10) ERROR: TypeError: in DQExperience, in A, expected A<:(AbstractArray{T<:Real, N} where N), got Type{Vector{Float64}} Stacktrace: [1] top-level scope @ REPL[5]:1 julia> Vector{DQExperience{Int32,…

It’s slightly more subtle. If the rewards are Float64 then your observe should return Vector{Float64} as well.

Also can you create an issue on the github? We should try to automatically convert these to float anyway because neural networks can only take that as input anyway.

Is this an issue for the CommonRLInterface.jl or POMDPs.jl? Or both?

Also, after casting to float with: RL.observe(env::GridWorld) = [Float64(env.state[1]), Float64(env.state[2])] I still get the error: ERROR: LoadError: TypeError: in DQExperience, in A, expected A<:(AbstractArray{T<:Real, N} where N), got Type{Vector{Float64}} I’ve noticed Int64 and Float64 ar…

Consider this: julia> Type{Vector{Float64}} <: AbstractArray{<:Real, N} where N false julia> Vector{Float64} <: AbstractArray{<:Real, N} where N true The issue does not have to do with Float64 <: Real or Vector{Float64} <: AbstractArray{Real} (which is not true). Rather, somewhere some code got p…

Do you mean this: From DeepQLearning, From Prioritized_replay_buffer.jl, function PrioritizedReplayBuffer(env::AbstractEnv, max_size::Int64, batch_size::Int64; rng::AbstractRNG = MersenneTwister(0), …

That line does not seem like the culprit - the parameters should be types. What is the stack trace of the TypeError: in DQExperience error? Look in that trace.

StackTrace: ERROR: LoadError: TypeError: in DQExperience, in A, expected A<:(AbstractArray{T<:Real, N} where N), got Type{Vector{Float64}} Stacktrace: [1] PrioritizedReplayBuffer(env::GridWorld, max_size::Int64, batch_size::Int64; rng::Random.MersenneTwister, α::Float32, β::Float32, ϵ::Float32) …

Hmmm, I guess that must be the line that it is. I guess observe(env) is returning a type instead of an instance?

RL.observe(env::GridWorld) = [Float64(env.state[1]), Float64(env.state[2])] Should be that Vector{Float64}

Using AbstractEnv from CommonRLInterface with POMDPs

Specific Domains Machine Learning

Tyler_Ingebrand September 27, 2021, 7:09pm 16

Results:
[2.0, 8.0]
Vector{Float64}

Topic		Replies	Views
What is the status of ReinforcementLearning.jl Machine Learning reinforcement-learni	24	1296	April 8, 2025
Issue with ReinforcementLearning.jl BasicDQN with custom environment Machine Learning question , error	3	768	June 22, 2021
Error with CircularBufferArrays in ReinforcementLearning.jl Machine Learning question , package , error	5	785	July 11, 2021
Bounds Error when following tutorial of ReinforcementLearning.jl Machine Learning question	0	97	May 8, 2024
LoadError: Can't differentiate loopinfo expression Machine Learning differentiation , flux	8	752	October 4, 2021

Using AbstractEnv from CommonRLInterface with POMDPs

Related topics