Using AbstractEnv from CommonRLInterface with POMDPs

Specific Domains Machine Learning
19

Just tried this in the REPL:

julia> Vector{DQExperience{Int32, Float32, Vector{Float64}}}(undef, 10)
ERROR: TypeError: in DQExperience, in A, expected A<:(AbstractArray{T<:Real, N} where N), got Type{Vector{Float64}}
Stacktrace:
 [1] top-level scope
   @ REPL[5]:1

julia> Vector{DQExperience{Int32, Float32, Vector{Float32}}}(undef, 10)
10-element Vector{DQExperience{Int32, Float32, Vector{Float32}}}:
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef

Apparently, as the code stands, observe needs to return an abstract vector of Float32 because it needs to match the reward type.

In my opinion, this is a bug due to overly constrictive parameterization of the DQExperience type. Here is my proposed fix: https://github.com/JuliaPOMDP/DeepQLearning.jl/pull/64

I would also argue that the way they display the error is a confusing bug and should be fixed. @Tyler_Ingebrand do you want to take a shot at reporting this bug to Julia with my help, or shall I? (trying to teach as many people how to do this kind of thing as possible :slight_smile: )