# ReinforcementLearning Gym spaces

**URL:** <https://discourse.julialang.org/t/reinforcementlearning-gym-spaces/87061>\
**Category:** Machine Learning\
**Created:** [September 11, 2022, 5:04am UTC](https://discourse.julialang.org/t/reinforcementlearning-gym-spaces/87061 "2022-09-11T05:04:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![chunky](https://avatars.discourse-cdn.com/v4/letter/c/ed655f/32.png) [@chunky](https://discourse.julialang.org/u/chunky)\
**Post date:** [September 11, 2022, 5:04am UTC](https://discourse.julialang.org/t/reinforcementlearning-gym-spaces/87061/1 "2022-09-11T05:04:09Z")

</div>

I have a C++ Gym exposed as a C .dll/.so whose API was originally designed to be used for a Python OpenAI Gym using ctypes. Specifically, this only relies on “Box” for both action and observation; the continuous, multidimensional, space. In Python, one creates a “Box” by passing two parallel arrays, one of “high” range values, one for “now”, and the C API aligns with this idea.

For a toy test case, I’ve ginned up an example. The C code is thus:

```nohighlight
int get_action_len() {
        printf("In get_action_len\n");
        int len = get_action_space(NULL, NULL, 0);
        return len;
}

int get_action_space(double obs_low[], double obs_high[], int len) {
        printf("In get_action_space. length=%d\n", len);
        if(NULL != obs_low && NULL != obs_high) {
                obs_low[0] = -1.0;
                obs_high[0] = 1.0;
        }
        return 1;
}

```

And as it stands, this is the Julia code I’ve created:

```julia
function RLBase.action_space(env::DLLGymEnv)
    action_len = ccall(("get_action_len", "../empty_gym"), Cint, ())
    arr_high = Array{Cdouble}(undef, action_len)
    arr_low = Array{Cdouble}(undef, action_len)
    new_action_len = ccall(("get_action_space", "../empty_gym"), Cint, (Ptr{Cdouble}, Ptr{Cdouble}, Cint),
        arr_high, arr_low, action_len)
    # I don't know what to do here to create a thing to return
end

```

What should I be doing to close the loop for the last line? [Obligatory “this is my first time using Julia; I’ve been using C for several decades, python for about 3 years, and Julia for about an hour and a half”]

Thank you !  
Gary

---

<div class="post-metadata">

**Author:** ![chunky](https://avatars.discourse-cdn.com/v4/letter/c/ed655f/32.png) [@chunky](https://discourse.julialang.org/u/chunky)\
**Post date:** [September 12, 2022, 2:40am UTC](https://discourse.julialang.org/t/reinforcementlearning-gym-spaces/87061/2 "2022-09-12T02:40:17Z")

</div>

I figured it out by reading pendulum code. It uses IntervalSets, so my new RLBase.action\_space function is:

```julia
using IntervalSets

function RLBase.action_space(env::DLLGymEnv)
    action_len = ccall(("get_action_len", "../empty_gym"), Cint, ())
    arr_high = Array{Cdouble}(undef, action_len)
    arr_low = Array{Cdouble}(undef, action_len)
    new_action_len = ccall(("get_action_space", "../empty_gym"), Cint, (Ptr{Cdouble}, Ptr{Cdouble}, Cint),
        arr_low, arr_high, action_len)
    Space([(arr_low[i] .. arr_high[i]) for i in 1:action_len])
end

```

Cheers  
Gary

---

<div class="post-metadata">

**Author:** ![chunky](https://avatars.discourse-cdn.com/v4/letter/c/ed655f/32.png) [@chunky](https://discourse.julialang.org/u/chunky)\
**Post date:** [September 13, 2022, 6:46am UTC](https://discourse.julialang.org/t/reinforcementlearning-gym-spaces/87061/3 "2022-09-13T06:46:51Z")

</div>

My code, as it stands, is working for my needs; I’ve pushed it here, for anyone in future with similar questions: [GitHub - chunky/julia\_gym\_dll: Julia glue for a Gym written in C/C++](https://github.com/chunky/julia_gym_dll)

Cheers,  
Gary