# How do I check my custom Env in Reinforce.jl?

**URL:** <https://discourse.julialang.org/t/how-do-i-check-my-custom-env-in-reinforce-jl/80808>\
**Category:** Machine Learning\
**Created:** [May 10, 2022, 12:24pm UTC](https://discourse.julialang.org/t/how-do-i-check-my-custom-env-in-reinforce-jl/80808 "2022-05-10T12:24:44Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hupdup](https://avatars.discourse-cdn.com/v4/letter/h/7ba0ec/32.png) [@Hupdup](https://discourse.julialang.org/u/Hupdup)\
**Post date:** [May 10, 2022, 12:24pm UTC](https://discourse.julialang.org/t/how-do-i-check-my-custom-env-in-reinforce-jl/80808/1 "2022-05-10T12:24:44Z")

</div>

Hello all,

I have written a custom enviroment in reinforce.jl and now I want to check if it is correct (i.e. if the actions and the step function work as thought).  
For this I wanted to call the functions one by one in the Julia REPL and execute them one after the other.  
But then it always looks like in the uploaded example (here with the default CartPole environment, which is included in the package).

Does someone know how tho see what the actual values would be or how I can check my environment properly?

Thanks in advance!

Best,  
Joschka

 ![Code in Repl](https://global.discourse-cdn.com/julialang/original/3X/9/f/9f58247d5466f08d3a1ffb18f65a1bd3b4becebd.jpeg)

---

<div class="post-metadata">

**Author:** ![mkschleg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkschleg/32/8035_2.png) [@mkschleg](https://discourse.julialang.org/u/mkschleg)\
**Post date:** [May 11, 2022, 2:29pm UTC](https://discourse.julialang.org/t/how-do-i-check-my-custom-env-in-reinforce-jl/80808/2 "2022-05-11T14:29:33Z")

</div>

You will have to create an instance of the CartPole environment. I’m not quite sure what the constructor is for CartPole (you can likely figure that out through `?CartPole` in the repl to get documentation.

It will be something like:

```julia
env = CartPole()
start!(env)
step!(env, action, ...)

```

You should also be able to get what methods are available for instances of a type with `methodswith(CartPole)`.

---

<div class="post-metadata">

**Author:** ![Hupdup](https://avatars.discourse-cdn.com/v4/letter/h/7ba0ec/32.png) [@Hupdup](https://discourse.julialang.org/u/Hupdup)\
**Post date:** [May 11, 2022, 3:35pm UTC](https://discourse.julialang.org/t/how-do-i-check-my-custom-env-in-reinforce-jl/80808/3 "2022-05-11T15:35:37Z")

</div>

Thanks a lot for your help!!! I never did the env = CartPole() line.

Have a great day!

---

<div class="post-metadata">

**Author:** ![Hupdup](https://avatars.discourse-cdn.com/v4/letter/h/7ba0ec/32.png) [@Hupdup](https://discourse.julialang.org/u/Hupdup)\
**Post date:** [May 12, 2022, 9:12am UTC](https://discourse.julialang.org/t/how-do-i-check-my-custom-env-in-reinforce-jl/80808/4 "2022-05-12T09:12:06Z")

</div>

@mkschleg would you be interested in giving me online tutoring for 1-2 hours?  
My goal is to write a simple DQN Algorithm, which controls the heatpump in a Smart Home with PV energy generation and connection to the grid. I already wrote the env which would be the Smart Home then. I want to get the data for current electricity demand etc. via a CSV (my proplem at the moment is, that I don`t get Reinforce with DataFrames in its dependencys working).  
The Agent will have the 3 decision possibilities: Heatpump\_off, Heatpump\_hotwater, Heatpump\_floorheating.  
If you have the time, I would be very happy for your help to write the matching Algorithm (and double check my env with me)!

Thanks a lot for your help again!

---

<div class="post-metadata">

**Author:** ![mkschleg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkschleg/32/8035_2.png) [@mkschleg](https://discourse.julialang.org/u/mkschleg)\
**Post date:** [May 15, 2022, 7:23pm UTC](https://discourse.julialang.org/t/how-do-i-check-my-custom-env-in-reinforce-jl/80808/5 "2022-05-15T19:23:26Z")

</div>

I unfortunately don’t have time to take on anymore responsibilities/projects/students beyond my current load. I’m in my final(ish) year of my PhD, so have to focus on my research and thesis writing.

You may find it easier to get something working outside of reinforce, and tbh that package hasn’t been maintained in quite awhile and should be archived. I would move to [ReinforcementLearning.jl](https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl).

I also have a repo [MinimalRLCore.jl](https://github.com/mkschleg/MinimalRLCore.jl) that I use to do research. It is much less capable than but it is also a slightly less opinionated and lets me scaffold more quickly. It is more in-line w/ Reinforce with some changes, but doesn’t provide any batteries like ReinforcementLearning.jl does.