Okay, I would implement this outside of Reinforce then.
Create a project folder and add all packages in that environment (Reinforce/DataFrames/CSV). Create your custom RL environment there and have it just import the things it need from those packages.