Efficient data access of wrapped C libraries and structs

I’ve been doing RL research recently using the Mujoco physics engine as my robotics simulator. Up until now I’ve just written my own library that exposed just the functionality I needed as a subset of all that Mujoco can do, but I figure it may be a good time to do a re-write with all functionality exposed for the community. The issue I’m trying to wrap around is Mujoco contains all its information in two structs, mjModel and mjData, and how to efficiently interact with the values they contain when they are both pointers. The following is a example of what they look like, but there are a few hundred fields in each struct in the actual library:

// in C:
struct _mjModelExample {
  int nposition;
  int nvelocity;

  mjOption opt; // other c struct
  mjVis  vis;   // other c struct
}
struct _mjDataExample {
  int nbuffer;
  double* time; 
  void* buffer;
  double* positions; // pointers into the void* buffer field
  double* velocity;
}
mjModel* m = mj_loadXML("file.xml");
mjData* d = mj_makeData(m);
mj_step(m, d); // advance the model one time-step; values are stored in d

I can get around the struct-within-struct issue of mjModel by making everything immutable and not changing fields for the time being, but the bigger issue is accessing and swapping values in mjData’s fields, as Julia seems them as Ptr{Cdouble} types. What I’ve thought about doing is replicating the struct’s structure as a Julia type and start doing unsafe_wraps to access the fields.

type mjDataJulia
  nbuffer::Integer
  time::Float64
  positions::Vector{Float64}
  velocity::Vector{Float64}
end
d = unsafe_load(d_pointer_from_c)
m = unsafe_load(m_pointer_from_c)
j_d = mjDataJulia(0, 0.0,
                  unsafe_wrap(Array, d.positions, m.npositions),
                  unsafe_wrap(Array, d.velocity, m.nvelocity))
j_d.positions = j_d.positions .* 0.5
mj_step(m_pointer_from_c, d_pointer_from_c) // Advance time; fields in mjData can be modified and accessed

This way I could ‘write’ into positions and velocity without having to do unsafe_wraps later on, but doesn’t address how the non-pointer fields could be mirrored. The structs are too large to always do copies whenever I need values, so having direct memory access is ideal.

I feel like there’s something I’m missing, however, in that there’s a simple solution staring me in the face that I’m not seeing. Any advice from folks that have encountered these kinds of data structures or this situation before would be tremendously appreciated!

1 Like

Given the large number of fields, have you tried Clang.jl?

Second, replacing individual items will be tricky unless you access the memory locations directly (e.g., with pointer). But in-progress work may soon make that easier.

I did indeed use Clang.jl to generate some files that have been useful to play around with.

What I figure at this point is the correct course of action (albeit tedious), is like I mentioned above, in having a struct/immutable that is the same memory layout as the c-struct, and then doing unsafe_wraps around the immutable into a more julia friendly struct for actual work. I have not see many other wrapped libraries that are as data-heavy as Mujoco can be, so was not confident in the approach I thought of.

Thanks for your response!

I think Cxx.jl might be able to help you here. I honestly haven’t had the opportunity to play with Cxx.jl as much as I would like, but I think you should be able to wrap a C++ pointer in a Julia struct. Then you can write Julia functions that access the fields directly using something like icxx"mjmodel->nposition".

Wow, Cxx.jl provides a pretty attractive interface; you can just evaluate C code as if it were native with what seems to be a slightly separate memory space (still need to unsafe_wrap pointers, etc). I think in my case I’d still have to be clever about exposing the appropriate data fields in the structs, but It looks like the perf hit of parsing the icxx"mjmodel->nposition" line can be exposed as a function:

set_npos(np) = icxx"mjmodel->nposition = $np;"
step() = icxx"mj_step(m, d);"

Definitely worth some investigation! Thanks!

Cxx is amazing and is great for local use or in a controlled environment. But if you want to distribute code more broadly: be aware that Windows isn’t quite supported, and as far as I can tell there isn’t a good solution for package distribution yet.

A lightweight Julia object with wrappers makes sense. It should be fast enough for most purposes. If you can’t afford the extra indirect, then reflecting the structs is still going to be the best option. You can use unsafe_store! and unsafe_load and avoid some manual pointer math with sizeof and fieldoffset.

That’s a good point. I’ve already used Clang.jl to help with the conversion but there’s lots of manual tweaking involved, which I wanted to lazily avoid with CXX :slight_smile:

Better to do the right thing from the start, however. Thanks!