Julia interface to Bullet Physics?

I came across the Bullet Physics SDK and was impressed by the rich collection of examples. Now I wonder:

  1. Is anybody interested in a Julia interface to the Bullet Physics SDK?
  2. Did somebody already start working on this?
  3. I guess RigidBodySim is the closest alternative in the current julia ecosystem to the Bullet Physic SDK. How do they compare? Would you rather recommend to write an interface to the Bullet Physics SDK or extend the collection of examples for RigidBodySim? I am particularly interested in Reinforcement Learning applications, where Bullet Physics has already many examples.

RigidBodySim and RigidBodyDynamics (Iā€™m the main author) currently have pretty limited support for contact: only point-to-halfspace contacts are supported, with a penalty-based contact model. I have plans for extending that to more contact primitives including general convex polytopes (using GitHub - JuliaRobotics/EnhancedGJK.jl: GJK signed distance algorithm for convex bodies in Julia), but itā€™s currently just not a priority for me.

Iā€™m not aware of the existence of any Julia Bullet bindings.

Edit: sorry, somehow I thought you were specifically asking about contact (I guess because thatā€™s Bulletā€™s forte). What kind of systems are you interested in?

I am interested in having a few control problems with continuous action spaces as benchmarks for reinforcement learning methods. Since problems based on MuJoCo are currently popular in RL I guess it would be nice to have support for contact, but it would be nice to have also other benchmark problems for RL.

OK. I see you have a pendulum and a cart-pole model as examples; something like that is very easy to set up. RigidBodyDynamics can load URDFs, so you could just take the cart-pole URDF from bullet and load it as follows:

urdf = download("https://raw.githubusercontent.com/bulletphysics/bullet3/0e1dce41eab75fd210ec73a52adbf249710c8edf/data/cartpole.urdf", "cartpole.urdf")
using RigidBodyDynamics
cartpole = parse_urdf(Float64, urdf)

which results in

Spanning tree:
Vertex: world (root)
  Vertex: slideBar, Edge: slideBar_to_world
    Vertex: cart, Edge: slider_to_cart
      Vertex: pole, Edge: cart_to_pole
No non-tree joints.

Then, to evaluate the dynamics, you can do:

state = MechanismState(cartpole)
fixedjoint, slidingjoint, pinjoint = joints(cartpole) # unpack joints
configuration(state, slidingjoint) .= 0.1
configuration(state, pinjoint) .= 0.2
velocity(state) .= 0
result = DynamicsResult(cartpole)
dynamics!(result, state)

after which the joint accelerations can be retrieved from result using

julia> result.vĢ‡
2-element RigidBodyDynamics.CustomCollections.SegmentedVector{RigidBodyDynamics.JointID,Float64,Base.OneTo{RigidBodyDynamics.JointID},Array{Float64,1}}:
 -3.29629
  7.39932

And you can follow the RigidBodySim quickstart guide to simulate. If you need gradients, check out this notebook.

Note that there are some visualization changes coming soon in RigidBodySim (https://github.com/JuliaRobotics/RigidBodySim.jl/pull/64): fully switching to MeshCat.jl for visualization, making RigidBodySim quicker to install and available on Windows, among other things. Reducing load time and dropping more dependencies is also on my list.

1 Like

PyBullet is built on pure C API to Bullet. So you should have most Bulletā€™s functionality via ccall. https://github.com/bulletphysics/bullet3/blob/master/examples/SharedMemory/PhysicsClientC_API.h

Iā€™m planning on writing a Rigid Body Dynamics package in Julia focused on efficient contact mechanics. Mainly Iā€™d like to explore some alternative ways of solving the LCP/QP problem that arises.

Iā€™m happy to see the EnhancedGJK package. Good collision detection with penetration depth/direction is one of more time consuming parts of writing a robust RBD engine.

2 Likes

Awesome, thanks a lot.

(I am also looking forward to the visualization changes; was about to give up trying to install the DrakeVisualizer on my Arch Linux machineā€¦)

Cool! Iā€™m glad that more people are starting to think about Julia for robotics/dynamics. Also check out @rdeitsā€™ ConditionalJuMP, especially the Stewart & Trinkle example, as well as GitHub - rdeits/LCPSim.jl (more researchy code), built on top of ConditionalJuMP and RigidBodyDynamics.

1 Like

Looks good. I see those LCP solvers are leveraging either Gurobi ( not free for commercial use ) or Cbc ( restricted to integers? ). Are there are any free for commercial use floating point LCP or Sequential Quadratic Programming solvers you know of? I think Iā€™ve seen an example of SQP using JuMP and non-commercial solvers.

There is an iterative relaxation solver that is specifically used for real-time rigid body dynamics Iā€™ll implement at some point. Itā€™s a bit trickier to efficiently integrate with generalized coordinate solvers, but can be done. The idea there is to lazily update the state vector by starting at the leaves first.
http://www.bulletphysics.com/ftp/pub/test/physics/papers/IterativeDynamics.pdf

Iā€™m using Gurobi because I have access to it (via their academic program) and because Iā€™m familiar with it, but there are other options. In fact, in LCPSim, Iā€™m constructing a linear complementarity problem as a general mixed-integer linear program (for simulation) or mixed-integer quadratic program (for trajectory optimization). For the simulation case, thatā€™s really kind of overkill: a basic LCP solver like PATH would probably be more appropriate. At the time, integrating with PATH required a lot of code changes that I wasnā€™t interested in doing, but it should be much easier with the upcoming versions of JuMP and MathOptInterface. Iā€™m not sure what PATHā€™s license model is, but it canā€™t possibly be as expensive as Gurobi :slightly_smiling_face:

RigidBodyDynamics.jl could actually also be useful for the ā€˜maximal coordinatesā€™ type of approach used in the paper you referenced. Continuing the cart-pole example:

urdf = download("https://raw.githubusercontent.com/bulletphysics/bullet3/0e1dce41eab75fd210ec73a52adbf249710c8edf/data/cartpole.urdf", "cartpole.urdf")
using RigidBodyDynamics
cartpole = parse_urdf(Float64, urdf)
cartpole_maxcoord, _ = maximal_coordinates(cartpole)

which results in a version of the cart-pole with a flat tree structure (each body has a quaternion-parameterized floating joint connecting it directly to the world) and a bunch of non-tree joints that restrict the motion between the bodies:

Spanning tree:
Vertex: world (root)
  Vertex: slideBar, Edge: slideBar
  Vertex: cart, Edge: cart
  Vertex: pole, Edge: pole
Non-tree joints:
slideBar_to_world, predecessor: world, successor: slideBar
slider_to_cart, predecessor: slideBar, successor: cart
cart_to_pole, predecessor: cart, successor: pole

after which itā€™s pretty easy to compute the constraint Jacobian needed in the approach from your paper:

state = MechanismState(cartpole_maxcoord)
rand!(state)
result = DynamicsResult(cartpole_maxcoord)
RigidBodyDynamics.constraint_jacobian!(result, state) # currently not exported, but could be

This updates the constraintjacobian field of DynamicsResult, which is (at least part of) the J you need. constraint_jacobian! is reasonably fast and it doesnā€™t allocate. Itā€™s automatically used internally in the dynamics! function for Mechanisms with non-tree (loop) joints.

Just a suggestion of course, I understand if you want to write it yourself.

Thatā€™s a convenient function. Thatā€™s much less complicated than what I had in mind. Iā€™ll have to study your code, I havenā€™t kept up to date with the RBD algorithms in robotics since the original book from Featherstone :slight_smile: .

What Iā€™ve done in the past is to provide a general interface to the LCP solver where you can query an object for itā€™s ā€œeffective massā€ given a position and a set of basis vectors ( just one needed if no friction ) and then another to apply an impulse along that direction at that point. Which I suppose is a simple model of a contact patch. This allows the LCP solver to work with any object implementing that interface.

I love Bullet3, would be really nice to see it in Julia :slight_smile: Of course if we have a performant Julia version, thatā€™d be even more delightful!

Well, to get things started, I just created a binary builder for Bullet3: Releases Ā· SimonDanisch/Bullet3Builder Ā· GitHub
So if anyone wants to work on wrapping Bullet3 let me know! I can give you acces to:
GitHub - JuliaGeometry/Bullet3.jl: Wrapper for Bullet3 physics engine

1 Like

After BinaryBuilder, just run WrapperBuilder.jl!

Seriously though, are there ways to (semi-)automate that kind of task? I realize there is much more to it than parsing C header files, but I feel a big chunk of it could be automated, and maybe for several target languages in one go.

are there ways to (semi-)automate that kind of task?

have you checked Clang.jl?

Not yet, but that looks like exactly what I meant. Thanks!

you can take VulkanCore.jlā€™s script as an example: https://github.com/JuliaGPU/VulkanCore.jl/blob/master/gen/generator.jl

1 Like

See Sundials.jl. It is wrapped via BinaryBuilder/BinaryProvider to get the binaries and uses Clang.jl to auto-generate a bunch of Julia functions wrapping the whole API.

1 Like

If itā€™s useful, Iā€™ve written up a wrapper for MuJoCo Physics GitHub - klowrey/MuJoCo.jl: Julia wrapper for the MuJoCo Physics Engine

Itā€™s very much research code in that maybe not all features are supported or there are bugs, but Iā€™ve tried to keep it very similar to the C-API interface specified by the mujoco header files, with some Julia specific conveniences. Again, very much research code (Iā€™ve used it to publish some RL papers and played around with some trajectory optimization), so I have not widely announced it, but have wanted to share it with interested folks. Iā€™ll probably include the visualizer when I get it fixed up for Julia-0.7.

Kudos to @tkoolen and @rdeits for turning me onto Julia in the first place!

3 Likes

The visualization changes have been merged (Pkg.checkout for now, no new release yet; still want to fix a couple more issues in the next few days). See the updated quick start guide. Please let us know if there are any issues on Arch and if you have any comments.

Thanks a lot. I did not experience any issues on Arch.