ZMQ.Message super slow?

General Usage
2

Warning: I’m not one of the ZMQ.jl devs. I may be entirely wrong about everything.

I see the same performance on my machine. ProfileView.jl shows that almost all of the time is spent in the gc_protect_handle function, which, according to the comments, is needed to enable zero-copy ZMQ messages which share data with Julia. It looks like the idea is that, rather than copying the data into the message, ZMQ.jl shares the data between Julia and the message, and this gc_protect scheme ensures that the data isn’t garbage-collected until ZMQ is done with it.

But I think that in the case of a tiny message like "1", a copy might be more efficient. This would require changing the constructor of Message in ZMQ.jl to do something like:

  • if the length of the data is less than some threshold, then:
    • instead of calling Message(origin, ::Ptr{T}, len)…
    • call the constructor Message(len::Integer) where len is the number of bytes in the Julia message data
    • get the pointer to the zmq message data by calling the zmq_msg_data C function
    • copy the Julia message data to that pointer
  • otherwise use the current shared-memory behavior

This is probably not too hard to do, and I would encourage you to try it. This could be a great way to make your life better and contribute to Julia. I’m happy to clarify any of my above statements if they’re not clear.

Also, it’s probably worth posting an issue over at ZMQ.jl before diving too deeply into this, since, as I said, I’m not one of the devs and I could be totally wrong.