So, I am trying to stream data from a device over a serial port (RS232 via USB) in realtime. I saw two packages to talk to serial ports, SerialPorts (a wrapper on the Python SerialPorts package) and LibSerialPorts (a wrapper around the C library libserialports). I wanted to avoid the python dependency and any extra lag or risk from that of breakage down the road (which I am perhaps biased to assuming), so I thought LibSerialPort would be the cleaner/lighter option. It has not worked out.
In short, when reading data, it comes through corrupted. It appears bytes are occasionally dropped or changed. I am receiving packeted data with checksums, and about 1/10 have errors in the checksum computation (data is coming through at 100 Hz). While I would love to figure out why that is happening, I didn’t see anything obvious from looking at the Julia code.
Reading the data with the software that came with the device produces no such errors. Redirecting the serial port file (e.g., /dev/ttyUSB0) to save the data and processing it later also has no errors. I even just tried opening the file with open in Julia and read it with no errors. Which brings me to my question - is this a reasonable thing to do? I am concerned there there would be some assumptions that open makes that might break when the file descriptor is for a continually updating serial port, does anyone know? I know I can’t configure the port this way, but the default configuration seems fine.
So, I can still try SerialPorts, but as I am trying to do realtime data reads, I am looking for a cleaner solution with less potential lag. Quick tests indicates it works just fine to just open the file in Julia… any advice from someone more accustomed to working with serial ports?
Thanks!
I use LibSerialPort.jl routinely and rarely have issues, but I think the package may still have rough edges.
If you find issues with it that aren’t replicated in other packages or non-julia serial interface methods, please open issues on LibSerialPort with MWEs
I can do that, but it seems rather device specific and non-actionable. Can’t really do a MWE when it involves errors interacting with hardware.
I have also used LibSerialPort before without issue, but it wasn’t in a streaming context. I don’t know if it is buffer configurations or what, but I have at least isolated to issue to that package, so I am looking for a workaround.
I have seen this behavior before as well, LibSerialPort seemed to occasionally drop specific bit patterns. I had the same problem creating an example, I’m still not sure what is wrong with it.
Julia’s IO system has odd buffering that makes it tricky to use for serial ports.
I ended up using ccall to call open, ioctl, and read directly as needed.
Glad to hear its not just me… I should only need to read the stream from Julia (can use other software for configuration) but would favor reliability. I haven’t used ccall before… do you have any example code for what you did?
Thanks.
I do not have code I can share, but I’ll try to make something. I’ll update this post if I have a chance to make an example.
Could you specify, what you mean by real time (soft or hard limits) and what you aim to do?
In my experience serial communication is rarely used for low latency communication. We have several measurement devices. Communication boils down to “please give a current measurement values”, the device then fills its buffer, which we can read out. Each step takes time and we have no knowledge about how long it takes until the buffer is ready. Loss of data is usually an indicator that the buffer has not jet been filled. Our solution is to wait a bit after we have issued a request. The optimal pause time is different for each device of cause.
You can find a lot of code examples in our package MPIMeasurements.jl, which uses SerialPorts to avoid the python overhead. More specifically, we have a implementation for serial devices (here) as base, which we use to implement communication protocols with specific devices, such as a 3-axis gaußmeter. Once, we one has figured out the pause settings it works like a charm.
@moeddel, I don’t need hard real-time, but it is not a request-reply - the device is streaming out data on a clock cycle (100 Hz), and I am just receiving it. I am currently dealing by computing checksums and dropping bad packets, but I would rather just not have the errors. This is a device that will eventually be embedded in a system using RS232 provide data real time data to a hard real-time control system. For now the serial port is going over USB to a computer and the conditions are relaxed, but the dropped data is a problem.
I don’t know your specific configuration but when I had an IO device streaming packets/lines of data at 100HZ I discovered that the USB serial-io device could not keep up with a heavily loaded system.
I believe the problem was the device driver for the USB-2-serial adapter. When we switched to a “real” serial device without a usb converter, we were able to reliably sustain 100HZ data rates.
This was for a windows system. I don’t know if the issue comes up with other OSes.