I use Libwebsockets.jl, YYJSON.jl, and RDKafka.jl as my Feed Handler (FH), and Redpanda as my Tickerplant (TP) and TP Log File. I do not yet have a Real-Time Database (RDB); however, I am currently implementing a Single Producer - Single Consumer (SPSC) Shared Memory (SM) interface between Julia and q/KDB, with the goal of using KDB-X for this purpose. This is still a work in progress, though most major functionalities already seem to work. For the On-Disk File System (OFS) I use BeeGFS, and, as mentioned above, QuestDB serves as the Historical Database (HDB). Additional components include Grafana and TimeStored Pulse for visualizations, as well as Arroyo for stream processing. In the future I plan to extend the system with Rembus.jl and potentially add FPGA support via XRT.jl. My design philosophy is focused on: (i) connectivity, (ii) flexibility, (iii) usability, and (iv) performance, in that order.
As far as I understand, CEP in Julia is usually implemented with Rocket.jl, a reactive programming framework that simplifies working with asynchronous data. In addition, among other packages, i.e OnlineStats.jl provides high-performance single-pass algorithms for statistics and data visualization.
In general, I believe Julia is very performant. However, the real question may be about GC (Garbage Collection) behavior and latency predictability in concurrent, streaming, or real-time workloads, especially compared to q, which as far as I know, is also garbage-collected. Do you have any information about q/KDB GC behavior and its architecture? By the way, I recently came across the paper Reconsidering Garbage Collection in Julia – A Practitioner Report (de Souza Amorim, Lin, Blackburn, Netto, Baraldi, Daly, Hosking, Pamnany, Smith, ISMM 2025).
I’m also curious, how are you finding Julia’s DataFrames.jl as a time-series and in-memory database alternative? I recently came across Deephaven.io, a query engine and framework for live dataframes that builds data maps using a directed acyclic graph. I wonder whether it would be possible to extend DataFrames.jl with Rocket.jl, Rembus.jl, Dagger.jl, and BonitoBook.jl to create… a slightly more advanced version in Julia?
May I ask, how has the response been so far?