What is OpenTelemetry.jl ?
OpenTelemetry.jl is a Julia implementation of the OpenTelemetry Specification. It provides a set of unified interfaces to create and collect telemetry data (like traces, metrics, and logs) from your system.
For a long time, we’ve been using stack traces and loggings to analyze and diagnose our systems. The logging state and backtrace are treated as the first citizen in a Task object. And the logging related facilities in Julia are quite rich (see packages under the JuliaLogging organization). However, we’re now living in the age of cloud computing, people are talking about microservices everywhere. We need more flexible tools to track the life circle of each request across different languages, services or processors. That’s exactly where OpenTelemetry.jl fits in. People may argue that Julia is focused on numerical computing, the demand for traces and metrics doesn’t seem to be that strong compared to other web-focused languages. However, I believe every sufficiently mature system must integrate them to improve observability and stability. With more and more companies getting started to adopt Julia in production, I estimate that there will be a much stronger interest in the future. [1]
Call for contributors
One cool feature of OpenTelemetry is drop-in instrumentation. But there’s no free lunch. We package developers have to write instrumentation code for many common packages in the Julia ecosystem (HTTP.jl/Distributed.jl/Genie.jl/Redis.jl and some more). As you may figure it out, that needs a lot of effort. Until now I’ve only implemented a very draft version of API and the SDK in my spare time. [2]
Fortunately, we have an enthusiastic community! Thanks to Reiley Yang’s help, we’ve put forward a proposal to create a special interest group under the OpenTelemetry organization. The original repo will be transferred there when we have at least another 2~3 persons who would like to contribute/review the code.
So if you share the same OTel Mission, Vision, and Values, please reply in this THREAD so that we can work on this project together. I believe the whole Julia community could all benefit from it.
[1]: ServerMetrics.jl is a package open sourced by RelationalAI. It provides two built-in exporters (datalog and prometheus) to collect metrics.
[2]: I started OpenTelemetry.jl this summer when I was trying to prototype a distributed computing system (Oolong.jl). Then I saw @jonathan-laurent also had a similar requirement in AlphaZero.jl#76 So I decided to implement it first with a higher priority. (By the way, I’m open to new challenges and opportunities. Feel free to contact me if my work sounds interesting to you)