[ANN] ProtocGen.jl protobuf support for julia

Package Announcements
1

ProtocGen.jl

CI

Julia code generator for Protocol Buffers.
It is meant to be used as a protoc plugin.
Passes the official required proto2 and proto3 conformance tests for both binary and JSON.

Install

Install protoc-gen-julia binary:

julia> using Pkg
pkg> app add ProtocGen

Usage

cd examples
mkdir out

protoc --julia_out=out addressbook.proto

where

// addressbook.proto
syntax = "proto3";

package tutorial;

import "google/protobuf/timestamp.proto";

enum PhoneType {
    PHONE_TYPE_UNSPECIFIED = 0;
    PHONE_TYPE_MOBILE = 1;
    PHONE_TYPE_HOME = 2;
    PHONE_TYPE_WORK = 3;
}

message PhoneNumber {
    string number = 1;
    PhoneType type = 2;
}

message Person {
    string name = 1;
    int32 id = 2;
    optional string email = 3;
    repeated PhoneNumber phones = 4;
    google.protobuf.Timestamp last_updated = 5;
}

message AddressBook {
    repeated Person people = 1;
}

This generates out/addressbook_pb.jl. It depends on the
ProtocGen.jl package:

include("out/addressbook_pb.jl")

person = Person(
    name = "Alice",
    id = Int32(42),
    email = "alice@example.com",
    phones = [
        PhoneNumber(number = "+1-555-0100", type = PhoneType.MOBILE),
        PhoneNumber(number = "+1-555-0101", type = PhoneType.WORK),
    ],
    last_updated = google_protobuf.Timestamp(seconds = Int64(1_715_000_000)),
)

# Binary wire format
bytes = encode(person)
@assert decode(bytes, Person) == person

# Canonical protobuf JSON mapping (Timestamp renders as RFC 3339)
js = encode_json(person)
# {"name":"Alice","id":42,"email":"alice@example.com",
#  "phones":[…],"lastUpdated":"2024-05-06T12:53:20Z"}
@assert decode_json(Person, js) == person

Alternatives

ProtoBuf.jl is a good alternative.
It is meant to be used as a julia package, not protoc plugin.

Acknowledgement

The wire codec under src/codec/ is copied with light modifications
from ProtoBuf.jl
(MIT-licensed, © 2022 RelationalAI / Tomáš Drvoštěp / contributors).
ProtoBuf.jl is the long-running Julia Protocol Buffers library; this
package takes a different architectural route (descriptor-driven, with
proto3 optional presence and proto2 required semantics fixed) but
stands on its codec. See LICENSE.md.

2

If I recall correctly, Protobuf.jl 0.x was a protoc plugin, and my user experience improved significantly with the standalone 1.x versions.

In what use cases is it more attractive to have a protoc plugin?

3

Great question! As a julia enthusiast, I favor the flexibility of having a julia proto package with detailed julia api to do stuff. As a protoc user for whom Julia is just another language in the stack that some colleagues prefer, I want the protoc plugin.
So there are a bunch of arguments in both directions.

For me, the protoc plugin was the lazier option. Protoc handles the frontend stuff like import resolution etc for you. You mostly need to produce strings from a fully resolved data structure. Also there are standard testing tools for protoc plugins that you can just reuse.
Still, with the help of LLMs a standalone variant could also be coded pretty quickly. If people actually use this package and have strong arguments for a pure julia package variant, I am open to implement that.

If you wonder why I did not use ProtoBuf.jl? I initially tried, I ran into a bunch of issues and the package feels “collectively unmaintained”.

4

Ah, that’s the bigger picture I was missing. The Protobuf package has worked well for me, but I haven’t exactly stressed it much.