I’ve written up a new protocol that would allow us to estimate how many unique Pkg clients there are for Julia and for each package in a way that doesn’t allow for tracking individual users. This works as a rough proxy for the number of users of Julia and various packages, so it’s of interest for many reasons, including getting funding for packages. It also allows accurately estimating the number of users sliced by OS, Julia version, package versions, regions, etc. — whatever one may care to count, which can help us understand things like the relative importance of different operating systems and versions, and if we’re under-serving certain parts of the world.
I’ve also made a reference implementation of the whole thing, with comprehensive testing to show that it works, and opened a pull request to Pkg, with an implementation of the client-side part of the protocol. The Pkg servers don’t implement the server-side part yet, and until that happens we cannot test end-to-end.
The protocol is fairly simple to implement, but the theory behind why and how it works is involved to require a significant writeup. Here’s what I wrote in the very top of the “Executive Summary”:
I’m proposing a new (as in novel) way to estimate how many unique clients use Julia—and each package, sliced by OS, version, and region—without being able to track anyone. It stacks three layers:
- Resource-class sharding
- HyperLogLog sampling
- RSA encryption
The first two layers provide the privacy: you can’t follow a client across packages, and every client hides in a crowd of lookalikes in most classes. The RSA layer doesn’t change the privacy model, but stops clients from faking their samples to skew the counts. So if you accept the privacy of sharding + HLL and believe two short proofs about the RSA layer, then you should be ok with the whole protocol.
I’d like to implement the client side of this protocol in Julia 1.14, and then use the data the server collects to publish client-count estimates alongside other Pkg server stats.
What I’d live to get is feedback on this idea and for people to review the design and concept and give feedback. For those who can review the proofs in the writeup, I’d very much appreciate checking of the proof. This has all been vetted by a smallish collection of people who hang out no the JuliaLang Slack #triage channel already, and by Claude (Fable, even), and so far there are no soundness issues.
To answer one question that has come up already on PR review: Do other package managers do this? No, they do not; but they also couldn’t since there isn’t any previously existing protocol to estimate user counts without tracking users. This protocol is new in the sense that I created it specifically to do this.