Pkg.jl telemetry should be opt-in

Internals & Design
374

Thanks for the productive discussion and suggestions, everyone. We’ve tossed this around a bit among the Julia committers group, and I think it’s clear that all this requires some additional design work, so here’s how we’re going to proceed:

  1. For 1.5, we will remove the UUID and all other stateful data from Pkg requests. Since there won’t be any tracking, that also obsoletes the opt-in/opt-out question for 1.5, since nothing is sent that allows profiling the user in any way. I have made a pull request that implements this change.

  2. A number of people have pointed out that “telemetry” is a misleading term, because it evokes the kind of all-out behavioral tracking that some of the big tech companies do, which this isn’t. (Technically, telemetry is when you collect data offline and send it later, which doesn’t happen here.) Accordingly, we will be renaming the few remaining non-user-specific headers to “request metadata”. As mentioned above, no data will be sent that can be used to track or profile anyone.

  3. Several alternative estimators have been proposed, but one big problem is that we have no way to determine whether or not such estimators would be effective, or to evaluate any privacy-precision tradeoffs. To help with this, we will keep the opt-out UUIDs on master for now. This gives us more time to evaluate appropriate trade-offs, look at some real data and experiment with alternative approaches, without the time pressure of needing it done in 1.5. With this data, we will be able to simulate several non-UUID based schemes that allow us to estimate population-level usage patterns with real request data and see what the impact on the precision of the final analysis would be. We can’t say yet what the final solution will look like, but there have been several good proposals that we want to look at.

Finally I want to thank people for their level-headed responses and suggestions. While there was some frustration at times, I think by and large people made helpful suggestions along several different axes and we’ll try to incorporate them as we go along.

107 Likes