RFC: Proposal for estimating unique Julia clients without tracking

General Usage
, , ,
41

I want to chime in with another perspective to what these kinds of threads are usually full of.
I should state up front that I am in no way associated with JuliaHub - keep that in mind, because if the following text was written by someone from that company it world be public relations suicide.

I am also not writing to any one particular person. Rather, I write to counter a set of ideas that have an outsized influence on these kinds of online discussions. See for example, last this was discussed.

I think that pearl clutching over Julia’s package servers storing a few bits of privacy-preserving information is utterly insane and grounded in delusional and antisocial ideas of what freedom means.
When you’re downloading a Julia package, you’re not exercising some ethereal God-given right from the spirit realm, you are consuming a very worldly resource that other people are paying for with their real money. Thinking that people owe you not just free bandwidth, but cannot be allowed to learn anything about you is analogous to beliving your electricity provider must not only provide you free electricity, but that it’s an affront to your freedom if they have any information about how many customers they have, who they are, or where they are placed.

It is an otherwordly belief that can only spread in communities not tethered to physical reality, because this kind of belief is obviously unworkable in real life. When you go shopping with your balaclava and gloves on (so your neighbors can’t identify you), paying for your yoghurt in unmarked cash (your credit card has your name on it!), does it worry you deeply that the supermarket might do statistics on how many cups of yoghurt they sell? Luckily you don’t have to worry they know that they’re selling it to you - we’re not discussing personal identifiable information in this thread - but the fact that they could have numbers on the amount of yoghurt they sell in each store? The audacity!

I don’t know what Julia’s package infrastructure cost to run just in hardware. Maybe € 100k / year? And of course, Stefan Karpinski probably didn’t write the HLL over RSA protocol in his free time, I’m guessing he was on the clock on a JuliaHub salary while coming up with that and making the PR. Thinking that you’re the person being wronged when the company, in return for all those real, actual resources, want to know who they’re giving the resources to is beyond entitled.

And I do mean “beyond entitled”, because this level of entitlement is not a personal defect, but can only be reached through ideology. An ideology I see too often in online circles, especially those with FOSS people: The belief that “freedom” means you can act in society and affect other people, without the possibility of being held accountable to your actions, or even having to defend or identify with your actions because you can be totally anonymous. It’s the same antisocial ideal that brought us cryptocurrency, anonymous holding companies and shadow banking. This belief is foundationally against what “society” means.

Of course you are entitled to your opinion. And I don’t even think Julia would be better off if people with these beliefs disappeared - Julia is better when there are all kinds of people, including people with stupid political and social ideas. But the idea itself is not worthy of respect, and we should not base our discussion on a set of premises so outside the Overton window that physical reality demands. I’m frustrated that every discussion on privacy is hijacked by these foil hat extremists for whom the idea of nuance is lost.

42

I certainly wouldn’t feel comfortable putting it this strongly, and I have a lot more empathy for the privacy-cautious here, but yeah, I more or less agree with you. The standards and concerns here are at some point quite silly.

Still though, I deeply appreciate @StefanKarpinski’s efforts to find a way to get useful information here in as privacy friendly a way as possible, and being so sensitive to people’s fears.

43

Could we produce two set of downloads: one with the privacy-preserving unique Julia client estimation code enabled and another with the code disabled or even removed? Thus one could opt for one or the other at download.

44

I think this would defeat the purpose of getting an unbiased estimate of usage (because it introduces selection bias)

45

If we allow for an opt-out, would we not also introduce selection bias?

46

All user count estimates will have biases, for example, I have different julia installs for CI, running julia on my university cluster, or debugging or testing Pkg.jl itself. Those probably shouldn’t be thought of as different “users”. I’m not sure how useful a user count is, I care more about the contributor and the dependency graph, which we already have nice statistics for.

47

it would, yes. I would not be in favor of any opt-out. I think the privacy controls here are already extremely good compared to the telemetry attached to pretty much literally any other activity one does online (unless living like the unabomber). I would tend to agree that criticizing this proposal on privacy grounds comes across as naive to me.

48

There is no way to disallow people from opting out, it is just a question of how the opt-out should be implemented.

49

We already try to separate out CI — and to be kind you can make sure that one of those environment variables is set in your CI system.

50

That is life. There absolutely MUST be an opt-out, and I think nobody in their right mind thought that not having an opt-out is an option.

The selection bias is not a problem: The “user-agent”, i.e. Pkg, would report “user declined to participate in hll counting”.

I would not use separate builds, and rather store a config variable somewhere (for persistent opt-out), that can be overridden by some environment variable.

Just imagine what the alternative “implementation” of opt-out would be: Some angry people on discourse post a bunch of patches that includes an opt-out mechanism. “oh, you just need to apply this patch to that package (pray that there is no merge conflict) and build from source”. Linux distros (or e.g. AUR) start shipping their forks that include an opt-out mechanism. Obviously each distro with differently named flags. Security updates get delayed for people on the unmaintained private forks. People get forced into using weird versions.

Ultimately, the spirit of free software is that stuff you run on your computer is serving YOU, and not the developer of the software. This is backstopped by the usual mix of
1.“by the power of your own hands”, i.e. your capability of adversarially forking (or binary patching / modding) the piece of software
2. “by the power of your community”, i.e. your peer’s capability of maintaining forks / software that serves YOUR needs (and your power of community-organizing!)
3. “by politics / laws” (e.g. copyright covering GPL stuff, e.g. GDPR, establishing at least some privacy protections)

Adding some more or less privacy-preserving usage statistics, and enabling that by default, accords to that: It is a nudge towards a tiny act of solidarity with the community that ensures that your computer serves YOU.

Excluding an opt-out to this would be utter insanity.