# Pkg.jl telemetry should be opt-in

**URL:** <https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209>\
**Category:** Internals & Design\
**Created:** [June 28, 2020, 9:11pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209 "2020-06-28T21:11:37Z")\
**Posts on this page:** 20\
**Page:** 7

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [July 3, 2020, 1:44am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/122 "2020-07-03T01:44:28Z")

</div>

> [@StefanKarpinski](#):
>
> How do you estimate a number of users larger than 65k with only that many unique values?

I think the idea is that you use statistics. If everyone is getting a random number you would not expect 65k users to all have a different number. I’m sure there’s some fancy math that @Karajan is referring to that gives more precision, but here’s my naive attempt

```julia
function bootstrap_users(ids, users)
    fraction_ids = Float64[]
    for i in 1:500
        u = rand(1:ids, users)
        push!(fraction_ids, length(unique(u)) / ids)
    end
    return fraction_ids
end

maximum(bootstrap_users(65_000, 65_000)) # 0.635200
maximum(bootstrap_users(65_000, 200_000)) # 0.956615
maximum(bootstrap_users(65_000, 500_000)) # 0.999754

```

So even with 500k users, with only 65k numbers the maximum number used in 500 iterations was 64,984

EDIT: because I couldn’t resist:

 ![Screen Shot 2020-07-02 at 9.59.20 PM](https://global.discourse-cdn.com/julialang/original/3X/0/d/0df141094f6f4471b3ed04c2c865f8f23d9056ad.png)

> **code**
>
> function bootstrap\_users(ids, users)  
> fraction\_ids = Float64  
> for i in 1:500  
> u = rand(1:ids, users)  
> push!(fraction\_ids, length(unique(u)) / ids)  
> end  
> return fraction\_ids  
> end
> 
> u65k = bootstrap\_users(65\_000, 65\_000)  
> u200k = bootstrap\_users(65\_000, 200\_000)  
> u500k = bootstrap\_users(65\_000, 500\_000)  
> u600k = bootstrap\_users(65\_000, 600\_000)  
> u800k = bootstrap\_users(65\_000, 800\_000)  
> u1m = bootstrap\_users(65\_000, 1\_000\_000)
> 
> using StatsPlots
> 
> plot(histogram(u65k, primary=false, title=“65k”),  
> histogram(u200k, primary=false, title=“200k”),  
> histogram(u500k, primary=false, title=“500k”),  
> histogram(u600k, primary=false, title=“600k”),)

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [July 3, 2020, 1:59am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/123 "2020-07-03T01:59:28Z")

</div>

> [@anon67531922](#):
>
> I would love to see the stats for my package. What if the price for me to see the stats about my packages is that I opt in to telemetry?

If the majority of users that needed to be counted had public packages, presumably they could just count contributors to the packages in General.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [July 3, 2020, 2:01am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/124 "2020-07-03T02:01:59Z")

</div>

I’m not 100% sure, but I think that the error bounds of this grow quite large as the number of users increases.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [July 3, 2020, 2:07am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/125 "2020-07-03T02:07:54Z")

</div>

Based on the plots (just added in an edit), it seems pretty tight, though I guess the real thing I should do is model how the fraction of IDs predicts the true number of users rather than the other way around as I’m doing (I’m not a very good statistician 😆). I presume there’s some application of the normal distribution that leads to this conclusion:

> [@Karajan](#):
>
> 65400 unique numbers (of 65535) corresponds to 400k ± 15k unique users. 65500 unique numbers corresponds to 500k ± 30k unique users

But I think the point is that, once the error bounds get to big, you can just increase the maximum. You’d still have enough overlapping IDs that they wouldn’t be useful as identifiers for people, but presumably your error bounds would shrink again.

EDIT: again, just 'cuz.

 ![Screen Shot 2020-07-02 at 10.25.28 PM](https://global.discourse-cdn.com/julialang/original/3X/c/0/c04fa39538b5fa26dd10b26174dad5d3b1974e54.png)

> **code**
>
> ```julia
> function stupid_bootstrap(ids_taken, uids; samples = 200)
> users = Int[]
> for i in 1:samples
> ids = Set(Int[])
> n_users = 0
> while length(ids) < ids_taken
> n_users +=1
> push!(ids, rand(1:uids))
> end
> push!(users, n_users)
> end
> return users
> end
> 
> i60k = stupid_bootstrap(60_000, 65_000)
> i63k = stupid_bootstrap(63_000, 65_000)
> i64k = stupid_bootstrap(63_000, 65_000)
> 
> plot(histogram(i60k, primary=false, title="60k taken", xlabel="mean=$(round(mean(i60k), digits=2)) std=$(round(std(i60k), digits=2))"),
> histogram(i63k, primary=false, title="63k taken", xlabel="mean=$(round(mean(i63k), digits=2)) std=$(round(std(i63k), digits=2))"),
> histogram(i64k, primary=false, title="64k taken", xlabel="mean=$(round(mean(i64k), digits=2)) std=$(round(std(i64k), digits=2))"))
> 
> ```

---

<div class="post-metadata">

**Author:** ![helgee](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/helgee/32/2022_2.png) [@helgee](https://discourse.julialang.org/u/helgee)\
**Post date:** [July 3, 2020, 5:03am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/126 "2020-07-03T05:03:11Z")

</div>

> [@StefanKarpinski](#):
>
> Frankly, I think a lot of people are getting pretty tired of these “click here to make this ass-covering privacy notice go away” popups that now litter the experience of using a computer.

Slight tangent: As an EU citizen, I feel that the EU’s data privacy regulations are a good idea but the implementation leaves a lot to be desired and has a few nasty side effects. Cookie prompts are now even worse than before and stuffed to the brim with dark UI patterns to trick people into opting in, e.g. “necessary cookies” is preselected but the highlighted button will accept all categories and not just the selected one.

**TL;DR:** I fully agree with Stefan that it is much better to only collect the data that is essential to provide your service and be upfront about it.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [July 3, 2020, 5:30am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/127 "2020-07-03T05:30:49Z")

</div>

> [@anon67531922](#):
>
> It would only take a small group of highly vocal people to turn this into a significant issue and an unfortunate distraction.

I am not sure about this; _“Julia package manager collects anonymous package usage data in a transparent and open way — unless you opt out”_ is hardly headline material even for certified nerds.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [July 3, 2020, 5:35am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/128 "2020-07-03T05:35:29Z")

</div>

> [@Tamas\_Papp](#):
>
> _Julia package manager collects anonymous package usage data in a transparent and open way — unless you opt out”_ is hardly headline material even for certified nerds.

you’re assuming accuracy… the headline would read “Open source software spies on users by default” or such.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [July 3, 2020, 5:43am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/129 "2020-07-03T05:43:40Z")

</div>

Outlandish claims can be made about anything; this does not necessarily mean that they will get traction.

Implicitly, I am assuming that the target audience of Julia consists of mostly rational, level headed people who would investigate such claims themselves. As long as we make this easy to do, it will be hard to spin this in a malicious way.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [July 3, 2020, 5:45am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/130 "2020-07-03T05:45:53Z")

</div>

> [@Tamas\_Papp](#):
>
> target audience of Julia consists of mostly rational, level headed people

and their boss, their consulting clients, insurance companies, lawyers…

---

<div class="post-metadata">

**Author:** ![heliosdrm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/heliosdrm/32/3851_2.png) [@heliosdrm](https://discourse.julialang.org/u/heliosdrm)\
**Post date:** [July 3, 2020, 5:48am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/131 "2020-07-03T05:48:15Z")

</div>

> [@anon67531922](#):
>
> v1.5 rc is already in the wild and there isn’t even a draft privacy document for us to have a look at.

This has cited several times in this thread:  
[https://julialang.org/legal/data/](https://julialang.org/legal/data/)

> [@anon67531922](#):
>
> it would suck if a big negative headline about Julia’s data privacy policies hit the newspapers.

As others have pointed out, that would be fallacious and ill-intentioned exaggeration. If someone is willing to ruin the credit of Julia spreading out lies, even removing all telemetry could be an event to make up a negative headline.

---

<div class="post-metadata">

**Author:** ![anon67531922](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@anon67531922](https://discourse.julialang.org/u/anon67531922)\
**Post date:** [July 3, 2020, 6:31am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/132 "2020-07-03T06:31:28Z")

</div>

> I am not sure about this; _“Julia package manager collects anonymous package usage data in a transparent and open way — unless you opt out”_ is hardly headline material even for certified nerds.

The risk could be close to zero - I have no idea - but I fear there could be a risk of something real happening, e.g. some data breach or some unforeseen (as of yet) issue that is real and headline worthy. If a hacker somehow accessed all the collected telemetry data, is there something that could be done with it? I don’t know. What could happen if UUIDs linked to IP addresses were released accidentally? I have worked on projects linking “anonymous” data to personal identifying data. It isn’t that hard. If there is no risk at all, that is great. If someone could explain the potential risks, if any, that would be good.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [July 3, 2020, 6:39am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/133 "2020-07-03T06:39:24Z")

</div>

I second Eric’s question from above - how is the separation of IP addresses and UUID handled? It seems to me that the combination of both is the potentially (not saying actually!) most problematic aspect of this - apologies if I have missed an explanation somewhere in this long thread, but I can’t find IP addresses mentioned in the data policy section of the Julialang website linked above.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [July 3, 2020, 6:54am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/134 "2020-07-03T06:54:54Z")

</div>

> [@dlakelan](#):
>
> and their boss, their consulting clients, insurance companies, lawyers…

… can be directed to [the details](https://julialang.org/legal/data/), read through them, and move on (in addition, the lawyers will be delighted to send an invoice, thank you so much).

Dilbert cartoons notwithstanding, the private sector is not completely staffed with gullible idiots. If/when the question of Julia’s package telemetry comes up, it will be reviewed by people who have seen a quite a few data collection policies, 99% of which are orders of magnitude less transparent and innocuous.

---

<div class="post-metadata">

**Author:** ![Karajan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/karajan/32/8545_2.png) [@Karajan](https://discourse.julialang.org/u/Karajan)\
**Post date:** [July 3, 2020, 7:01am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/135 "2020-07-03T07:01:55Z")

</div>

Yes, pretty much what @kevbonham wrote: some statistics, relying on the fact that is highly unlikely _not_ to get clashes (birthday paradox) and therefore not use up all the available random numbers.  
I took `N` random `UInt16`s and looked how many unique ones are in there on average, plus a 90% quantile. Then calculate back `N` from the unique IDs the server would see.

 ![fuzzyIDs](https://global.discourse-cdn.com/julialang/original/3X/d/9/d9e494e644049a012c7282a42184838a877e836f.png)

> **Code**
>
> ```julia
> using StatsBase
> using Plots
> 
> uniquerands(i) = rand(UInt16, i) |> unique |> length
> 
> function collect_stats(rng)
> len = length(rng)
> means = zeros(len)
> low = zeros(len)
> up = zeros(len)
> Threads.@threads for i in 1:len
> ur = Int[]
> for _ in 1:500
> push!(ur, uniquerands(rng[i]))
> end
> means[i] = mean(ur)
> l, u = quantile(ur, (0.05, 0.95))
> low[i] = l
> up[i] = u
> end
> means, low, up
> end
> 
> i = 300_000:10_000:600_000 |> collect
> s = collect_stats(i)
> 
> plot(i, s[1], ribbon = (s[1] .- s[2], s[3] .- s[1]), label = "mean with 90% quantile")
> hline!([typemax(UInt16)], label = "typemax(UInt16)")
> scatter!([405_000, 495_000], [65400, 65500], xerr = [10_000, 20_000], label = "estimate for unique users")
> plot!(legend = :bottomright, ylims = (65_000, 65600), xlabel = "unique users", ylabel = "unique IDs")
> 
> ```

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [July 3, 2020, 7:48am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/136 "2020-07-03T07:48:41Z")

</div>

As long as the collected data is as minimal as it is and contains **no personal** data I would say that the IP adress together with the set of telemetry data is no problem even if it stays connected to the telemetry data. In other scenarios the IP adress would be the connection to an individium to e.g. anonymized **personal** data and this would be a problem.

But despite that I think the IP adress should be mentioned in the [data document](https://julialang.org/legal/data/) as it is clearly part of the gathered data even if it is not stored together with the telemetry data.

---

<div class="post-metadata">

**Author:** ![stev47](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stev47/32/14090_2.png) [@stev47](https://discourse.julialang.org/u/stev47)\
**Post date:** [July 3, 2020, 8:54am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/137 "2020-07-03T08:54:43Z")

</div>

Reliable reidentification [has been performed](https://www.nature.com/articles/s41467-019-10933-3) with less unique data than IP address and telemetry. The key is that linking data together gives you much more information that you can imagine. Just link the data about fetched packages together with time and public git commits to `Project.toml` files on github and maybe his public bug reports where he posted his `versioninfo()` and I’m sure you can infer the person whom a UUID belongs to in quite a few cases.

This discussion is less about whether people at Julia are to be trusted or not, but about whether collecting not strictly necessary data that has the potential to be misused is a good idea or not.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [July 3, 2020, 9:02am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/138 "2020-07-03T09:02:15Z")

</div>

> [@StefanKarpinski](#):
>
> How do you estimate a number of users larger than 65k with only that many unique values?

You use a non-uniform distribution for the random ID:s. For example, the [HyperLogLog](https://julialang.org/legal/data/#hyperloglog) distribution, as pointed out by @chrisvwx [above](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/8).

(This is not perfect. Some users will get an ID that is so rare that it effectively identifies that user uniquely, and if those users are more likely to opt-out (or get a new ID) than the average user, then that will skew the statistics.)

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [July 3, 2020, 9:40am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/139 "2020-07-03T09:40:16Z")

</div>

I am with you. I don’t think that the IP adress should be collected. At first it is collected as part of any protocoll, so this is fact. Therefor it should be mentioned in the data document. And it should be discarded and how this is done should also be part of the description.

This is easy to do which makes it even more important.

But, what I also think: it is from a privacy viewpoint **in this case** not such a big deal as there is actually **no personal data**. IP adress can be personal data but it isn’t per definition.

> Just link the data about fetched packages together with time and public git commits to `Project.toml` files on github

And this is not part of the telemetry we are talking about. It can be done, sure. But if you argue with that we all have to go offline right now!

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [July 3, 2020, 10:12am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/140 "2020-07-03T10:12:33Z")

</div>

> [@stev47](#):
>
> I’m sure you can infer the person whom a UUID belongs to in quite a few cases.

So, if I understand correctly, the worst case scenario is that people who have access to the disaggregated data can learn which packages from the general registry I have installed.

Sorry to be dense, but I still don’t understand why this is a big deal. Yes, I can imagine misuse scenarios (eg math book publishers getting this data for targeted advertising based on my preferences for various numerical methods), but they all seem pretty far-fetched given the payoffs. Technically, when de-anonymized this would constitute personal data, but only in the sense of some people worrying about citric acid as a food additive.

---

<div class="post-metadata">

**Author:** ![stev47](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stev47/32/14090_2.png) [@stev47](https://discourse.julialang.org/u/stev47)\
**Post date:** [July 3, 2020, 10:57am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/141 "2020-07-03T10:57:08Z")

</div>

Not saying these are relevant to most people, but a few things I have heard to be possible:

- IP and timings give off your location, maybe up to your employer and people close to you (when linking)
- timings give off your usual working hours, might disclose when you are on vacation (maybe inferring when your house is vacant)
- Version/System information together with package versions give off information about potential vulnerabilities that an attacker can use (there are databases linking software versions with corresponding exploits)

[Previous page](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209.md?page=6)

[Next page](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209.md?page=8)