# Pkg.jl telemetry should be opt-in
**URL:** https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209
**Category:** Internals & Design
**Created:** [June 28, 2020, 9:11pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209 "2020-06-28T21:11:37Z")
**Posts on this page:** 20
**Page:** 9
### Author:  [@Tero\_Frondelius](https://discourse.julialang.org/u/Tero_Frondelius)
#### Post date: [July 4, 2020, 12:46pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/162 "2020-07-04T12:46:59Z")
> [@chakravala](#):
>
> Funding should not be primarily based on usage stats,
Unfortunately you are not into the position to decide this. One need to play with the given rules or get to the position to change the rules. Open source is not free software, the open source developers need bread and butter to keep coding.
I really don’t understand all of you who are against. Many pointed out that you are giving more data just using the internet. Basically, if you would be really concerned about your privacy you couldn’t participate this discussion, because this discussion happens to be on the internet.
---
### Author:  [@chakravala](https://discourse.julialang.org/u/chakravala)
#### Post date: [July 4, 2020, 12:57pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/163 "2020-07-04T12:57:33Z")
The point I am making about funding is not related to privacy. What I’m saying is that funding should not be determined by usage stats. What if someone invents a highly valuable software only used by a small handful of people? Usage stats (which can be faked also) should never be the primary value indicator for a piece of software.
You are mistaken if you believe that my point about that is related to privacy.
I do have concerns about privacy, but they are separate from my points about funding.
---
### Author:  [@chakravala](https://discourse.julialang.org/u/chakravala)
#### Post date: [July 4, 2020, 1:05pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/164 "2020-07-04T13:05:25Z")
This post was temporarily hidden by the community for possibly being off-topic, unfocused, inappropriate, or spammy.
---
### Author:  [@Tero\_Frondelius](https://discourse.julialang.org/u/Tero_Frondelius)
#### Post date: [July 4, 2020, 1:09pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/165 "2020-07-04T13:09:49Z")
I think I understood your point. Did you understood mine? My point is that it is pointless to complain what criteria someone else than you uses to decide to whom he/she is giving him/her own money. (As an example Gates foundation.) If organization is distributing taxpayer’s money then there are normal democratic ways to affect the criterias and decisions (slow and difficult close to impossible in short term).
---
### Author:  [@chakravala](https://discourse.julialang.org/u/chakravala)
#### Post date: [July 4, 2020, 1:16pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/166 "2020-07-04T13:16:18Z")
It’s not pointless. If i think you are doing a stupid job at allocating financial resources in a free software community like Julia, then i will speak up about it. There are a lot of flaws with relying primarily on usage stats for funding free software. This is especially the case for scientific and technical software, which should not be driven by usage stats, but by scientific value.
We’re not talking about bill gates here, we are talking about funding for mostly scientific free software.
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [July 4, 2020, 2:00pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/167 "2020-07-04T14:00:22Z")
You seem to be missing that this isn’t about the Julia project funding packages, but outside foundations funding the Julia project. You should take this up with numfocus, and other funding agencies.
---
### Author:  [@viralbshah](https://discourse.julialang.org/u/viralbshah)
#### Post date: [July 4, 2020, 2:12pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/168 "2020-07-04T14:12:57Z")
The telemetry information is necessary for much more beyond Julia Computing. Reliable information about packages is extremely useful to package developers, many of whom have chimed in. Such information helps figure out the size of the user base, its growth over time, decisions about deprecations and support for older versions, and so much more. Every package ecosystem collects these stats. For example, here’s [numpy stats](https://pypistats.org/search/numpy), but it is unclear what those numbers mean.
There has been some misunderstanding about funding in the thread, and many have addressed it. I will echo it again. Pkg stats forms a small but useful part of the picture to persuade a reviewer about a project, which is often funded on the basis of innovation proposed, the submitter’s track record, and a bunch of other things. If funds were allocated primarily on the basis of downloads, open source software would not have a sustainability problem.
However, this is not simply about funding. There is a larger problem, especially in academia, where it is hard to justify the impact of your software. Unless you are building something recognizable that a huge number of people use (say Jupyter, numpy, lapack, or even Julia itself), it is extremely hard to convince your department, your dean, your boss, or your colleagues about the impact of your contributions and work. Having quantifiable information makes it part of your CV and your career advancement. This is all the more useful for niche packages, which often do not, by design, have large userbases.
-viral
---
### Author:  [@randyzwitch](https://discourse.julialang.org/u/randyzwitch)
#### Post date: [July 4, 2020, 2:39pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/169 "2020-07-04T14:39:38Z")
> [@chakravala](#):
>
> On the contrary, this only goes to prove that you don’t really care about getting reliable usage data. If the usage data also counts CI usage, then you are not actually counting human users, and the usage stats will be skewed towards CI pipelines.
> [@StefanKarpinski](#):
>
> Serving requests to CI processes is expensive—network bandwidth is the primary cost of running a pkg server, not compute. Telemetry data from CI systems helps understand what people are doing in those automated processes and mitigate those expenses. For example, by deploying package servers that are colocated with CI services (so bandwidth is cheaper or even free)
> [@StefanKarpinski](#):
>
> Knowing which CI services people are using is also helpful for prioritizing quality of support for those CI services. Right now we collectively are good at supporting Travis and AppVeyor because that’s what Julia itself uses, but if we find out from CI variables that a ton of people are using Azure Pipelines, for example, then it may be worth the time and effort to make sure that works really flawlessly in the Julia ecosystem. Without those telemtry headers, we can’t know to spend time and energy on that.
This seems like a perfectly reasonable explanation to me. As open-source software, the least I can do is help the maintainers lower the cost of serving me software for free (though of course, as a GitHub sponsor, I do hope I’m at least covering my bandwidth used every month)
---
### Author:  [@kevbonham](https://discourse.julialang.org/u/kevbonham)
#### Post date: [July 4, 2020, 3:01pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/170 "2020-07-04T15:01:02Z")
> [@chakravala](#):
>
> On the contrary, this only goes to prove that you don’t really care about getting reliable usage data. If the usage data also counts CI usage
Given the information collected (eg environmental variables about CI platforms), it will be very easy to disaggregate CI from not-CI. Making this claim suggests you haven’t actually read the very clear documentation.
> [@chakravala](#):
>
> You say it’s okay without first consulting the community. You should have held a public discussion before even making a decision on what to do.
I don’t follow issues and PRs enough to know when this started to be discussed, but
1. Presumably Stefan didn’t do this by fiat. Given that a number of other core language maintainers on this thread have been speaking out in support of the move, it seems like many people in a position to decide agreed.
2. Given the engagement on this topic, the fact that development happened in the open and that there’s really clear documentation written (all before this thread started), and that a warning is printed the first time you use the new system, are all pretty clear indication that no duplicity is intended.
3. Despite the real collaboration that is fostered by the engagement of the core developers with the community, and the solicitation of feedback and contribution, open source projects are not democracies.
> [@chakravala](#):
>
> So you decided it is okay to automatically collect data on users, where does it stop? In Julia 1.6 you’ll probably keep adding more telemetry.
I haven’t seen any evidence that remotely supports this kind of statement. To the contrary, the clarity of propose, ease of opt-out, and the conversation here are pretty strong evidence against such a claim.
> [@chakravala](#):
>
> If 0% of users want to opt in, maybe that means it is a feature most people would rather not have
This betrays a rather naive understanding of human behavior. Cf. [https://science.sciencemag.org/content/302/5649/1338.full](https://science.sciencemag.org/content/302/5649/1338.full)
> [@chakravala](#):
>
> This opt-out telemetry is a breach of trust for the Julia community, since it’s not clear where this will stop.
You are of course entitled to your opinion, but you do not have a monopoly on what is right. I am satisfied that this decision, especially in light of the implementation and discussion of it, is the right call, nicely balancing competing interests.
Slippery slope arguments, especially of this nature, are pretty lazy. As a vegetarian, I don’t go around saying “well if you’re willing to eat animals, next week you’ll probably start eating people.”
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [July 4, 2020, 3:13pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/171 "2020-07-04T15:13:37Z")
> [@kevbonham](#):
>
> are all pretty clear indication that no duplicity is intended.
I agree with you, but I wonder if it would have been better to have this discussion after a tentative decision has been reached, but _before_ the PR was merged.
Yes, I know, the code is not necesarily final. But hearing about the telemetry (which, again, I fully support and find innocuous) first in a topic like this is just different.
---
### Author:  [@kevbonham](https://discourse.julialang.org/u/kevbonham)
#### Post date: [July 4, 2020, 3:26pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/172 "2020-07-04T15:26:40Z")
> [@Tamas\_Papp](#):
>
> I agree with you, but I wonder if it would have been better to have this discussion after a tentative decision has been reached, but _before_ the PR was merged.
Given the reaction, I think you’re probably right. Certainly, it would be nice to be able to point to a more open discussion that happened before hand. At the same time, if the decision was made, giving the illusion that the community had veto power might actually have been duplicitous. On the other hand, seems like there might be legit design decisions that could have benefited from early community feedback (eg keeping IP addresses separate)."
---
### Author:  [@gcv](https://discourse.julialang.org/u/gcv)
#### Post date: [July 4, 2020, 10:40pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/173 "2020-07-04T22:40:53Z")
This telemetry business has been done on other open-source (CLI) systems, like Homebrew, and was _never_ well-received. See, e.g., [https://news.ycombinator.com/item?id=11566720](https://news.ycombinator.com/item?id=11566720)
To be clear — I get the need to raise money, so I sympathize with the plight of the Julia team. Nevertheless, I do not support opt-out tracking, no matter how well-intentioned, privacy-oriented, or well-designed. If this telemetry will be opt-out-only, I would like to request that Julia at least respect the global `DO_NOT_TRACK` environment variable proposed on [https://consoledonottrack.com](https://consoledonottrack.com)
---
### Author:  [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)
#### Post date: [July 4, 2020, 11:15pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/174 "2020-07-04T23:15:06Z")
Why do I get a feeling that at least some of the loud voices here opposed to the tracking have happily signed over their data to FB, Google, Microsoft, Twitter, …?
---
### Author:  [@dlakelan](https://discourse.julialang.org/u/dlakelan)
#### Post date: [July 4, 2020, 11:31pm UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/175 "2020-07-04T23:31:25Z")
So, it’s hard to even have a phone without at least google or apple involved in your life. However the fact that they are leveraging their position as dominant market-cornering organizations and that people have accepted that it’s necessary to allow this in order to even have a phone in the modern world is not necessarily evidence that people are _happy_ about it.
I don’t let any FB or Twitter apps on my phone. I have turned off location history tracking in google’s phone settings. I look carefully at the permissions requested by phone apps, I use Firefox browser with FB container, and temporary containers. I use DNS over TLS for my home network, I run an open source router, I don’t use SMS instead favoring Signal… I use [startpage.com](http://startpage.com) as my search engine, I don’t allow Chrome on any of my machines though I install chromium for use when certain sites need it. I turn off all the google specific features in chromium. I don’t use Zoom or Google Meet or MS Teams, preferring [https://meet.jit.si](https://meet.jit.si) I don’t use my ISP or google DNS resolver, preferring the privacy focused cloudflare resolver (1.1.1.1) I have some IP cameras, but they are on a totally separate VLAN on my home lan, and that VLAN is 100% firewalled and can’t send or receive packets from the internet, I have a VPN running on my home server so when I’m at a cafe or visiting a campus I can turn on VPN and 100% privatize my traffic… etc
So no, i’m not _happily_ signing over data. I’m ok with Julia doing this tracking if they take a few steps to ensure UUID and IP address are stored separately. Otherwise I"ll opt out.
oh, and I donate cash annually to Julia
---
### Author:  [@jling](https://discourse.julialang.org/u/jling)
#### Post date: [July 5, 2020, 2:33am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/176 "2020-07-05T02:33:33Z")
consider this:
`Pkg.jl` shows you source code, whereas… IDK, I use startpage and firefox FB container too, but like, do you trust google options / android permissions do what they do and not that there are other ways to leak identifiable info?
> **[Google Tracks You Even If Location History's Off. Here's How to Stop It](https://www.wired.com/story/google-location-tracking-turn-off/)**
>
> A new report shows that Google still tracks your location even if you thought you opted out.
> **[AP Exclusive: Google tracks your movements, like it or not](https://apnews.com/article/828aefab64d4411bac257a07c1af0ecb)**
>
> SAN FRANCISCO (AP) — Google wants to know where you go so badly that it records your movements even when you explicitly tell it not to.
Do you trust 1.1.1.1 never collecting anything you didn’t opt-in explicitly? Do you trust Chromium source code more because it’s from Google? and did Chromium ever ask you to opt-in collection? or did you have to turn some of them off?
I think ~50% noise here is unfair to Julia, as if doing it openly and politely only attracts more ‘discussion’ because ~million other sites / software don’t even have lines of code you can point to (for tracking).
---
### Author:  [@dlakelan](https://discourse.julialang.org/u/dlakelan)
#### Post date: [July 5, 2020, 3:27am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/177 "2020-07-05T03:27:49Z")
Yes, clearly Julia is better than all that… My point was just that there _are_ things you can do to reduce data collection, and _I do a lot of them_. The argument that somehow most people offering criticism here are happily giving away their stuff elsewhere doesn’t hold water, at least not for me and probably not for others. Even IP addresses. As a frequent contributor to OpenWrt’s forum I can tell you it’s a daily event for us to get questions about how to set up VPNs so people can hide their IP addresses.
I have **one specific** agenda here which is to ensure that Julia does a good job of actually designing their data collection to achieve their goals… I take the response so far to be constructive, we’ve identified areas of concern such as collecting a UUID and an IP address together being a bad idea. I don’t think this was previously considered in its full implications.
> [@jling](#):
>
> doing it openly and politely only attracts more ‘discussion’
In my opinion, the purpose of doing things openly and politely is to **get discussion**. It’s a bit of a concern that this was designed and inserted with no prior discussion in the forum. The discussion has been mostly productive I think.
---
### Author:  [@stev47](https://discourse.julialang.org/u/stev47)
#### Post date: [July 5, 2020, 6:17am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/178 "2020-07-05T06:17:32Z")
> FB, Google, Microsoft, Twitter
Other wrongdoings don’t justify ones own actions.
Also compared to webservices or social media you normally wouldn’t expect a
locally installed development tool to need to collect and report your personal
information.
---
### Author:  [@c42f](https://discourse.julialang.org/u/c42f)
#### Post date: [July 5, 2020, 8:09am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/179 "2020-07-05T08:09:05Z")
I haven’t been involved in the telemetry discussion or design, but it seems that the Julia-HyperLogLog header is for counting the number of users without resorting to a unique identifier. See [https://julialang.org/legal/data/#hyperloglog](https://julialang.org/legal/data/#hyperloglog) and [HyperLogLog - Wikipedia](https://en.wikipedia.org/wiki/HyperLogLog#Algorithm).
HyperLogLog seems attractive because the numbers sent have a high probability of collision (like uint16 ids would do, but even more so based on the small number of 1024 buckets in the current implementation). It’s also designed to estimate the cardinality of a set _while never storing the elements_ of that set (that is, it’s an approximate, streaming algorithm for set cardinality). The streaming aspect doesn’t seem entirely necessary here, but it does reduce the attack surface even further if the server avoids ever logging that header and does the aggregation online. I don’t think this is necessary mind you, given the high probability of collision.
I personally think opt-out data collection here is completely reasonable as it’s likely to do good for the entire community (in indirectly helping provide development resources which make the ecosystem better), and because it’s minimal enough that nobody has been able to present a _credible harm_ that could come to users even if an attacker were to compromise all telemetry data.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [July 5, 2020, 8:18am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/180 "2020-07-05T08:18:57Z")
> [@dlakelan](#):
>
> The argument that somehow most people offering criticism here are happily giving away their stuff elsewhere doesn’t hold water
I disagree — I think it is a relevant argument.
If someone is willing to accept a lot of sophisticated black-box tracking as the cost of using various social media and web services, I find it somewhat disingenuous to complain so loud about some minimal, transparent, and anonymous telemetry about something that is not interested in your shopping habits, love life, social network, _just the Julia packages you use_, even if it technically does represent a marginal increase in the data collection you are subject to.
Don’t get me wrong, I think it is perfectly fine to grouch a bit about yet another darn thing collecting data. But if one is resigned to all the tracking that big web companies are doing, it becomes difficult to present it as a matter of principle. A single change in the algorithms they are using (that you, of course, _won’t know about_) will probably increase the collection of relevant personal data (ie data you would probably prefer not to be collected) about you orders of magnitude more than this telemetry.
---
### Author:  [@stev47](https://discourse.julialang.org/u/stev47)
#### Post date: [July 5, 2020, 8:28am UTC](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209/181 "2020-07-05T08:28:12Z")
> it seems that the Julia-HyperLogLog header is for counting the number of
> users without resorting to a unique identifier.
HyperLogLog is a great concept, but it’s purpose is somehow defeated when you
send a unique identifier along with it by default.
The Opt-out HyperLogLog key would be more reasonable if the UUID was opt-in.
[Previous page](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209.md?page=8)
[Next page](https://discourse.julialang.org/t/pkg-jl-telemetry-should-be-opt-in/42209.md?page=10)