The State of the Julia Ecosystem

Jeff calling the global scope change a debacle is one thing—he was being charmingly self-deprecating. Using his own words to chastise him (for the umpteenth time) seems to me a bit uncharitable and a bit unnecessary given how much the issue has already been discussed. To be clear we do think it’s a debacle but probably not for the reasons you imagine. The debacle from my perspective is that so many people clamored for us to simplify Julia’s scoping rules over the years, so we listened and did. When we did, no one—except for Steve Johnson—complained, until it was too late, at which point everyone decided to make their opinion heard loudly and repeatedly but still too late for that feedback to be of much use. The debacle from our perspective is people demanding things they don’t understand, not providing feedback in a timely fashion, and then complaining that we gave them the thing they asked for in the first place. I realize this is a bit unfair since there were many people who never complained about the old scope behavior, and other people who like the new behavior, so the “people” here is not really a coherent entity, but this is how it looks from our end. So if there’s anything to be learned from the “global scope debacle” it’s that users often don’t understand the consequences of what they’re asking for and we should stand our ground more firmly on issues like this.

Julia 0.7 was always intended to come out at the same time as 1.0, which was communicated by Viral, Keno, myself and others. It was executed as intended and in retrospect I think it went very well. It might have been good to go a little slower between RCs, but only by a matter of weeks—which would have prevented us from releasing 1.0 live at the JuliaCon reception. I think most people who were there will agree that it was well worth it to rush a bit. I certainly have no regrets about that. It was an unforgettable night that generated far greater positive effects in the community than any negative effects that rushing a bit to meet a long-announced deadline may have caused. Also keep in mind that a huge portion of the package ecosystem was upgraded in a single day through the collective efforts of everyone at the JuliaCon hackathon, which would not have happened if we had released a week or two later. It’s entirely possible that if we’d missed the JuliaCon release it would have taken much longer to upgrade packages.

Regarding package compliance, I don’t really understand what you’re referring to—the registry “compliance” claims are correct and have been for months. Pkg.julialang.org was an unofficial best effort thing and has been removed from the website. What is it that’s still incorrect?

21 Likes

Perhaps the structure of these discussions underrepresents people who

  1. complained about the old scoping rules, and
  2. just love the 1.0 semantics and are very happy with the change.

I belong to this group, but I feel that just repeating this sentiment in every topic about the issue is somewhat redundant. Perhaps I should get some “I :heart: #19324 stickers :wink:.

24 Likes

Yes, I do fear you and others in that camp are being underrepresented in the current conversations, which is one of the reasons we wanted to wait and see if the fuss and complaints died down after a while but we still see a steady stream of them. But anyway let’s not turn this into another discussion thread about scope issues.

9 Likes

If you do, let me know and I’ll dm you my address :stuck_out_tongue:.

I was going to say I’m guessing the Venn diagram of “them” complaining and the “they” that asked for the change may have less overlap than it appears (I know both you and Tamas basically said this). Speaking as someone that probably would have been in that overlapping part if I actually followed Julia issues - that is I would have advocated for simpler scoping roles and was one of the people surprised and annoyed by the result - I just wanted to reiterate that after all of the discussion, I’m convinced you all made the right call and the complaints (my own initial complaint included) are more a matter of being used to something different and not something principled.

The fact that you have been able so often to maintain convenience, simplicity, and performance in the language design has us spoiled, such that when you have to sacrifice one for the sake of the others, it’s jarring.

10 Likes

4 posts were split to a new topic: Is Intel ParallelAccelerator.jl still maintained?

A follow up on getting an accurate count of available packages in Julia 1. I am registering DependenciesParser.jl which should make dependency-related utilities quite easy. The current count of installable packages for Julia v1.1.0 is now 1,588. It is still a bit under the advertised 1,900 Julia packages on the website.

If you want the list of packages that are installable for Julia v"1.1", using Julia v"1.1",

using DependenciesParser
pkgs = installable.(DependenciesParser.data) # About 3 minutes
DependenciesParser.data[first.(pkgs)]
8 Likes

By the way, there’s a slight difference between the web site’s “the Julia community has developed over 1,900 Julia packages.” and “there are 1900 registered” packages”, namely that not all packages are registered. :joy:

(I have many unregistered packages, some failures, some still working in Julia 1.0. For example I’m not going to be registering Mellan anytime soon but I don’t want to delete them either :grinning:)

8 Likes

Good point. I took the number of packages I could see in METADATA.jl when I wrote that, and updating it now to 2,400 registered packages for community use. There probably should be other tools that let you explore what this means deeper.

Also updated the download stats to 4 million (3M from julialang.org + 1M docker).

5 Likes

I am responsible for one million of them.

5 Likes

And I have downloaded it once and it is being used by 30 people on a cluster :wink:

7 Likes