Well over 10,000 Julia packages, with JLLs at 11,549 Julia registered packages. Congratulation Julia community!

Palli · March 29, 2020, 9:23pm

There are finally again over 3000 registered Julia packages (there were more in the past, then those not compatible with Julia 1.0 thrown out of the registry).

This is not yet reflected at Juliacomputing.com while it is at Julialang.org, where I proposed the change at its Github. I had done so in the past (then not realizing jll packages were not wanted in the count), and again recently.

I liked the sound of “over 3,000”, more than “over 2,800”.

Actually there are 3307 packages if you include jll packages, but without them there are 3003 packages and counting (a lot more waiting with PRs to the registry to get them registered).

I’m just amazed by the Julia ecosystem, the breath and depth, not just for e.g. quantum computing (smallest particles) to AstroJulia (Celeste.jl, for largest survey of the whole Universe), but also ExoJulia, seemingly to help in finding exoplanes.

The longest names package names are, some of the very specific packages:

ReinforcementLearningEnvironmentClassicControl
ReinforcementLearningEnvironmentDiscrete
ModifiedHankelFunctionsOfOrderOneThird
PiecewiseDeterministicMarkovProcesses
ReinforcementLearningEnvironmentAtari
ValueOrientedRiskManagementInsurance

Someone asked for the formula for the number of packages, and it’s:

3321 (lines) - 13 - 304 (count of JLLs, by searching for _jll", in the file, with the comma) =

github.com

JuliaRegistries/General/blob/master/Registry.toml

name = "General"
uuid = "23338594-aafe-5451-b93e-139f81909106"
repo = "https://github.com/JuliaRegistries/General.git"

description = """
Official general Julia package registry where people can
register any package they want without too much debate about
naming and without enforced standards on documentation or
testing. We nevertheless encourage documentation, testing and
some amount of consideration when choosing package names.
"""

[packages]
00000000-1111-2222-3333-444444444444 = { name = "REPLTreeViews", path = "R/REPLTreeViews" }
0004c1f4-53c5-4d43-a221-a1dac6cf6b74 = { name = "TuringGLM", path = "T/TuringGLM" }
000d9b38-65fe-4c81-bdb9-69f01f102479 = { name = "BayesHistogram", path = "B/BayesHistogram" }
000eeb74-f857-587a-a816-be5685e97e75 = { name = "KCP_jll", path = "jll/K/KCP_jll" }
001e2b2b-9a5a-4695-8244-d905e27b08dc = { name = "ProgressView", path = "P/ProgressView" }
002418b5-1437-4398-8918-fc420664f8a4 = { name = "LevelDB2", path = "L/LevelDB2" }
0038a6a5-a77d-50b4-8f67-3bd55e463950 = { name = "oneAPI_Level_Zero_jll", path = "jll/O/oneAPI_Level_Zero_jll" }

This file has been truncated. show original

logankilpatrick · March 29, 2020, 10:21pm

Thanks for the PR and it’s truly an exciting time to be in the Julia Community!

Tamas_Papp · March 30, 2020, 8:19am

This is a nice development, but also a signal that we should be thinking about ways of navigating the package ecosystem.

Many registered packages are unmaintained, some are community maintained with varying levels of commitment, while some are experimental. This is fine and natural for a language embedded in the open source ecosystem, but newcomers to the language may find this overwhelming with the current tooling.

tlienart · March 30, 2020, 11:12am

One possible way forward would be to encourage people to add a repo status tl;dr a badge indicating whether the repo is wip/active/… you can get an idea for this by looking at the recent commits on GitHub but it can be difficult to directly distinguish between

stable, not many recent commits, but things are working and there will be support (e.g. PyPlot.jl)
main maintainer(s) have moved on, support is partial at best, probably doesn’t anymore (e.g. Mocha.jl)

PS: a way to get adoption would be to encourage people to use a general quality checker like Aqua.jl in which you could add checks for this kind of stuff

Palli · March 30, 2020, 1:49pm

Good idea, but at least in case of Mocha.jl, it’s actually not in the Registry any more (packages not compatible with 1.0 were dropped), and in addition it states:

“Update Dec. 2018 : Mocha.jl is now deprecated.” (but such is missing in an outdated fork of that project, however it’s easy to see it’s an outdated fork).

tlienart · March 30, 2020, 1:56pm

Yeah I wanted to take a super obvious package so that it didn’t seem like I was pointing the finger…

mschauer · March 30, 2020, 2:35pm

So what is package 3000?

Palli · March 30, 2020, 3:16pm

It seems to GitHub - sisl/AutomotiveVisualization.jl: Visualize automotive simulation from AutomotiveSimulator.jl

It’s actually package number 3301 in the Registry, but disregarding the JLL packages, it seems to be number 3000!

Not disregarding JLLs I found by scrolling through a lot of history, on Jan 30, New package: PeaceVote (2999th), New package: mlpack_jll (3000th) and New package: NiLang (3001st). I think these are from the last time, we got to 3000, not from before the purge (not sure when it happened), and I’m not sure I scrolled by it.

There are a lot of interesting packages, not just those above, also e.g. (I had to check what it was):

mlpack_jll is a wrapper for (yet one more) “fast and flexible C++ machine learning library”
https://www.mlpack.org/

NiLang.jl is A differential eDSL that can run faster than light and go back to the past.

The strangeness of reversible computing is mainly due to our lack of experience with it.—Henry Baker, 1992

Palli · May 1, 2020, 12:29am

Now over 3500 package if you count JLL packages with, strictly 3497 registered (and about 10 packages waiting to be registered).

Juliacomputing.com still only has “Over 2,800 registered packages” on their homepage, not wrong, but I suggest updating. There are now 3133 packages excluding JLL packages. I’m not actually sure if it’s best to exclude them.

Palli · August 16, 2020, 3:57pm

There are now over 4100 (4117 to be exact) registered packages. JuliaComputing.com still shows only “Over 3,500 registered packages”, while Julialang.org shows “over 4,000”. At JuliaCon “4000 packages” was announced, and I had been eyeing the General index regularly waiting for it and they beat me to announcing (as was busy watching JuliaCon when package 4000 dropped).

The 4100th package is Mesa_jll.

The one before, is ConstrainedRootSolvers, then Mesa_jll, Ising2D, then ARFIMA (it includes “any of its subclasses, like e.g. ARMA, AR, ARIMA, etc.”).

The 4000th package was ~~PDSampler.jl~~, EDIT: sorry assumed file in chronological order (it’s in UUID order, and I remember taking it into account when announcing previously), now (again) rather looking at the commit log, counting backwards, to 4100, just don’t have the patience to scroll back 117 packages).

Palli · September 23, 2020, 10:30am

We’re now up to 4314 packages. I want to skip for now introducing package 4300, as it’s an exception, was part of Julia Base.* It’s still very intriguing historically.

The next package, or that whole ecosystem is very interesting “quantum circuit simulation (the “Quant” part) on exa-scale HPC machines (the “Ex” part)”.

Package 4301 is GitHub - ICHEC/QuantExQASM.jl

Circuit composition and OpenQASM translation package for QuantEx project. Creates a backend-agnostic algorithm and circuit composition framework, which can subsequently be used to generate OpenQASM, or tie directly to other quantum assembly languages.

I wasn’t sure I would ever see quantum, and natural language processing (NLP) in the same sentence or package, but here you go, there’s a toolkit for that, with one of their packages: GitHub - ICHEC/QNLP: ICHEC Quantum natural language processing (QNLP) toolkit

See e.g at Wikipedia, and the example program there:

The program adds two four-bit numbers.
// quantum ripple-carry adder from Cuccaro et al, quant-ph/0410184

* Package 4300 is actually GitHub - JuliaAttic/Grisu.jl: The old Base.Grisu module made into a package, the Grisu algorithm is, I’m sure, very intriguing, but it was replaced by Ryu algorithm (in Julia 1.4, and rest of former code removed in Julia 1.6), both were implemented by our great @quinnj known for e.g. CSV.jl.

github.com/JuliaLang/julia

Switch float printing from grisu to ryu algorithm

JuliaLang:master ← JuliaLang:jq/ryu

opened 05:47PM - 05 Aug 19 UTC

quinnj

+2148 -78

Because it's faster and simpler. TODO: - [x] Update the license of ryu cod…e - [x] remove grisu code (might need to chat w/ @StefanKarpinski about printf here, but I think that's the only place we were using grisu internals) - [x] Currently, this prints Float16/Float32/Float64 the same all the time (like we do for `Signed`); personally, I'm not a huge fan of the current `1.0`, `1.0f0`, `Float16(1.0)` inconsistency, so I'd rather just treat them like signed integers and print them all the same. If people think that's too heretical, I can look into keeping the same "typed" logic Will fix #14698 and #22137

History:

github.com/JuliaLang/julia

Grisu

JuliaLang:master ← JuliaLang:jq/grisu

opened 08:24PM - 17 Jun 14 UTC

quinnj

+2658 -186

Implement grisu algorithm in native Julia. Fixes #5959. In particular, this crea…tes a more Julian API to grisu in allowing any size float to be printed in its "shortest" form (while still being parsed back to the correct value), which is a surprisingly tricky problem ([link](http://www.sengupta.net/musings/2012/07/grisu/)). Net code reduction is nice from 6K C++ to about 1K Julia. Initial performance benchmarks show it's just under 2x the native C++. I would also appreciate someone testing this out on a 32-bit machine; I don't foresee any cross-platform issues, but some of the bit-twiddling may cause problems between 32/64-bit.

Net code reduction is nice from 6K C++ to about 1K Julia. Initial performance benchmarks show it’s just under 2x the native C++.

github.com/JuliaLang/julia

Rewrite printf functionality

JuliaLang:master ← JuliaLang:jq/printf

opened 06:13PM - 10 Aug 19 UTC

quinnj

+1315 -2910

Based on https://github.com/JuliaLang/julia/pull/32799. The main commits uniq…ue here rewrite the internal `base/printf.jl` functionality for about half the code, with 2-3x perf improvements. It utilizes the new ryu float printing mechanics in https://github.com/JuliaLang/julia/pull/32799. These commits also remove all the grisu code. Status: this is pretty far along, I haven't done the `%a`/`%A` type specifier yet, or made it work for `BigFloat` either, and there's still a good chunk of tests we need, but it's pretty useable and worth starting to get reviews on.

tlienart · September 23, 2020, 2:40pm

would you be able to get the number of packages with at least one commit in the last 6 months and more than -say- 10 stars? that might give a rough idea of how many active packages are out there.

Zach_Christensen · September 23, 2020, 2:51pm

Depending on the goal of a package it might not need updating within 6 months (granted it would have to be a very focused and simple package).

jling · September 23, 2020, 2:59pm

I second this, e.g:
https://github.com/JuliaMath/Calculus.jl

Palli · September 23, 2020, 3:17pm

There’s a Github API, and GitHub - JuliaWeb/GitHub.jl: A Julia package for interfacing with GitHub seems would be helpful (but some packages are elsewhere, e.g. on Gitlab, while it’s known most are on Github). Someone could look into this (I will not).

There’s now one more package, intriguing one:

it was JLL package number 524. I.e. on-JLL packages are (only) up to 3791.

BLIS is some replacement for [Open]BLAS I believe. It would be nice to know more about how to switch between; and I also want a Julia distribution without either by default (without me compiling from source).

ChrisRackauckas · September 23, 2020, 3:18pm

Active is different from useful though. I’d still say Calculus isn’t a great package but there’s no reason not to use it: it’s mature and it shouldn’t need any updates other than when Julia 2.0 comes out. Getting a count of usable packages would be interesting to me IMO, but that seems hard to define.

Tamas_Papp · September 23, 2020, 3:23pm

Packages updated in the last 6 months or year would give some kind of a lower bound though.

“Activity” is a very weak proxy anyway for “being maintained” (eg the maintainer could approve CompatHelper PRs, while leaving critical issues unaddressed).

All such data is imperfect, but that’s OK. The exact number is not really meaningful but trends could be interesting.

Palli · February 3, 2021, 10:37am

We are up to 5000+ packages, and package 5000 is https://github.com/pbayer/Proquint.jl

At www.modulecounts.com/ Julia count is broken, seemingly because of this line:

https://github.com/edebill/modulecounts/blob/83f4d1f3e148b611780df3c1a4193455d1e09533/db/migrate/20190623190616_julia_pagination.rb#L4

Any idea where to get a similar file in json format or change the code to accept this toml format: https://github.com/JuliaRegistries/General/blob/master/Registry.toml (see also https://github.com/edebill/modulecounts/blob/83f4d1f3e148b611780df3c1a4193455d1e09533/db/migrate/20170402164232_add_julia.rb#L7 )

giordano · February 3, 2021, 10:56am

4 years ago pkg.julialang.org was very different…

Palli · July 15, 2021, 6:05pm

We’re now up to just over 6000 packages, which is a nice round number to achieve before upcoming JuliaCon. And that was a large enough number to break my browser in a sense…

Number 6000 is:

@MKAbdElrahman, I’m not sure, is this somehow related to Microsoft’s rather recent package with similar name and description?

There are now 6016-13 = 6003 packages (based on “6016 lines (6014 sloc)”) calculated from Registry.toml file. Note, you can’t just look at the last line to see the latest package, since the file is not ordered, chronologically, and thus neither count backwards to see which is number 6000. So I go into the git history to find out:

github.com

JuliaRegistries/General/blob/f4082f49e3257cafe1e9de69e7714d59fdc745c0/Registry.toml

name = "General"
uuid = "23338594-aafe-5451-b93e-139f81909106"
repo = "https://github.com/JuliaRegistries/General.git"

description = """
Official general Julia package registry where people can
register any package they want without too much debate about
naming and without enforced standards on documentation or
testing. We nevertheless encourage documentation, testing and
some amount of consideration when choosing package names.
"""

[packages]
00000000-1111-2222-3333-444444444444 = { name = "REPLTreeViews", path = "R/REPLTreeViews" }
000eeb74-f857-587a-a816-be5685e97e75 = { name = "KCP_jll", path = "K/KCP_jll" }
001e2b2b-9a5a-4695-8244-d905e27b08dc = { name = "ProgressView", path = "P/ProgressView" }
0038a6a5-a77d-50b4-8f67-3bd55e463950 = { name = "oneAPI_Level_Zero_jll", path = "O/oneAPI_Level_Zero_jll" }
0047d930-76bf-11ea-1377-bb3cda807e9f = { name = "HeartRateVariability", path = "H/HeartRateVariability" }
004fe424-f3b5-51a0-a814-026e7c39908e = { name = "CUTENSOR_CUDA110_jll", path = "C/CUTENSOR_CUDA110_jll" }
006bdf2e-00d8-555c-bee2-993406487cbf = { name = "gb_jll", path = "G/gb_jll" }

This file has been truncated. show original

and strangely even while seeing the appropriate " 6013 lines (6011 sloc)" I still only saw line 5680 as the last line of the file if I scroll down. It seems I just discovered a bug in Github (for more probably Firefox?), since if I refresh a few times I do get the full 6013 lines, so sometimes the file seems truncated (while footer always ok). Might be good to know, and probably only happens for large files. Can anyone replicate this in Firefox or other browser?

This is rather strange and can also happen (though less frequently?) is I’m just viewing the file (not a specific commit). Note, also Firefox crashed just after, as it does frequently with memory running out, but this keeps happening after restating it. If I do “View Page Source” I seemingly always get the full file, though, but it’s also unclear if by doing so retrieves the file again.