After some years of getting the package server architecture in place and working (mostly) reliably, @staticfloat and I finally had some time to work on collecting logs into a data warehouse (we’re using Snowflake ) and designing a set of queries over the logs that we can run and publish regularly. Th…

Yes, that’s something I want to do. Currently the code is all in snowflake worksheets. I’d prefer to use something that’s in a git repo, but haven’t figured out a good workflow with snowflake and git yet. But yes, the SQL queries should be public and people can propose new ones that we can add. Havi…

This data should be added to the badges that Juliahub currently exposes, soon (ish)

I guess we’d want to filter out downloads by GitHub Actions and other CI systems, if possible (based on IP)?

[image] oschulz: filter out downloads by GitHub Actions and other CI systems [image] StefanKarpinski: without any indicators of being a CI process.” I believe this has been done to some extent?

Yes, the Julia client sends indicators of whether environment variables that are set by CI services are present or not and we check for those to decide if something is a CI request.

Here’s the data for last month with UUIDs converted into package names if anyone is curious and don’t feel like wrangling the data themselves https://gist.github.com/baggepinnen/e05e8dc6989980f558691d9ce6771801 The first columns are sorted based on number of users and the last columns based on org…

[image] StefanKarpinski: *users = “Number of unique IP addresses which requested said package without any indicators of being a CI process.” To be clear, these numbers refer to unique IPs installing the package within the time window, right? So stable packages with unfrequent releases won’t b…

this is cool. here’s my suggestions: create a github repo with FAQ, Docs, etc and where everyone can contribute their scripts or codes to extract, analyze, cluster, predict, visualize the data have a list of questions we want to know given the datasets so that some of these scripts who have soluti…

[image] giordano: To be clear, these numbers refer to unique IPs installing the package within the time window, right? Yes, IIUC. So in theory one could just tag a new patch release to estimate the number of “unique active users”.

Announcing: package download stats!

Announcements

jeremiahpslewis October 2, 2021, 6:25am 14

Not sure if this is relevant as I haven’t used snowflake previously, but for Postgres and Redshift data proceeding, I’ve found dbt a brilliant tool for structuring and executing query based data pipelines.

Topic		Replies	Views
Pkg.jl telemetry should be opt-in Internals & Design	396	33875	August 3, 2021
Pkg Ecosystem Statistics Tooling	2	1049	October 20, 2020
API to query Juliahub package info? Package Management question , packages , juliahub	4	235	December 14, 2024
Well over 10,000 (non-JLL) Julia packages, with JLLs up to 11,846 Julia registered packages. Congratulation Julia community! Community	40	6918	November 1, 2024
Some Julia growth/usage stats Community	42	13557	July 13, 2025

Announcing: package download stats!

Related topics