# Q: automatically updating/detecting missed JLL packages updates
**URL:** https://discourse.julialang.org/t/q-automatically-updating-detecting-missed-jll-packages-updates/81413
**Category:** Tooling
**Tags:** question, security, binarybuilder, yggdrasil
**Created:** [May 21, 2022, 11:53am UTC](https://discourse.julialang.org/t/q-automatically-updating-detecting-missed-jll-packages-updates/81413 "2022-05-21T11:53:31Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@ImreSamu](https://discourse.julialang.org/u/ImreSamu)
#### Post date: [May 21, 2022, 11:53am UTC](https://discourse.julialang.org/t/q-automatically-updating-detecting-missed-jll-packages-updates/81413/1 "2022-05-21T11:53:31Z")
**context:**
- Some JLL packages are not updated to the latest version
- so important (security) patches not used.
- hard to find these packages in the local environments.
example:
- LibPQ\_jll ( PostgreSQL client) - based on v14.1 : `LibPQ_jll v14.1.0+1`
- current latest patch (+security) version : [v14.3 · 2022-05-12](https://www.postgresql.org/docs/14/release-14-3.html)
Question:
- Is it possible to automate the detection of these missed package updates? ( and update )
- in the local environment
- in the package libraries ( globally )
- any tools? packages ( or built in solution ; bots ) ?
- What is the best practice?
Side note:
- Thanks for the current maintainers! :juliaheartpulsing:
- I know - I can manually update this versions myself [1] - but I have a lot of other JLL files.
- [1] [Yggdrasil/build\_tarballs.jl at master · JuliaPackaging/Yggdrasil · GitHub](https://github.com/JuliaPackaging/Yggdrasil/blob/master/L/LibPQ/build_tarballs.jl)
Related problem:
- using Julia in the Enterprise world - I need to check the [vulnerability](https://en.wikipedia.org/wiki/Vulnerability_(computing)) risks of the Julia packages.
---
### Author:  [@giordano](https://discourse.julialang.org/u/giordano)
#### Post date: [May 21, 2022, 12:15pm UTC](https://discourse.julialang.org/t/q-automatically-updating-detecting-missed-jll-packages-updates/81413/2 "2022-05-21T12:15:23Z")
There is a problem with automatic updates: packages can introduce API/ABI-breaking changes in a completely _ **unpredictable** _ and often non-trivial to check way. Linux distributions don’t have this problem because most of them have a single version of a package at any given time and they rebuild the entire universe (well, the affected subset) when a package is updated. But we have to support multiple versions at the same time, and retroactively fix compat bounds if necessary, which, again, can’t really be automated. The risk is that automatically updating packages will cause breakage elsewhere, with zero possibility of automatically guarding against that.
With this I’m not saying that we shouldn’t update packages (we should! And pull requests are always welcome) but only that automated mass updates are greatly non-trivial. If someone wants to take the lead on that task, be welcome.
---
### Author:  [@ImreSamu](https://discourse.julialang.org/u/ImreSamu)
#### Post date: [May 21, 2022, 12:48pm UTC](https://discourse.julialang.org/t/q-automatically-updating-detecting-missed-jll-packages-updates/81413/3 "2022-05-21T12:48:31Z")
> [@giordano](#):
>
> There is a problem with automatic updates:
ok. automatic update is not easy …
First, a small step.
Is it possible to add some (new) metadata to the binarybuilder scripts?
- **package versioning policy / versioning number policy**
- [PostgreSQL: Versioning Policy](https://www.postgresql.org/support/versioning/)
- PostgreSQL X.Y - where Y is a (minor-patch) == not breaking change version, so low risk to upgrade. ( and recommended )
- **release detecting method:**
- github tags + regexp
- [https://github.com/postgres/postgres/releases/tag/REL\_14\_3](https://github.com/postgres/postgres/releases/tag/REL_14_3)
so
- I can detect the missing (minor/patch) releases …
- I can write a (half) automatic update scripts - as I like …
---
### Author:  [@giordano](https://discourse.julialang.org/u/giordano)
#### Post date: [May 21, 2022, 12:55pm UTC](https://discourse.julialang.org/t/q-automatically-updating-detecting-missed-jll-packages-updates/81413/4 "2022-05-21T12:55:32Z")
> [@ImreSamu](#):
>
> Is it possible to add some (new) metadata to the binarybuilder scripts?
What do you mean exactly? I’m not very happy about adding ad-hoc random metadata. In what format? How that can be used? How that can be generalised? In the meantime it’s much easier to do a pull request to update the package before getting solid answers to all these questions
---
### Author:  [@ImreSamu](https://discourse.julialang.org/u/ImreSamu)
#### Post date: [May 24, 2022, 2:21pm UTC](https://discourse.julialang.org/t/q-automatically-updating-detecting-missed-jll-packages-updates/81413/5 "2022-05-24T14:21:11Z")
> [@Giordano](#):
>
> What do you mean exactly?
still researching / thinking 🤔
1.) “DEBIAN” best practice
the debian using a `watch_file` format and uscan tool
- [uscan](https://manpages.debian.org/bullseye-backports/devscripts/uscan.1.en.html) - “scan/watch upstream sources for new releases of software”
- status: [https://udd.debian.org/cgi-bin/upstream-status.json.cgi](https://udd.debian.org/cgi-bin/upstream-status.json.cgi)
```julia-auto
wget https://udd.debian.org/cgi-bin/upstream-status.json.cgi
cat upstream-status.json.cgi | grep '"status"' | sort | uniq -c
3344 "status": "error",
7195 "status": "newer package available",
898 "status": "only older package available",
12 "status": "package available",
19710 "status": "up to date",
```
- [udd=UltimateDebianDatabase](https://wiki.debian.org/UltimateDebianDatabase/) database contains the upstream metadata ; and there are 3 “upstream” tables for string metadata
- [public.upstream](https://udd.debian.org/schema/udd.html#public.table.upstream)
- [public.upstream\_metadata](https://udd.debian.org/schema/udd.html#public.table.upstream-metadata)
- [public.upstream\_status](https://udd.debian.org/schema/udd.html#public.table.upstream-status)
2.) Fedora - [https://release-monitoring.org](https://release-monitoring.org)
“ **Anitya is a release monitoring project.**
Its goal is to regularly check if a project has made a new release. When Anitya discovers a new release for a project, it publishes a RabbitMQ message via fedora messaging. This makes it easy to integrate with Anitya and perform actions when a new release is created for a project. For example, the Fedora project runs a service called the-new-hotness which files a Bugzilla bug against a package when the upstream project makes a new release.”
example geos: [Making sure you're not a bot!](https://release-monitoring.org/project/13493/)
3.) Repology [https://repology.org/](https://repology.org/)
## example - “geos” package"
connect to udd
`psql "postgresql://udd-mirror:udd-mirror@udd-mirror.debian.net/udd"`
and check the `watch_file` columns
```sql
psql (12.10 (Ubuntu 12.10-0ubuntu0.20.04.1), server 11.14 (Debian 11.14-0+deb10u1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
udd=> \dt+ upstream*
List of relations
Schema | Name | Type | Owner | Size | Description
--------+-------------------+-------+-------+---------+-------------
public | upstream | table | udd | 12 MB |
public | upstream_metadata | table | udd | 4768 kB |
public | upstream_status | table | udd | 4928 kB |
(3 rows)
udd=> select * from upstream where source like 'geos';
-[RECORD 1]-----------+----------------------------------------------------------------------------------
source | geos
version | 3.10.2-1
distribution | debian
release | sid
component | main
watch_file | version=4 +
| opts=\ +
| dversionmangle=s/\+(debian|dfsg|ds|deb)\d*$//,\ +
| uversionmangle=s/(\d)[_\.\-\+]?((RC|rc|pre|dev|beta|alpha)\d*)$/$1~$2/;s/RC/rc/ \+
| https://download.osgeo.org/geos/ \ +
| (?:|.*/)geos-(?:[_\-]v?|)(\d\S*)\.(?:tar\.xz|txz|tar\.bz2|tbz2|tar\.gz|tgz) +
|
signing_key_pgp |
signing_key_asc |
debian_uversion | 3.10.2
debian_mangled_uversion | 3.10.2
upstream_version | 3.10.2
upstream_url | https://download.osgeo.org/geos/geos-3.10.2.tar.bz2
errors |
warnings |
status | up to date
last_check | 2022-05-21 00:21:20.012495
```
fedora : geos: [Making sure you're not a bot!](https://release-monitoring.org/project/13493/) Latest version : 3.10.2
the Julia `geos_jll` does not contain the patched versions.
- version = v"3.10.0" [Yggdrasil/G/GEOS/build\_tarballs.jl at master · JuliaPackaging/Yggdrasil · GitHub](https://github.com/JuliaPackaging/Yggdrasil/blob/master/G/GEOS/build_tarballs.jl)
- [juliahub - GEOS\_jll v3.10](https://juliahub.com/ui/Packages/GEOS_jll/4qIWY/3.10.0+0)
side note:
- debian uscan `version=5` (more simple version ) is under developments:
- [uscan version=5 roadmap](https://www.mail-archive.com/debian-devel@lists.debian.org/msg370860.html)
Summary:
- maybe we can link JLL packages to other repo metadata?
- or just using uscan tool and metadata format.
- setting **“safe update policy”** for every package
- “patch” changes
- “minor” changes: Postgresql using this .. and safe to update.
- “manual” - …
- maybe we can add the “key-value” metadata
- wikidata ID :
- upstream bug reporting
- upstream documentation
so for the metadata for `geos_jll` i.e. the format could be :
- repology\_id=[geos](https://repology.org/project/geos/information)
- watch\_file= `< debian uscan v=5? format >` ( alternative )
- safeupdatepolicy=`patch`
- metadata (key-value)
- wikidata=[Q1503283](https://www.wikidata.org/wiki/Q1503283)
- bug\_tracking=[https://github.com/libgeos/geos/issues](https://github.com/libgeos/geos/issues)
- license=`LGPLv2.1`
- website=[https://libgeos.org/](https://libgeos.org/)
- …
🤔