Release Please: a tool for multi-package multi-language releases

Tooling
1

Release Please is this nice tool from Google for automating package releases across languages.

I don’t have any involvement with it but I have trying it in a rust package and really like it. They support about ~20 languages right now including go, rust, node, and python. I think Julia support would be hugely useful if someone is interested – I think it could be used to wrap JuliaRegistrator and automatically trigger new registrations.

Here are some of the features:

Parses and understands entire commit history using Conventional Commits

Conventional commits is a machine-readable standardized format for commits, with things like:

  • feat: description of feature
  • fix: description of bug
  • ci: changes to continuous integration
  • feat!: – anything with a ! is a breaking change
  • feat(gui): – can also subcategorize commits with parentheses
  • Or whatever other categories chosen for a repository (it’s configurable)

Because Conventional Commits has specific patterns to indicate breaking changes in individual commits, as well as new features, the version chosen by Release Please is automatically chosen according to SemVer.

Release please works by creating release PRs that you merge. Then other CI tools would run to actually publish the package (release please does not publish packages itself).

Keeps CHANGELOG up-to-date

Because Conventional Commits declares categories of changes, Release Please can add detailed information to the PR, releases, and CHANGELOG using information parsed from the commits. Many PRs have multiple changes across commits, so conventional commits can break this down more than the standard GitHub automatic changelog.

Multiple packages in the same repository

The new version of Release Please which was released earlier this year allows for multiple packages declared in a single schema. For example, here’s what the configuration file for my rust package rip2 looks like:

{
  "packages": {
    ".": {
      "changelog-path": "CHANGELOG.md",
      "release-type": "rust",
      "bump-minor-pre-major": true,
      "bump-patch-for-minor-pre-major": true,
      "draft": false,
      "prerelease": false,
      "include-component-in-tag": false,
      "include-v-in-tag": true
    }
  },
  "$schema": "https://raw.githubusercontent.com/googleapis/release-please/main/schemas/config.json"
}

(The bump-minor-pre-major just indicates that 0.1.0 → 0.2.0 indicates a breaking change, before the release of 1.0.0)

If there was a Julia option in release please, it could easily allow one to maintain multiple languages for a package in the same repository – similar to how PythonCall.jl has both the Julia PythonCall as well as the Python package juliacall in the same repo. I think having multiple frontends for Julia packages can be nice – not only for opening a package to more users, but also for general adoption of Julia. For example I’ve had good results in the usage of PySR which is just a Python frontend for SymbolicRegression.jl (in addition to acting as a “gateway drug” to Julia for users) – maybe these packages could one day sit in the same repo.

I remember reading that it was a bit tricky to set up continuous releases for multiple interlinked Julia packages in a monorepo so perhaps this tool might help with such situations. It is also aware of which package lives in which directory, so seems able to split the CHANGELOG and release info based on parsed commits.

5 Likes
2

I tend to think that a changelog is for users while commit log is for maintainers, and that using commit messages for both (instead of manually writing changelog entries) makes for complicated changelogs that are hard to read for users who just want to know what changed.

An commit-based changelog is probably better than no changelog, though.

3

That’s a good point. At the same time this might encourage you to have good, multi-line commit messages that have quality on par with changelogs, rather than small messages like "update feature X". But to me the changelog automation is just gravy on top of the release infrastructure.

I also really like how it (well, Conventional Commits in general) automatically validates SemVer. If I committed a breaking change to some API two months ago, I might forget about it when I create a new version. But as long as the commit two months ago has feat!: ... in the commit message, the version will be chosen to indicate a breaking release.

Speaking of which, angular.js is a nice example for a large project that uses this, including the sub-categorization of commits:

^Indeed, this is another killer feature for me. For small packages I don’t have much of an incentive to write out detailed information about changes in three separate places: the commits, the PR, and the changelog. Much nicer to have it all aggregated.

1 Like
4

This would be quite nice for my DataToolkit.jl packages I think. It’s exactly the case of multiple packages in a single repo, and if you look at the commit history you can see it’s all scoped conventional commits.

3 Likes
5

I have no insights about this tool but

and

seems contradictory?

6

They are slightly different things. The “release PR” feature of Release Please is a pull request that updates the changelog and version information and kicks off the packaging CI.

So for Julia it could update Project.toml. Then the act of merging Release Please’s PR will trigger CI to run specific to the language. In Julia’s case, that’s what would trigger JuliaRegistrator to run and register the package.

7

Sounds like a good project proposal for the JuliaCon 2024 hackathon!

3 Likes
8

That sounds largely orthogonal to what Relase Please helps you do and would be the same if you created the PR manually or with some other tool. I would absolutely love to have such a CI script though.

9

:metal:

I’m not sure what you mean, sorry. Maybe check out the README for more info: GitHub - googleapis/release-please: generate release PRs based on the conventionalcommits.org spec

10

I might be misunderstanding things but from what I read Release Please automates a PR for you and whatever happens in CI after you merge that PR has nothing to do with Release Please.

11

So the PR bumps the version (i.e., in Project.toml) according to SemVer – which is parsed from Conventional Commits, and also updates the CHANGELOG.md according to the feat for new features, fix for fixes, feat! for breaking features, etc.

After merging, there’s another CI step (part of Release Please) that creates a GitHub release and tag. So that step would normally be done by TagBot but would now be done by Release Please.

But that’s all it does, the other CI stuff is up to you. In my Rust project I have all the packaging stuff get triggered by a release-please PR merging: rip2/.github/workflows/ci.yml at 4ddc3e70e2c3d1a9530e78a38938ce6558242181 · MilesCranmer/rip2 · GitHub (which indicates a new version has been created).

In Julia I think one would just call the JuliaRegistrator in the CI file based on a conditional:

    if: needs.release-please.outputs.created

To get compatibility for Julia all that needs to happen is for someone to describe the Project.toml format in TypeScript: release-please/src/updaters at main · googleapis/release-please · GitHub and then the default “strategy” here: release-please/src/strategies at main · googleapis/release-please · GitHub which just says which other files need to be updated (I guess just the Manifest.toml, if present).

12

I suspect the point where we diverge is that you consider

In Julia I think one would just call the JuliaRegistrator in the CI file

to be a simple step, whereas I believe it’s the difficult part.

13

How about this:

jobs:
    ...
  trigger_registrator:
    name: Trigger Julia Registrator
    runs-on: ubuntu-latest
    needs: release-please
    if: needs.release-please.outputs.created
    steps:
      - uses: peter-evans/commit-comment@v3
        with:
          body: @JuliaRegistrator register

Upon merging the Release Please PR, this will add a comment on the commit SHA where it runs (Commit Comment · Actions · GitHub Marketplace · GitHub), which is how you would manually trigger the registration anyways.

It’s simple but it should work. No need to overengineer things :slight_smile:

Note that TagBot avoids creating a tag/release if the tag/release already exist, so this should work. (There’s probably a way to explicitly request this in the @JuliaRegistrator command too)

14

The reason tagbot works the way it does is that it creates the tag based on what was actually registered in the end. The problem is that when you go to trigger registration, you may have e.g. forgotten a compat bound, or gotten a blocking comment on the registration PR due to some concern from the community. This may require updating the code and retriggering registration. Thus, the commit when one “goes to register” is not necessarily the final registered commit. TagBot waits until the registration PR is merged, and uses that as it’s trigger to create the tag. That way the tag always corresponds to what is actually registered (without e.g. needing to mutate the tag, and without the tag happening “too early”).

It sounds like Release Please wants the tag to come first, which is kind of backwards for the Julia release process.

1 Like
15

That’s a good point!

Although in practice I find it’s quite rare for me to run into registration issues now that I use Aqua.jl on all my packages which checks this stuff. The Release Please CI would only run after tests pass (similar to how I use it for rust), so would only run once when the Project.toml is validated.

In the cases there is still a registration issue, I think mutating a tag is okay. On the GitHub releases page it’s pretty easy to edit which commit is referenced which also auto-updates the tag.

Actually I wonder if this might actually lower the error rate for me. Sometimes I’ve been impatient and registered a package before the tests passed (since I didn’t want to sit for my 20 minute integration tests to write a GitHub comment) — only for the tests to fail due to some merge issue. I’ve also had silly issues like commenting the wrong commit. Maybe baking the registration process into the CI could help with these.

(But this is highly subjective, I’m sure it varies by person)

16

Looks like there’s also a skip-github-release flag! So you could even stick with TagBot if desired.

I think the generated comment on the commit could even dump the release-please changelog so that the TagBot release has the same richly detailed info. (Noting that you can declare the release notes via the same GitHub comment)