How to properly add releases with PkgDev?

I am probably doing something wrong, but I didn’t find it documented anywhere.

Currently, whenever I need to tag a new release for a package of mine, I do PkgDev("Foo"), go to the package directory of (i.e. ~/.julia/v0.5/Foo), push the tag manually with git push origin tag_just_created and open the pull request on the web interface of GitHub in my fork of METADATA.jl.

The problem is that, after the pull request is merged, I cannot perform updates with Pkg.update(), Julia says there is a divergence in METADATA.jl and it cannot fast-forward.

What am I doing wrong and how can I make this process more efficient?

This is how I do it:

You get that METADATA problem when you don’t tag from a different branch. You should always make PRs from the non-master branch (for any repo you work with). If you add to your workflow:

  1. Check out a new branch on METADATA
  2. Push changes to this branch
  3. PR from that branch to JuliaLang/metadata-v2
  4. Check out the local metadata-v2 again

you should be fine.

Can we conclude that PkgDev is currently causing more harms than benefits?

I was using it in the beginning because it was saving me from this manual process, nowadays with this fast-forward issue, I will adopt your solution and do things manually.

Attobot is the way to go now thanks to Simon. It’s amazing.

4 Likes

Amazing! Thank you for sharing @tbreloff, I will try to configure it already for my packages.

Can’t say what might be wrong here without more information. If this happens you should post the output of git status in your local METADATA.

@tkelman the output of git status is as follows:

On branch metadata-v2
Your branch and ‘origin/metadata-v2’ have diverged,
and have 1 and 1 different commits each, respectively.
(use “git pull” to merge the remote branch into yours)
nothing to commit, working tree clean

what is the difference between git log -1 and git log -1 origin/metadata-v2 then? if there’s nothing to commit and the working tree is clean, seems strange that it couldn’t update. it should have a fallback to git pull --rebase if fast forward fails.

I don’t have it anymore @tkelman, I cleaned everything and will adopt the Attobot approach from now on.

Wish you had made a copy instead of deleting it entirely. We’ll never fix issues like this if we can’t properly reproduce and debug them.

I know, I have reported this issue in the past somewhere else, but I don’t remember where and if someone has replied to it at all. The issue should be easily reproduced though.

I am not doing anything more than PkgDev.tag(“Foo”) in a clean ~/.julia/v0.5 directory. After the pull request is merged in METADATA, the fast-forward issue is there in my local machine when I try to Pkg.update().

I’ve had similar issues in the past, and I always found myself just deleting the metadata-v2 branch and starting over by branching it off origin/metadata-v2 again; after doing this a couple of times I wrote attobot.

I suspect the underlying issue may have something to do with the introduction squash merging in GitHub (as that was roughly when I started noticing it). I haven’t tried it, but I think the following might recreate the problem:

  1. user does Pkg.tag("Foo"), which creates local METADATA commit on metadata-v2 branch with hash XXX
  2. user runs Pkg.publish(): a PR is opened on METADATA.jl
  3. PR is squash-merged, commit is changed to hash YYY
  • user runs Pkg.update() and gets the above error message (as local has commit XXX, and remote has commit YYY)
1 Like

@simonbyrne you should post Attobot in the Julia blog, it is such an useful addition to the development workflow!

Does command-line git give an error here if you try git pull --rebase in this situation? If not, then the libgit2 implementation that replaced that was flawed and incorrect. I would think git and libgit2 should be able to figure this out - it’s not an exact fast forward but there’s no conflict here.

Kind of a shame since I prefer the cleaner metadata history (one commit for a tag instead of two, the merge commit doesn’t serve much purpose except clutter) when squash merges are used.

My current METADATA.jl fork is a good example:
https://github.com/simonbyrne/METADATA.jl/tree/metadata-v2

It differs from origin by one commit, 671ba7ea4, which was squash-merged as ec011dda.

git pull --rebase does work correctly, but Pkg.update() just gives:

julia> Pkg.update()
INFO: Updating METADATA...
WARNING: Cannot perform fast-forward merge.
NFO: Computing changes...
WARNING: ForwardDiff is fixed at 0.2.5+ conflicting with requirement for Optim: [0.3.0,0.4.0)
WARNING: JLD is fixed at 0.6.4+ conflicting with requirement for BenchmarkTools: [0.6.6,∞)
WARNING: Juno is fixed at 0.2.3+ conflicting with requirement for DataFrames: [0.2.4,∞)
WARNING: DiffBase is fixed at 0.0.0- conflicting with requirement for ForwardDiff: [0.0.1,∞)
WARNING: Compat is fixed at 0.9.3+ conflicting with requirement for SHA: [0.9.4,∞)
INFO: No packages to install, update or remove

and HEAD remains unchanged.

There were some problems in how we were calling libgit2. I’ve put together a fix that should hopefully fix it (Fixes broken LibGit2.rebase! by simonbyrne · Pull Request #19624 · JuliaLang/julia · GitHub).

2 Likes