How to properly add releases with PkgDev?

package

#1

I am probably doing something wrong, but I didn’t find it documented anywhere.

Currently, whenever I need to tag a new release for a package of mine, I do PkgDev("Foo"), go to the package directory of (i.e. ~/.julia/v0.5/Foo), push the tag manually with git push origin tag_just_created and open the pull request on the web interface of GitHub in my fork of METADATA.jl.

The problem is that, after the pull request is merged, I cannot perform updates with Pkg.update(), Julia says there is a divergence in METADATA.jl and it cannot fast-forward.

What am I doing wrong and how can I make this process more efficient?


#2

This is how I do it:

You get that METADATA problem when you don’t tag from a different branch. You should always make PRs from the non-master branch (for any repo you work with). If you add to your workflow:

  1. Check out a new branch on METADATA
  2. Push changes to this branch
  3. PR from that branch to JuliaLang/metadata-v2
  4. Check out the local metadata-v2 again

you should be fine.


#3

Can we conclude that PkgDev is currently causing more harms than benefits?

I was using it in the beginning because it was saving me from this manual process, nowadays with this fast-forward issue, I will adopt your solution and do things manually.


#4

Attobot is the way to go now thanks to Simon. It’s amazing.


#5

Amazing! Thank you for sharing @tbreloff, I will try to configure it already for my packages.


#6

Can’t say what might be wrong here without more information. If this happens you should post the output of git status in your local METADATA.


#7

@tkelman the output of git status is as follows:

On branch metadata-v2
Your branch and ‘origin/metadata-v2’ have diverged,
and have 1 and 1 different commits each, respectively.
(use “git pull” to merge the remote branch into yours)
nothing to commit, working tree clean


#8

what is the difference between git log -1 and git log -1 origin/metadata-v2 then? if there’s nothing to commit and the working tree is clean, seems strange that it couldn’t update. it should have a fallback to git pull --rebase if fast forward fails.


#9

I don’t have it anymore @tkelman, I cleaned everything and will adopt the Attobot approach from now on.


#10

Wish you had made a copy instead of deleting it entirely. We’ll never fix issues like this if we can’t properly reproduce and debug them.


#11

I know, I have reported this issue in the past somewhere else, but I don’t remember where and if someone has replied to it at all. The issue should be easily reproduced though.

I am not doing anything more than PkgDev.tag(“Foo”) in a clean ~/.julia/v0.5 directory. After the pull request is merged in METADATA, the fast-forward issue is there in my local machine when I try to Pkg.update().


#12

I’ve had similar issues in the past, and I always found myself just deleting the metadata-v2 branch and starting over by branching it off origin/metadata-v2 again; after doing this a couple of times I wrote attobot.

I suspect the underlying issue may have something to do with the introduction squash merging in GitHub (as that was roughly when I started noticing it). I haven’t tried it, but I think the following might recreate the problem:

  1. user does Pkg.tag("Foo"), which creates local METADATA commit on metadata-v2 branch with hash XXX
  2. user runs Pkg.publish(): a PR is opened on METADATA.jl
  3. PR is squash-merged, commit is changed to hash YYY
  • user runs Pkg.update() and gets the above error message (as local has commit XXX, and remote has commit YYY)

#13

@simonbyrne you should post Attobot in the Julia blog, it is such an useful addition to the development workflow!


#14

Does command-line git give an error here if you try git pull --rebase in this situation? If not, then the libgit2 implementation that replaced that was flawed and incorrect. I would think git and libgit2 should be able to figure this out - it’s not an exact fast forward but there’s no conflict here.

Kind of a shame since I prefer the cleaner metadata history (one commit for a tag instead of two, the merge commit doesn’t serve much purpose except clutter) when squash merges are used.


#15

My current METADATA.jl fork is a good example:

It differs from origin by one commit, 671ba7ea4, which was squash-merged as ec011dda.

git pull --rebase does work correctly, but Pkg.update() just gives:

julia> Pkg.update()
INFO: Updating METADATA...
WARNING: Cannot perform fast-forward merge.
NFO: Computing changes...
WARNING: ForwardDiff is fixed at 0.2.5+ conflicting with requirement for Optim: [0.3.0,0.4.0)
WARNING: JLD is fixed at 0.6.4+ conflicting with requirement for BenchmarkTools: [0.6.6,∞)
WARNING: Juno is fixed at 0.2.3+ conflicting with requirement for DataFrames: [0.2.4,∞)
WARNING: DiffBase is fixed at 0.0.0- conflicting with requirement for ForwardDiff: [0.0.1,∞)
WARNING: Compat is fixed at 0.9.3+ conflicting with requirement for SHA: [0.9.4,∞)
INFO: No packages to install, update or remove

and HEAD remains unchanged.


#16

There were some problems in how we were calling libgit2. I’ve put together a fix that should hopefully fix it (https://github.com/JuliaLang/julia/pull/19624).