Switching package registration systems soon

Just tried it out on a new package I’m thinking of registering soon, using a comment here. I’m not sure, but it doesn’t seem to be doing anything.

In terms of best practices, if the “true” version number now lives inside the Project.toml, should one make sure that it only has a release version number for one single git commit? So say, normally it is v0.1.1-dev, and then I only change that to v0.1.1 for one commit, register that commit in General, tag that commit, and then immediately in the next commit change it to v0.1.2-dev?

On one hand that seems sensible, but also quite cumbersome without some tooling. In an ideal world I would actually be able to tell registrator: “I want to register version v0.1.1”, and it would then do one commit to my repo that bumps the version number to v0.1.1 in the Project.toml, tags that commit, updates General accordingly, then adds another commit to my repo where the version in Project.toml is bumped to v0.1.2-dev.

10 Likes

I think you need backticks surrounding register()

1 Like

Thanks for the confirmation. Will you add a description of a recommended workflow, like the one I wrote, to the pkg docs about how ro register new packages and new versions? If I find time, I can try to add it myself, but that is a bit tricky when not knowing what exact workflow you envision. Furthermore, I find that the pkg docs have improved a lot since I read them last. For me the recipe like descriptions are really helpful.

Finally, I very much appriciate all the good work done on code sharing in julia. I think this is the part of julia I like most, and I find it works much better than in R, matlab and python.

2 Likes

I had originally planned on switching it off, but can keep it running if people are still using it.

The main question is: should it error (i.e. open an issue) if you attempt to tag version > 0.6, or silently ignore it?

silently ignore, so that the automated attobot does not create unnecessary issues if it is not yet removed from a repository that does not support v0.6 anymore

2 Likes

Also, perhaps some additional clarification/reminders about when it’s appropriate to include a Manifest.toml? A bunch of people in this thread are asking about both project and manifest, and the manifest is created when doing ]add to generate a project.toml, but my sense is that most things being registered by Registrator should probably not have a Manifest, is that right?

There’s some debate about that. First, note that if it does have a manifest it is ignored when the package is used as a dependency—only the top-level manifest matters. So the only effect of having a manifest checked in is to provide a record of what versions of dependencies a package has been developed and tested with. I think that recording that is quite valuable and therefore support having a manifest as well as a project file. However other pkg devs disagree (@fredrikekre iirc at least). I think that time will tell.

3 Likes

Yeah, it would be nice to have this kind of tooling. See What should version in Project.toml be between releases? · Issue #351 · JuliaLang/Pkg.jl · GitHub for a previous discussion (basically “no” to make it mandatory, IIUC).

Just a nit, but it probably is better to use v0.1.1-DEV so that we have v0.1.1-DEV < v0.1.1-alpha < v0.1.1-beta < v0.1.1-rc.

2 Likes

This I didn’t know. Is it also ignored if I do ] add? That’s the case I’m most worried about - if a bunch of packages that I’m adding are using a bunch of the same packages as dependencies and there are mismatches because people are just blindly checking in the manifest.

Then again, I suppose there’s no harm in using older versions of things if tests pass. Just seems like a lot to keep track of. I definitely plan to use the manifest for projects - i think they’re an amazing tool. I guess I just don’t really grok how they will play out in packages.

I wonder if this hard deadline set to Monday is appropriate when the tooling is still not there for a smooth transition. I anticipate a lot of stress coming from package maintainers, specially those who have a couple of packages live and running.

Ideally, we would have a good replacement to attobot ready to use—a simple button press—and only then maintainers would be requested to change their workflow. We are doing the opposite here where the change is being requested without the tooling. I have to say that the migration process is a little bit confusing, and it makes me anxious. I will likely pollute my git history with trial and error.

I’m not sure where you’re getting the idea that the tooling doesn’t exist. Registrator is the replacement for attobot. We’ve requested testing and gotten some feedback but it works already.

I’m not sure why any of this would pollute your git history. How would that happen? The worst case is that you try to register a package version and it doesn’t work.

1 Like

Sorry if I misunderstood, but I got that Registrator.jl isn’t capable of tagging the repo, creating releases, and so on because of security concerns. I also understand that similar tooling is on the way under the name of TagBot.jl. Is that incorrect?

I saw people discussing commits and commit name conventions like v1.1.0-dev, tag v1.1.1, register, and then commit v1.1.2-dev, and so on. I may have confused things, and that is why I thought this could pollute my git history.

Actually it was capable of that but people complained about the bot requiring write access so we changed it. So you’re damned if you do and you’re damned if you don’t. Package maintainers are perfectly capable of copying a few commands to tag versions themselves.

1 Like

Yes, I think I am too used to the attobot way. It is really convenient. One button press, and TaDa! everything is in place. You get the PR wrong by some simple mistake, no worries, just delete the release+tag, and redo. TaDa! attobot updates the PR. This is something of great value to someone that doesn’t want to fight with Git all the time.

2 Likes

Unfortunately, in practice people don’t actually delete tags that go wrong. Instead they tag five attempts all at once with four being wrong. I also don’t understand where the “fighting with git” comes in.

I have this personal principle where I try to minimize the time I spend using Git :slight_smile: jokes aside, I am happy with GitHub releases integration that attobot provides, and will miss it badly in the 1.0 era if that is not in the near-term plans.

1 Like

There are near term plans. If you wait until everything is 100% complete and perfect to everyone’s possible hypothetical demands then you never ship anything. You’ll also develop the wrong tool that solves problems that don’t actually exist instead of problems people actually have. The transition to the new registry system has already taken too long. We’re doing this next week.

14 Likes

We should all stop distracting @StefanKarpinski from writing that REQUIRE->Project.toml conversion script :slight_smile:

14 Likes

I 100% support this :+1::point_up:

@Nishanth_Kottary and the whole #pkg-dev team have done a tremendous job preparing this :raised_hands:

For sure it is not perfect and all this feedback will go into making the next versions better and better, but getting off METADATA needs to happen and is definitely worth any initial bumps.

9 Likes