The current METADATA release process


#1

As some of you might have noticed, the process of releasing and registering packages in METADATA has changed over last months. We have experimented with making the process more automatic such that new versions can be released faster and with less labor involved in the process. I’ve given some advice to the new process in individual PRs but there hasn’t been any announcements of what has changed. This is the purpose of this post.

The way things work now is that new releases are merged automatically every half hour provided that

  • The release is made with Attobot
  • METADATA Travis tests pass
  • CIBot reverse dependency tests pass
  • No comments have been posted after the latest Attobot comment
  • No labels have been attached

For new package registrations, the same rules apply but furthermore they have to stay open for three days. As a consequence of the automated process, you are now even more strongly encouraged to use Attobot for releases. Otherwise, the release process will typically be slower since the release will require actions from a METADATA maintainer.

As part of the transition to the more automated process, it is now expected that the package maintainers are active in identifying issues with their PRs by reading the logs. Once the issue has been identified, the author should update the PR, see https://github.com/attobot/attobot#updatingdeleting-a-release, (and not create a new tag). Hence in general, METADATA maintainers won’t comment on (or merge) PRs with failing tests. If you are unable to identify why tests are failing then please leave a comment in the issue and we’ll try to help you.

Sometimes the CIBot tests takes time to finish. There can be several reasons for this. Most often it is simply because many packages have been released within a short period so there is a queue. For packages with many reverse dependencies, there will be a lot of packages to tests. Sometimes, the tests won’t finish because one of the packages hangs (we are working on adding a time limit). Finally, it sometimes, but rarely, happens that the process that handles the webhooks fails. If that happens, it requires actions from one of the CIBot maintainers. If you think the CIBot tests are taking too long then please leave a comment but remember that your comment will disable the automatic merging mechanism so please wait at least six hours.

Some packages can’t be tested by CIBot, e.g. if it depends on proprietary solvers of if the package requires special setup of the CI system. Such packages can be marked as untestable and the CIBot tests will be skipped. Please leave a comment in the PR if the package can’t be tested by CIBot. When possible, it is best to organize your tests such that CIBot is able to test your package since the reverse dependency testing often detects breaking changes.

Some packages require special setup when run through CI. In particular, some packages with Python dependencies need certain Python packages to be installed. CIBot uses Conda when installing PyCall so it is possible to install Python packages via Conda conditionally on the environment variable CI=true. As an example of how this can be done see https://github.com/tkf/IPython.jl/blob/6b4b7995b8d02dbd68f1a7fca95996a440fc988c/test/runtests.jl#L3-L5.

Finally, CIBot no longer has any issues with project files. There was a period when CIBot couldn’t handle project files and I therefore asked people to delete project files from packages. This is no longer the case. However, the General registry is still based on METADATA which is again based on the REQUIRE file. CIBot is based on the General registry so project (or manifest) files will not affect the reverse dependency testing. Once we transition to releasing directly to General, see https://github.com/JuliaLang/Pkg.jl/issues/849, the REQUIRE file will no longer be needed.

Feel free to ask any questions.


TravisCI build with Python deps
#2

Does it mean ENV["PYTHON"] is set to Conda?


#3

Does it mean ENV["PYTHON"] is set to Conda ?

Almost. It’s set to ENV["PYTHON"]="" which defaults to Conda.


#4

I see. Thanks.


#5

It seems to me like a lot of (all?) PyCall-backed packages just don’t work when ENV["PYTHON"] != "". This has always been a minor frustration for me. If even METADATA CI is defaulting to ENV["PYTHON"] = "" now, should the default in PyCall just be changed?


#6

What exactly is the “minor frustration”? That PyCall.jl does not auto-magically install Python packages when not using Conda.jl?


#7

I guess, if that’s what’s happening. I’m not familiar with PyCall internals. All I know is that when e.g. using SymPy in a test on Travis, I need to set ENV["PYTHON"]="" or it doesn’t work.


#8

Just wanted to say, thanks so much for writing all this up! I think the current transitional state of METADATA/General has been a source of confusion for post-1.0 Julia newcomers, and having all this info in one place really helps to clarify the situation.


#9

I guess in principle using Conda.jl can be the default in PyCall.jl for Linux as well. Maybe open an issue in github? There is always a trade-off and I’m not sure what’s the best. Though @stevengj may already have some reasons to avoid it.


#10

Originally PyCall defaulted to whatever Python you had installed, and installed Conda as a fallback. On Mac and Windows we switched to defaulting to Conda because we encountered broken Python environments extremely often on those platforms.

We continued to default to the system python (now python3) on other Unix platforms, partly to be conservative, partly because GNU/Linux systems tend to have a working up-to-date Python, and partly because non-Mac Unix users have different expectations and might be upset if we defaulted to installing our own Anaconda Python distro rather than using a python in their PATH.

If there is a clamor by Ubuntu users for us to default to installing our own Python, we can easily switch to using Conda by default, but I haven’t heard that until now. Travis testing is another matter, but normally I would suggest testing with the Ubuntu packages of sympy etcetera in order to mirror the setup of a typical user.


#11

One benefit of Conda.jl being default in Linux is that we can be sure that the CI setup mirrors typical users’ setup (:= no custom ENV["PYTHON"] configuration) even across Linux distros.

(Not that I prefer this default personally. But it’s already configurable so I won’t care much.)


#12

This is my sentiment exactly.


#13

Thanks for writing this up, and thanks for implementing the new automated system. I think it is quite intuitive - I managed to infer a lot of the points you make above just by messing around with new releases on a package for an hour or so a few days back.


#14

I have a question about CIBot in the specific case of the StaticArrays 0.9.0 tag. Here’s the CIBot output. Why didn’t CIBot test RigidBodyDynamics.jl (even though it isn’t marked as untestable)? In general, I would think that StaticArrays has many more reverse dependencies.


#15

That is good question. I forgot to mention that in the top post. We currently limit the number of tested package to ten to avoid tests running for too long.Personally, I think it would be great to set that limit higher since a lot of reverse dependencies reflects that many people rely on a package so it’s worth spending the extra time and compute verifying that nothing breaks. We might gradually increase the number of packages tested to see how it goes.