# The current METADATA release process

**URL:** <https://discourse.julialang.org/t/the-current-metadata-release-process/16672>\
**Category:** Announcements\
**Created:** [October 23, 2018, 9:23am UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672 "2018-10-23T09:23:51Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![andreasnoack](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andreasnoack/32/27_2.png) [@andreasnoack](https://discourse.julialang.org/u/andreasnoack)\
**Post date:** [October 23, 2018, 9:23am UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/1 "2018-10-23T09:23:51Z")

</div>

As some of you might have noticed, the process of releasing and registering packages in METADATA has changed over last months. We have experimented with making the process more automatic such that new versions can be released faster and with less labor involved in the process. I’ve given some advice to the new process in individual PRs but there hasn’t been any announcements of what has changed. This is the purpose of this post.

The way things work now is that new _releases_ are merged automatically every half hour provided that

- The release is made with Attobot
- METADATA Travis tests pass
- CIBot reverse dependency tests pass
- No comments have been posted after the latest Attobot comment
- No labels have been attached

For new package _registrations_, the same rules apply but furthermore they have to stay open for three days. As a consequence of the automated process, you are now even more strongly encouraged to use Attobot for releases. Otherwise, the release process will typically be slower since the release will require actions from a METADATA maintainer.

As part of the transition to the more automated process, it is now expected that the package maintainers are active in identifying issues with their PRs by reading the logs. Once the issue has been identified, the author should update the PR, see [GitHub - attobot/attobot: Julia package release bot](https://github.com/attobot/attobot#updatingdeleting-a-release), (and not create a new tag). Hence in general, METADATA maintainers won’t comment on (or merge) PRs with failing tests. If you are unable to identify why tests are failing then please leave a comment in the issue and we’ll try to help you.

Sometimes the CIBot tests takes time to finish. There can be several reasons for this. Most often it is simply because many packages have been released within a short period so there is a queue. For packages with many reverse dependencies, there will be a lot of packages to tests. Sometimes, the tests won’t finish because one of the packages hangs (we are working on adding a time limit). Finally, it sometimes, but rarely, happens that the process that handles the webhooks fails. If that happens, it requires actions from one of the CIBot maintainers. If you think the CIBot tests are taking too long then please leave a comment but remember that your comment will disable the automatic merging mechanism so please wait at least six hours.

Some packages can’t be tested by CIBot, e.g. if it depends on proprietary solvers of if the package requires special setup of the CI system. Such packages can be marked as _untestable_ and the CIBot tests will be skipped. Please leave a comment in the PR if the package can’t be tested by CIBot. When possible, it is best to organize your tests such that CIBot is able to test your package since the reverse dependency testing often detects breaking changes.

Some packages require special setup when run through CI. In particular, some packages with Python dependencies need certain Python packages to be installed. CIBot uses `Conda` when installing `PyCall` so it is possible to install Python packages via `Conda` conditionally on the environment variable `CI=true`. As an example of how this can be done see [https://github.com/tkf/IPython.jl/blob/6b4b7995b8d02dbd68f1a7fca95996a440fc988c/test/runtests.jl#L3-L5](https://github.com/tkf/IPython.jl/blob/6b4b7995b8d02dbd68f1a7fca95996a440fc988c/test/runtests.jl#L3-L5).

Finally, CIBot no longer has any issues with project files. There was a period when CIBot couldn’t handle project files and I therefore asked people to delete project files from packages. This is no longer the case. However, the General registry is still based on METADATA which is again based on the `REQUIRE` file. CIBot is based on the General registry so project (or manifest) files will not affect the reverse dependency testing. Once we transition to releasing directly to General, see [https://github.com/JuliaLang/Pkg.jl/issues/849](https://github.com/JuliaLang/Pkg.jl/issues/849), the `REQUIRE` file will no longer be needed.

Feel free to ask any questions.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [October 24, 2018, 12:55am UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/2 "2018-10-24T00:55:29Z")

</div>

> [@andreasnoack](#):
>
> CIBot uses `Conda` when installing `PyCall`

Does it mean `ENV["PYTHON"]` is set to `Conda`?

---

<div class="post-metadata">

**Author:** ![andreasnoack](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andreasnoack/32/27_2.png) [@andreasnoack](https://discourse.julialang.org/u/andreasnoack)\
**Post date:** [October 24, 2018, 7:16am UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/3 "2018-10-24T07:16:56Z")

</div>

> Does it mean `ENV["PYTHON"]` is set to `Conda` ?

Almost. It’s set to `ENV["PYTHON"]=""` which defaults to Conda.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [October 24, 2018, 8:11am UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/4 "2018-10-24T08:11:09Z")

</div>

I see. Thanks.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [October 24, 2018, 9:08am UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/5 "2018-10-24T09:08:22Z")

</div>

It seems to me like a lot of (all?) PyCall-backed packages just don’t work when `ENV["PYTHON"] != ""`. This has always been a minor frustration for me. If even METADATA CI is defaulting to `ENV["PYTHON"] = ""` now, should the default in `PyCall` just be changed?

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [October 24, 2018, 6:25pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/6 "2018-10-24T18:25:01Z")

</div>

What exactly is the “minor frustration”? That PyCall.jl does not auto-magically install Python packages when not using Conda.jl?

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [October 24, 2018, 6:36pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/7 "2018-10-24T18:36:41Z")

</div>

I guess, if that’s what’s happening. I’m not familiar with PyCall internals. All I know is that when e.g. using SymPy in a test on Travis, I need to set `ENV["PYTHON"]=""` or it doesn’t work.

---

<div class="post-metadata">

**Author:** ![jrevels](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jrevels/32/10393_2.png) [@jrevels](https://discourse.julialang.org/u/jrevels)\
**Post date:** [October 24, 2018, 6:40pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/8 "2018-10-24T18:40:05Z")

</div>

Just wanted to say, thanks so much for writing all this up! I think the current transitional state of METADATA/General has been a source of confusion for post-1.0 Julia newcomers, and having all this info in one place really helps to clarify the situation.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [October 24, 2018, 9:19pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/9 "2018-10-24T21:19:02Z")

</div>

I guess in principle using Conda.jl can be the default in PyCall.jl for Linux as well. Maybe open an issue in github? There is always a trade-off and I’m not sure what’s the best. Though @stevengj may already have some reasons to avoid it.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [October 24, 2018, 9:26pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/10 "2018-10-24T21:26:56Z")

</div>

> [@tkf](#):
>
> I guess in principle using Conda.jl can be the default in PyCall.jl for Linux as well. Maybe open an issue in github? There is always a trade-off and I’m not sure what’s the best. Though @stevengj may already have some reasons to avoid it.

Originally PyCall defaulted to whatever Python you had installed, and installed Conda as a fallback. On Mac and Windows we switched to defaulting to Conda because we encountered broken Python environments extremely often on those platforms.

We continued to default to the system `python` (now `python3`) on other Unix platforms, partly to be conservative, partly because GNU/Linux systems tend to have a working up-to-date Python, and partly because non-Mac Unix users have different expectations and might be upset if we defaulted to installing our own Anaconda Python distro rather than using a `python` in their `PATH`.

If there is a clamor by Ubuntu users for us to default to installing our own Python, we can easily switch to using Conda by default, but I haven’t heard that until now. Travis testing is another matter, but normally I would suggest testing with the Ubuntu packages of sympy etcetera in order to mirror the setup of a typical user.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [October 24, 2018, 9:44pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/11 "2018-10-24T21:44:36Z")

</div>

> [@stevengj](#):
>
> mirror the setup of a typical user

One benefit of Conda.jl being default in Linux is that we can be sure that the CI setup mirrors typical users’ setup (:= no custom `ENV["PYTHON"]` configuration) even across Linux distros.

(Not that I prefer this default personally. But it’s already configurable so I won’t care much.)

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [October 24, 2018, 10:27pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/12 "2018-10-24T22:27:19Z")

</div>

> [@tkf](#):
>
> One benefit of Conda.jl being default in Linux is that we can be sure that the CI setup mirrors typical users’ setup

This is my sentiment exactly.

---

<div class="post-metadata">

**Author:** ![colintbowers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/colintbowers/32/8033_2.png) [@colintbowers](https://discourse.julialang.org/u/colintbowers)\
**Post date:** [October 24, 2018, 10:28pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/13 "2018-10-24T22:28:41Z")

</div>

Thanks for writing this up, and thanks for implementing the new automated system. I think it is quite intuitive - I managed to infer a lot of the points you make above just by messing around with new releases on a package for an hour or so a few days back.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [October 26, 2018, 12:09am UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/14 "2018-10-26T00:09:03Z")

</div>

I have a question about CIBot in the specific case of the [StaticArrays 0.9.0 tag](https://github.com/JuliaLang/METADATA.jl/pull/19043). [Here](https://s3.ap-south-1.amazonaws.com/juliarun-ci/84c276d804f6939edf373c0f8b1ac13ce706621f/StaticArrays_reverse_dependency_tests.html)’s the CIBot output. Why didn’t CIBot test RigidBodyDynamics.jl (even though [it isn’t marked as untestable](https://pkg.julialang.org/?pkg=RigidBodyDynamics&ver=release#RigidBodyDynamics))? In general, I would think that StaticArrays has many more reverse dependencies.

---

<div class="post-metadata">

**Author:** ![andreasnoack](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andreasnoack/32/27_2.png) [@andreasnoack](https://discourse.julialang.org/u/andreasnoack)\
**Post date:** [October 26, 2018, 5:45am UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/15 "2018-10-26T05:45:11Z")

</div>

> [@tkoolen](#):
>
> I have a question about CIBot in the specific case of the [StaticArrays 0.9.0 tag](https://github.com/JuliaLang/METADATA.jl/pull/19043). [Here](https://s3.ap-south-1.amazonaws.com/juliarun-ci/84c276d804f6939edf373c0f8b1ac13ce706621f/StaticArrays_reverse_dependency_tests.html)’s the CIBot output. Why didn’t CIBot test RigidBodyDynamics.jl (even though [it isn’t marked as untestable](https://pkg.julialang.org/?pkg=RigidBodyDynamics&ver=release#RigidBodyDynamics))? In general, I would think that StaticArrays has many more reverse dependencies.

That is good question. I forgot to mention that in the top post. We currently limit the number of tested package to ten to avoid tests running for too long.Personally, I think it would be great to set that limit higher since a lot of reverse dependencies reflects that many people rely on a package so it’s worth spending the extra time and compute verifying that nothing breaks. We might gradually increase the number of packages tested to see how it goes.

---

<div class="post-metadata">

**Author:** ![goedman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goedman/32/217_2.png) [@goedman](https://discourse.julialang.org/u/goedman)\
**Post date:** [November 29, 2018, 4:32pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/16 "2018-11-29T16:32:31Z")

</div>

Hi Andreas, thanks for all the great work on this! It’s been working great.

When trying to tag a new version of HyperDualNumbers.jl we ran into some issues. I have some suggestions/questions related to JuliaCIBot:

1. Should an update request be marked as fail vs. just letting the failed dependencies know about this? Or maybe just when it is updating the major version number?

2. Should dependent packages within a Github organization have preference to be tested? Assuming being part of the same Github organization could be established for this purpose.

3. Or is it a good idea to add a section for this to Project.toml? That way dependent packages can create a PR requesting to be added to such a section.

4. Finally, it would be really nice to be able to publish (or include in the docs) the JuliaCIBot page. Is it only on AWS?

Rob

---

<div class="post-metadata">

**Author:** ![andreasnoack](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andreasnoack/32/27_2.png) [@andreasnoack](https://discourse.julialang.org/u/andreasnoack)\
**Post date:** [December 4, 2018, 1:46pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/17 "2018-12-04T13:46:03Z")

</div>

> 1. Should an update request be marked as fail vs. just letting the failed dependencies know about this?

In the future, we could fine tune this but most breakages seem to be unintentional in which case the failure status is useful.

> 1. Should dependent packages within a Github organization have preference to be tested? Assuming being part of the same Github organization could be established for this purpose.
> 2. Or is it a good idea to add a section for this to Project.toml? That way dependent packages can create a PR requesting to be added to such a section.

I don’t understand what you are asking here. Could you elaborate?

> 1. Finally, it would be really nice to be able to publish (or include in the docs) the JuliaCIBot page. Is it only on AWS?

The reporting could definitely be improved. However, we’ll have to restructure and rewrite CIBot when migrating to the new non-METADATA based registration system so it’s unlikely that we’ll make changes before the new system is up and running.

---

<div class="post-metadata">

**Author:** ![goedman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goedman/32/217_2.png) [@goedman](https://discourse.julialang.org/u/goedman)\
**Post date:** [December 5, 2018, 3:01pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/18 "2018-12-05T15:01:26Z")

</div>

Hi Andreas, thanks for your response.

It’s great to have visibility into which dependent packages will fail after an update (which is shown on JuliaCIBot page). Items 2 and 3 are ideas to provide developers of packages some control and focus over when a package update is marked to have failed. Instead of testing 10 (or more in the future) randomly selected packages, the tested packages are part of a github organization or specified in a Project.toml section.

I do realize a lot of this is still in flux!

---

<div class="post-metadata">

**Author:** ![andreasnoack](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andreasnoack/32/27_2.png) [@andreasnoack](https://discourse.julialang.org/u/andreasnoack)\
**Post date:** [January 2, 2019, 1:48pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/19 "2019-01-02T13:48:04Z")

</div>

# Update: 2 January 2019

Because of problems caused by updates to Azure, we are currently unable to deploy worker nodes for CIBot. Hence, CIBot verification of METADATA PRs have been suspended and PRs will be merged if just the METADATA tests pass.

We currently don’t know when CIBot will be up and running again. Hence, for the time being, it is adviced that package authors are cautious about breaking changes.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [January 6, 2019, 2:51pm UTC](https://discourse.julialang.org/t/the-current-metadata-release-process/16672/20 "2019-01-06T14:51:28Z")

</div>

I don’t know if this has been discussed anywhere (I’d imagine it probably has), but what is the plan for supporting automatic METADATA/General merges for packages that aren’t hosted on Github? I host most of my packages on Gitlab, and plan to host some more on a different non-Github service in the near future. It would be nice if these packages could also benefit from a streamlined, automated METADATA/General release process.

[Next page](https://discourse.julialang.org/t/the-current-metadata-release-process/16672.md?page=2)