Understanding dependancy issues and CompatHelper

Tooling Package Management
1

First congratz on all contributors for the ForwardDiff.jl v1.0 release!

I have a package that depends on ForwardDiff, both directly and indirectly. I received the automatic PR of CompatHelper. The tests failed in the PR not because of incompatibility issues in the codebase of my own package, but because of the compat entries of my depedencies. By default, CompatHelper will run the tests by forcing the upper bounds of the compat entries. I kinda understand this choice, but at the same time, not really. :smile:

If everyone (as in every package) is waiting for everyone to update the ForwardDiff compat entries to "0.10, 1" in their Project.toml because of this issue, it feels like nobody will update it, no? Except if we are “lucky” enough and everything is done in the correct order. For exemple:

  1. A dependancy of OrdinaryDiffEq.jl update it’s codebase to be compatible with ForwardDiff v1.0 and update the compat entry
  2. OrdinaryDiffEq.jl update it’s codebase to be compatible with ForwardDiff v1.0 and update the compat entry
  3. ControlSystemsBase.jl update it’s codebase to be compatible with ForwardDiff v1.0 and update the compat entry
  4. ModelPredictiveControl.jl update it’s codebase to be compatible with ForwardDiff v1.0 and update the compat entry

And realistically, there is probably many steps before the first one. Am I misunderstanding something if I feel like “waiting for the other to update in the correct order” is almost a Mission: Impossible?

Thanks!

2

We’re doing it, it just takes steps. One repo after the other. SciML is probably half of the way through it already? The more repos the library is a part of, the more things to check. But ultimately it will get done in “not too long”, we estimated about 2 weeks for this one.

1 Like
3

Excellent, great news! But I still feel that this system is almost impossible to maintain in the long run, or I am misunderstanding a technicality (more probable).

For exemple, suppose that:

  • one dependency of the SciML ecosystem would be external to your organization
  • this dependency has the ForwardDiff = "0.10" in its [compat] section
  • is no longer maintained by the author.

ControlSystems.jl would be never able to update its compat entry for ForwardDiff.jl?

4

Yes, but if that’s the case we would either take this repo into SciML/other big Julia org to share the rights with more people, or drop it as a dependency.

1 Like
5

Alright I see. I forgot that since almost everything is open source with a liberal license in Julia, we can fork projects and update them if they are crucial for our repo.

6

Yes. And normally if you open a PR, a maintainer will come accept it and update a version. If they are “slow” (undefined), then we normally start to ask if we can move it to SciML/JuliaArrays/JuliaLinearAlgebra/etc. so we can get other people access to help with the maintenance.

In very drastic circumstances, say a maintainer hasn’t responded to any message for like 6 months, we have had some situations where a repo is forked, the URL is updated in the General registry, and then a new maintainer takes it on. We of course only do this in the most drastic of cases because it’s better to just have the original maintainer give their graces. I only know of one scenario where this was done. It was because nobody ever heard from the maintainer again and no one had admin access to the repo, so to keep the project alive the fork was turned into the successor. In all other cases, people were happy to say “yeah I got a new job and I haven’t had the time to maintain, I’d be happy to pass on the admin to xyz” and then the process continued.

3 Likes