Unclear where compatibility requirements are coming from

New to Julia
,
1

I’m trying to develop on a fork of the ForwardDiff.jl package (current master commit). When I try to add the repo to my dependencies I get the following compatibility error:

(@v1.9) pkg> dev /Users/olly/.julia/dev/ForwardDiff.jl
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package Zygote [e88e6eb3]:
 Zygote [e88e6eb3] log:
 ├─possible versions are: 0.1.0-0.6.60 or uninstalled
 ├─restricted by compatibility requirements with SpecialFunctions [276daf66] to versions: [0.4.21-0.5.1, 0.5.13-0.6.60] or uninstalled, leaving only versions: [0.4.21-0.5.1, 0.5.13-0.6.60]
 │ └─SpecialFunctions [276daf66] log:
 │   ├─possible versions are: 0.7.0-2.2.0 or uninstalled
 │   └─restricted to versions 1-2 by ForwardDiff [f6369f11], leaving only versions: 1.0.0-2.2.0
 │     └─ForwardDiff [f6369f11] log:
 │       ├─possible versions are: 0.11.0 or uninstalled
 │       └─ForwardDiff [f6369f11] is fixed to version 0.11.0-DEV
 ├─restricted by compatibility requirements with NaNMath [77ba4419] to versions: [0.4.21-0.5.1, 0.6.35-0.6.60] or uninstalled, leaving only versions: [0.4.21-0.5.1, 0.6.35-0.6.60]
 │ └─NaNMath [77ba4419] log:
 │   ├─possible versions are: 0.3.2-1.0.2 or uninstalled
 │   └─restricted to versions 1 by ForwardDiff [f6369f11], leaving only versions: 1.0.0-1.0.2
 │     └─ForwardDiff [f6369f11] log: see above
 └─restricted by compatibility requirements with ForwardDiff [f6369f11] to versions: uninstalled — no versions left
   └─ForwardDiff [f6369f11] log: see above

I don’t understand why this error is happening. I do not see any references to compatibility versions for Zygote in the project.toml file.

I get a similar issue for LoopVectorization.jl when trying to use this local version of ForwardDiff.jl in another project. Again, there is no mention of LoopVectorization in the [compat] section of ForwardDiff’s project.toml file.

What is going on here?

2

Here is the other error:

(NLLSsolver) pkg> dev /Users/olly/.julia/dev/ForwardDiff.jl
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package LoopVectorization [bdcacae8]:
 LoopVectorization [bdcacae8] log:
 ├─possible versions are: 0.1.0-0.12.159 or uninstalled
 ├─restricted by julia compatibility requirements to versions: 0.9.0-0.12.159 or uninstalled
 ├─restricted by compatibility requirements with ForwardDiff [f6369f11] to versions: 0.1.0-0.12.99 or uninstalled, leaving only versions: 0.9.0-0.12.99 or uninstalled
 │ └─ForwardDiff [f6369f11] log:
 │   ├─possible versions are: 0.11.0 or uninstalled
 │   └─ForwardDiff [f6369f11] is fixed to version 0.11.0-DEV
 ├─restricted by compatibility requirements with VectorizationBase [3d5dd08c] to versions: 0.12.19-0.12.159 or uninstalled, leaving only versions: 0.12.19-0.12.99 or uninstalled
 │ └─VectorizationBase [3d5dd08c] log:
 │   ├─possible versions are: 0.1.0-0.21.64 or uninstalled
 │   ├─restricted by julia compatibility requirements to versions: 0.20.8-0.21.64 or uninstalled
 │   └─restricted by compatibility requirements with ArrayInterface [4fba245c] to versions: [0.1.0-0.12.33, 0.21.59-0.21.64] or uninstalled, leaving only versions: 0.21.59-0.21.64 or uninstalled
 │     └─ArrayInterface [4fba245c] log:
 │       ├─possible versions are: 0.0.1-7.4.3 or uninstalled
 │       └─restricted by compatibility requirements with LinearSolve [7ed4a6bd] to versions: 7.3.0-7.4.3
 │         └─LinearSolve [7ed4a6bd] log:
 │           ├─possible versions are: 0.1.0-1.42.0 or uninstalled
 │           └─restricted to versions 1.41.1-1 by NLLSsolver [50b5cf9e], leaving only versions: 1.41.1-1.42.0
 │             └─NLLSsolver [50b5cf9e] log:
 │               ├─possible versions are: 2.0.0 or uninstalled
 │               └─NLLSsolver [50b5cf9e] is fixed to version 2.0.0
 ├─restricted by compatibility requirements with ArrayInterface [4fba245c] to versions: [0.1.0-0.8.26, 0.12.151-0.12.159] or uninstalled, leaving only versions: uninstalled
 │ └─ArrayInterface [4fba245c] log: see above
 └─restricted by compatibility requirements with RecursiveFactorization [f2c3362d] to versions: 0.10.0-0.12.159 — no versions left
   └─RecursiveFactorization [f2c3362d] log:
     ├─possible versions are: 0.0.1-0.2.18 or uninstalled
     └─restricted by compatibility requirements with LinearSolve [7ed4a6bd] to versions: 0.2.8-0.2.18
       └─LinearSolve [7ed4a6bd] log: see above

I do not understand where the following requirments on LoopVectorization come from:

restricted by compatibility requirements with ForwardDiff [f6369f11] to versions: 0.1.0-0.12.99 or uninstalled

As I say, there is nothing about this in the project.toml file, so I don’t understand where it comes from.

3

I have deleted (then regenerated) the auto-generated manifest.toml files from the relevant projects, but this doesn’t fix the issue. The errors remain the same.

4

I think I understand where the LoopVectorization compatibility issue comes from. LoopVectorization itself specifies a version of ForwardDiff it is compatible with, so one needs to go back to the version before that occurred for it to be compatible.

@Elrod Is there a reason not to use:

[compat]
ForwardDiff = ">= 0.9"

?

5

What you’re seeing is conflicts from packages which depend on ForwardDiff (Zygote, LoopVectorization) and not the other way around. The first error should be easy enough to fix by removing non-utility packages like Zygote from your global environment and/or ] deving packages into a separate environment. However, that’s exactly what you’ve done for the second error (separate environment), and it looks like you’ve found the compat issue

It’s not very SemVer friendly, because ForwardDiff 0.9 → 0.10 → 0.11 are all breaking changes.

2 Likes
6

Yes. Best practice there is to not make breaking changes, so that you don’t need to bump ForwardDiff from 0.10 → 0.11.

But if you have to, you can dev other packages you want to use to update them, and then be prepared to make PRs to them, too.

I thought ColPrac demanded that, but couldn’t find any statement to that effect on the page.

1 Like
7

ColPrac does recommend GitHub - SciML/ColPrac: Contributor's Guide on Collaborative Practices for Community Packages which would’ve prevented the dependency clash here, but in FowardDiff’s case I believe the 0.11 version bump was done out of necessity to work around unforeseen backwards compat challenges.

8

Got it! Thank you, both.