Bizarre Pkg bug reports: different versions when testing to make tests pass?

Has anyone seen this behavior where you can get different versions of a package when testing than when you using? I haven’t gotten it, but I am getting bug reports about it and am not sure how to tell users how to handle it. Reference:

It seems to be Linux related, maybe. I reported it upstream but didn’t find out much:

so I am hoping I can crowd-source some more information here.

There is a bug right now that makes Pkg update too much. From that issue it looks like ForwardDiff gets updated when the tests are run so they pass, while the local version he has installed is not 1.0 ready.

But why would the local version not use ForwardDiff v0.8.5 if the tests do?

Because it spuriously upgraded when testing.

How is the upgrade when testing spurious? That’s what we want.

No, we want to use the same version as when you are using the package (the one in the Manifest).

But why is it grabbing the older version when using? That’s the actual issue. It’s getting the right one when testing. We don’t have an upper bound on it so I’m not sure why a few people on one operating system aren’t getting the right version (Pkg3’s environments stop conflicts in this kind of case?)

Something is holding ForwardDiff.jl back?