Mostly it is because compat bounds of the packages that you have are old.
So Pkg manager downgrades one specific package so all the packages use compatible compat bounds.
This is also an issue with Flux. Colors compat bounds are old, but there are other packages that use newer Colors. This is when Pkg downgrades for example Flux to a point that it didn’t have any restriction or it was more relaxed.
If everyone use CompatHelper, and not to remove the support for older versions of a package for no reason, this issue gets resolved
You can also have different folders on your system, and activate there to keep separate environments that don’t clash with each other.
Yes, sure. The advantage of Julia Observer is that it gives you the list of reverse dependencies, so that you don’t have to go through the manifest file and look for packages that have the downgraded one among its deps (in this case DiffRules).
Regarding the version number, you can go to your local registry (.julia/registries etc.), and look into the Compat.toml file of the packages that depend on the one that was downgraded. It should be possible to create a script that does that search and tells you what packages are setting the upper compat limit.