This is an interaction between the upper bounds placed by the METADATA converter and https://github.com/JuliaLang/Pkg.jl/issues/110.
As it is right now, if a test dependency is not compatible with the versions of packages that gets resolved without the test dependency, the package will fail with a resolution error when tested.
The workaround is to check in a Manifest with versions that are compatible with the test dependencies. We should maybe think more about https://github.com/JuliaLang/Pkg.jl/issues/110 and loosen how hard the version of project dependencies are being kept.