Yes, I also had a fair share of troubles with CI because of a checked-in Manifest. Note also that if you have a checked-in Manifest and bump the compat bound of a package without updating the Manifest, you still test the old version.
Also, the Manifest is specific to a Julia version, which may not play very nicely with running CI across several Julia versions in some cases