You are right, the only concern is bugfixes. Small projects (\approx 1 person) may not have the reources to backport everything, so for those encountering bugs the advice is usually upgrading. Which is fine, but if the new package version also requires a higher version of Julia, it can set up a cascade of upgrades.
Still, with version control all such experiments are reversible, so it is not that painful.