We already do exactly that. We test all registered Julia packages (registered in the General registry) prior to making a Julia release. We also do this every day on the latest Julia master. See:
- ANN: Nanosoldier package evaluation -- with badges!
- GitHub - JuliaCI/NanosoldierReports: A repository for human-readable reports generated by the Nanosoldier.jl CI system.
- GitHub - JuliaCI/Nanosoldier.jl: A package for running JuliaCI services on MIT's Nanosoldier cluster
- GitHub - JuliaCI/PkgEval.jl: Keeping tabs on the julia ecosystem