I have this in my ci.yml and the occasional failure on the nightly build does not break the rest of CI.
- uses: julia-actions/julia-runtest@latest
continue-on-error: ${{ matrix.version == 'nightly' }}
The whole thing is at
https://github.com/ctkelley/SIAMFANLEquations.jl/blob/master/.github/workflows/ci.yml