# Automation to ensure green CI on master
**URL:** https://discourse.julialang.org/t/automation-to-ensure-green-ci-on-master/102616
**Category:** Internals & Design
**Created:** [August 8, 2023, 8:50pm UTC](https://discourse.julialang.org/t/automation-to-ensure-green-ci-on-master/102616 "2023-08-08T20:50:57Z")
**Posts on this page:** 4
**Page:** 2
### Author:  [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [August 9, 2023, 7:27pm UTC](https://discourse.julialang.org/t/automation-to-ensure-green-ci-on-master/102616/21 "2023-08-09T19:27:50Z")
And at least at one point, the second-most common was OS-specific failures. For example, my understanding is that certain aspects of OSX made the FileWatching stdlib likely to have rare failures; these failures are not observed on other platforms. There’s not much we can do when the underlying OS is supposed to pass us events, but doesn’t. IIRC, didn’t you (or someone) just have to relax certain tests on OSX?
---
### Author:  [@jar1](https://discourse.julialang.org/u/jar1)
#### Post date: [August 9, 2023, 7:49pm UTC](https://discourse.julialang.org/t/automation-to-ensure-green-ci-on-master/102616/22 "2023-08-09T19:49:01Z")
Relevant from another thread
> [@Making the review process more pleasant for new contributors](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/37):
>
> We don’t (deliberately) merge PRs that cause new failures. Most current causes of failures probably passed CI when they were initially submitted, but intermittent issues or hardware changes or subtle interactions with other changes might cause them to (sometimes) fail; if you have enough flaky tests and failure of just one is enough to call the CI run a failure, then CI failures are common.
>
> I’m not saying this is good, but chasing down the causes tends to be hard work and not that many people make this a core mission. If you want to find examples of PRs fixing flaky tests, a subset (probably only a minority) can be found with the “ci” label: [Pull requests · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pulls?q=is%3Apr+is%3Aclosed+label%3Aci+)
>
> Sometimes, though, things get through due to inadequate test coverage and only discovered later.
---
### Author:  [@jar1](https://discourse.julialang.org/u/jar1)
#### Post date: [August 10, 2023, 5:37pm UTC](https://discourse.julialang.org/t/automation-to-ensure-green-ci-on-master/102616/23 "2023-08-10T17:37:29Z")
## Proposal 1
In any PR, whenever a test fails for reasons irrelevant to the PR, do not merge before filing an issue labeled `test failure` referencing the test and the CI failure. Then anybody so inclined can go through the list of issues and decide whether it’s a bad test, bad code-under-test, or no-action-needed CI platform problem.
Thoughts, before I propose it on GitHub?
---
### Author:  [@gbaraldi](https://discourse.julialang.org/u/gbaraldi)
#### Post date: [August 10, 2023, 6:05pm UTC](https://discourse.julialang.org/t/automation-to-ensure-green-ci-on-master/102616/24 "2023-08-10T18:05:31Z")
I think that puts a burden into the PR maker that’s unnecessary. If you see a CI failure that you think is odd open an issue.
[Previous page](https://discourse.julialang.org/t/automation-to-ensure-green-ci-on-master/102616.md?page=1)