On the other hand, the purpose of the Test group is indeed to know whether or not our tests are passing, so failures of jobs in the Test group is bad. As you noticed, we have several non-deterministic failures that occur in jobs within the Test group. We’re working on fixing those, and eventually we’d like Buildkite to be consistently green on master, but we’re not there yet.
If anyone is interested in helping out, please join us in the #ci-dev channel on Slack.