Page History

Flaky tests are the ever-present enemy of a build system. A flaky test is any test which does not consistently pass ( or fail). These tests reduce the confidence in our build. The presence of flaky failures also means that we rarely see "green builds" on Pull Requests or trunk builds. A small negative signal (one flaky failure) is amplified into a large negative signal (failing build). This means that committers become accustomed to ignoring the failing builds. The result is that other negative signals in the build (like a checkstyle failure, or compilation error) will some times get ignored.

...