8 Matching Annotations
  1. Jun 2021
    1. Thus, by adding system tests, we increase the maintenance costs for development and CI environments and introduce potential points of failures or instability: due to the complex setup, flakiness is the most common problem with end-to-end testing. And most of this flakiness comes from communication with a browser.
  2. Jul 2020
  3. May 2020
    1. The test is being marked as skipped because it has randomly failed. How much confidence do we have in that test and feature in the first place.
    2. This policy allows the test suite to stay green while letting individual teams decide when they would like to put in the effort to write more deterministic tests. They may choose to do so right away, or delay until they work on the feature again.
    3. All tests that fail on the master branch will be considered flakes. These tests will be marked as skipped.