Commit d1bf5c0
committed
CI: retry testem ci once on failure
The Browserstack job occasionally fails on transient browser-side
network errors (e.g. Safari `TypeError: Load failed` seen in PR #32
run 24655116088). Those failures come from the BrowserStack tunnel,
not from code under test.
On first failure, re-run the whole suite once; if that also fails
the original exit code propagates and the job fails normally.
Tradeoff: masks flaky tests with <=50% failure rate. Acceptable
here because the observed browserstack-only flakes are infra, not
product. For product flakiness we'd want per-test retry, which
testem does not provide natively.
Rejected alternatives:
- `retry_count: 1` in testem config — not a real testem option
(verified against testem 3.19.1 source and docs); silently
ignored.
- `.catch()` on `ember browserstack:results` / `:disconnect` —
`process.exit(0)` on the success path skips the finally entirely,
so the cleanup `.catch()` only fires when testem already failed.
The linked PR #32 failure was a real test error, not a cleanup
failure; the `.catch()` would not have changed the outcome.1 parent 2297ebc commit d1bf5c0
1 file changed
Lines changed: 13 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
29 | 40 | | |
30 | 41 | | |
31 | 42 | | |
| |||
0 commit comments