case-studies.md

E2E Silent-Pass Case Studies

These are self-selected, real merged upstream PRs found while reviewing Playwright/Cypress suites with e2e-skills/e2e-reviewer. They provide adoption and concrete case evidence, not a representative validation sample or an accuracy estimate. The common pattern: CI was green, but the test either asserted nothing, skipped the assertion, or checked a value that could never fail.

Carbon Design System: locator truthiness did not prove progress state

• Repo: carbon-design-system/carbon
• PR: carbon-design-system/carbon#22564 — merged
• Pattern: expect(page.locator(...)).toBeTruthy() asserted the Locator object, not the rendered step state.

- expect(page.locator('.cds--progress-step--complete')).toBeTruthy();
+ await expect(page.locator('.cds--progress-step--complete')).toBeVisible();

Why it matters: page.locator() returns a Locator handle even when no matching element is visible. The fixed assertion waits for user-visible progress state.

Storybook: unawaited actions and discarded checks

• Repo: storybookjs/storybook
• PR: storybookjs/storybook#34141 — merged
• Pattern: Playwright actions were not awaited, and point-in-time isVisible() reads were discarded.

Why it matters: Playwright promises must be awaited, and boolean reads do not become assertions unless their result is asserted.

code-server: focused test leak and non-asserting checks

• Repo: coder/code-server
• PR: coder/code-server#7845 — merged
• Pattern: it.only skipped part of the suite for months; several checks used matcher-less expect() calls or one-shot reads.

Why it matters: focused-test leaks silently remove coverage from CI, while matcher-less expect() calls create false confidence.

SvelteKit: floating web-first assertions

• Repo: sveltejs/kit
• PR: sveltejs/kit#16068 — merged
• Pattern: web-first assertions were created but not awaited.

- expect(page).toHaveURL(expectedUrl);
+ await expect(page).toHaveURL(expectedUrl);

Why it matters: a missing await leaves the assertion Promise unsequenced. Current Playwright workers normally surface a rejection, but attribution is degraded and a resolving assertion can race later work.

Strapi: discarded boolean reads

• Repo: strapi/strapi
• PR: strapi/strapi#26630 — merged
• Pattern: isVisible() / isHidden() / isEnabled() results were read and discarded.

Why it matters: reading a boolean is not a test assertion. The fixed tests assert user-visible state with awaited matchers.

Ghost: async disabled-state checks always passed

• Repo: TryGhost/Ghost
• PR: TryGhost/Ghost#28712 — merged
• Pattern: expect(likeButton.isDisabled()).toBeTruthy() checked a promise-like value instead of the button state.

Why it matters: Promise-valued state checks need await or a web-first assertion; otherwise a truthy Promise object can be asserted instead of the disabled state.

Cal.com: weak assertions and hard waits

• Repo: calcom/cal.diy
• PR: calcom/cal.diy#28486 — merged
• Pattern: weak assertions and fixed sleeps in E2E tests.

Why it matters: replacing hard waits with web-first assertions makes tests fail on real regressions instead of timing artifacts.

More merged examples

• usebruno/bruno#8317 — awaited a WebSocket visibility assertion so it actually ran.
• QwikDev/qwik#8777 — replaced discarded assertion promises, locator toBeDefined(), and bare locator checks.
• element-hq/element-web#32801 — fixed always-passing assertions, unawaited checks, toBeAttached() misuse, and dead code.
• mui/mui-x#22982 — replaced an always-true Locator null check with a user-visible edit assertion.
• rancher-sandbox/rancher-desktop#10557 — replaced getByText(...).not.toBeNull() (always true) with web-first visibility assertions for each WSL integration name.