Updated e2e tests to use official Playwright images

[troyciesco]

closes https://linear.app/ghost/issue/NY-973

Summary

Run E2E tests inside a prebuilt Playwright container instead of installing Playwright browsers on the CI host. This simplifies CI setup and ensures consistent browser versions.

Changes

CI Workflow:

• Pull mcr.microsoft.com/playwright:v{version}-noble image (version from e2e/package.json)
• Start Docker Compose services on the host before running tests
• Fetch Tinybird config on the host (required because compose CLI cannot run inside container)
• Run tests in the Playwright container with --network host for service access

E2E Framework:

• Add "external compose mode" (GHOST_E2E_COMPOSE_RUNNING=true) for when tests run inside a container
• Skip Docker Compose CLI commands in this mode (volume paths don't resolve inside container)
• Use dockerode API directly for container operations (works via mounted socket)

Why this approach?

Docker Compose volume mounts use host paths. When running docker compose up from inside a container, the paths in compose.yml (like ../docker/caddy/Caddyfile.e2e) don't exist. By starting compose on the host and running tests in a container with --network host, we get the best of both worlds.

Results

This change consistently shows a ~15-20 second improvement in CI run time.

This workflow run was on this branch when the PR was first opened, with no changes. It's worth nothing that we were making other improvements to the E2E tests while this PR was in flight, such as switching to 8 shards instead of 4 - but the important piece to look at in this case is the Setup Playwright step, which takes 42 seconds.

We can see the improvements from this PR in this workflow run. At first it may seem like there's more steps and they take longer than Setup Playwright from above, but "Start compose services" and "Fetch Tinybird config" were actually happening in roughly the same amount of time at the beginning of "Run e2e tests", and we just couldn't see those steps broken down. They have to happen outside of "Run tests in Playwright container" now because docker compose uses host paths for volume mounts, which don't resolve correctly when running docker compose from inside a container. The services must be started on the host where those paths exist.

Anyway, the real comparison is "Setup Playwright" from before to "Pull Playwright image" now. As you can see, 17 second improvement.

[coderabbitai]

Walkthrough

The CI workflow now reads the Playwright version from e2e/package.json, pulls the matching Playwright image, starts docker-compose services on the host (with health checks and one-shot handling), extracts Tinybird credentials into e2e/data/state/tinybird.json, and runs Playwright tests inside a dedicated Playwright container using host networking and mounted volumes. Artifact collection for blob/report and test-results is retained. In code, EnvironmentManager adds isExternalComposeMode() and cleanupResourcesWithoutCompose() and branches globalSetup/globalTeardown to support host-managed (external) compose environments.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: switching to official Playwright images for E2E tests instead of installing browsers on CI host.
Description check	✅ Passed	The description is well-detailed and directly related to the changeset, explaining the motivation, implementation details, and performance improvements.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch sb_e2e

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.67%. Comparing base (5e287c6) to head (ddf5d11).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #26037   +/-   ##
=======================================
  Coverage   72.66%   72.67%           
=======================================
  Files        1560     1560           
  Lines      120151   120151           
  Branches    14504    14507    +3     
=======================================
+ Hits        87305    87314    +9     
+ Misses      31825    31814   -11     
- Partials     1021     1023    +2     

Flag	Coverage Δ
admin-tests	51.77% <ø> (-0.01%)	⬇️
e2e-tests	72.67% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

E2E Tests Failed

To view the Playwright test report locally, run:

REPORT_DIR=$(mktemp -d) && gh run download 21488885071 -n playwright-report -D "$REPORT_DIR" && npx playwright show-report "$REPORT_DIR"

[github-actions]

E2E Tests Failed

To view the Playwright test report locally, run:

REPORT_DIR=$(mktemp -d) && gh run download 21501991442 -n playwright-report -D "$REPORT_DIR" && npx playwright show-report "$REPORT_DIR"

[github-actions]

E2E Tests Failed

To view the Playwright test report locally, run:

REPORT_DIR=$(mktemp -d) && gh run download 21501991442 -n playwright-report -D "$REPORT_DIR" && npx playwright show-report "$REPORT_DIR"

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In @.github/workflows/ci.yml:
- Around line 1179-1190: The current step that builds tinybird.json uses printf
with raw variables (workspace_id, admin_token, tracker_token) which can produce
invalid JSON if tokens contain special characters; replace the printf line with
a safe jq construction such as using jq -n --arg workspace "$workspace_id" --arg
admin "$admin_token" --arg tracker "$tracker_token" '{workspaceId:$workspace,
adminToken:$admin, trackerToken:$tracker}' > e2e/data/state/tinybird.json so jq
handles proper escaping and newlines; keep the upstream variable extraction
(workspace_id, admin_token, tracker_token) and ensure jq is available in the
tb-cli container or install it in the step before use.

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In @.github/workflows/ci.yml:
- Around line 1171-1178: The step "Start compose services" currently masks all
startup failures with "|| true"; replace that with a deterministic startup +
explicit health validation: run "docker compose -f e2e/compose.yml -p ghost-e2e
up -d --wait" without "|| true", then poll/inspect the declared critical
services (e.g., mysql, caddy, analytics and any long-running ghost service
names) using docker compose/dockerd health/ps or docker inspect to ensure their
health status is "healthy"; allow the one-shot service (ghost-migrations) to
complete/exit without failing the job, but fail the step (exit non-zero) if any
critical service is unhealthy or not running within a timeout so CI does not
proceed against broken services.

🧹 Nitpick comments (1)

.github/workflows/ci.yml (1)

1195-1206: Consider running the container as the host user to avoid root-owned artifacts.

Root-owned files can break artifact uploads or cleanup on the runner. Add --user "$(id -u):$(id -g)" to keep ownership consistent.

♻️ Proposed change

-          docker run --rm --network host \
+          docker run --rm --network host \
+            --user "$(id -u):$(id -g)" \
             -v /var/run/docker.sock:/var/run/docker.sock \
             -v ${{ github.workspace }}:/workspace \