Skip to content

Add UI Automation Tests to All Features#1311

Open
scosman wants to merge 10 commits intomainfrom
scosman/ui_test_all
Open

Add UI Automation Tests to All Features#1311
scosman wants to merge 10 commits intomainfrom
scosman/ui_test_all

Conversation

@scosman
Copy link
Copy Markdown
Collaborator

@scosman scosman commented Apr 21, 2026

Summary by CodeRabbit

  • Tests

    • Added a comprehensive Playwright end-to-end test suite covering discovery, datasets, docs, models, prompts, optimization, fine-tuning, runs, setup flows, skills, tools, and sanity/inference checks.
    • Added test fixtures, global readiness setup, and local mock servers to support deterministic e2e runs.
  • Chores

    • Added CI workflow to run Playwright tests on push/pull requests.
    • Ignored test artifacts and added an npm script plus Playwright dev dependencies.

scosman and others added 9 commits April 19, 2026 21:30
Scaffolds the ActRight / Playwright harness for the Kiln web UI:

- Installs @playwright/test and @playwright/mcp (chromium only) in app/web_ui
- playwright.config.ts with testDir tests/e2e, baseURL http://localhost:6534,
  workers: 1 (shared backend), and a webServer array that starts the FastAPI
  dev_server on port 6535 with HOME redirected to a gitignored .e2e_home dir
  for state isolation, plus Vite on 6534
- .github/workflows/playwright.yml (monorepo-aware, installs uv + npm deps)
- Two sanity tests (act_sanity for install, act_app_loads for the full stack)
- Core fixtures in tests/e2e/fixtures.ts: freshState, registeredUser,
  seededProject, seededProjectWithTask -- backed by /api/projects, /api/tasks,
  and /api/settings, with act docstrings for ActRight

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Six new act-managed Playwright tests covering the edit task screen: load,
save, read-only schemas, clone button, delete flow, and breadcrumb nav.

Setup fixes needed to make browser-originated fetches work:
- Pass KILN_FRONTEND_PORT=6534 to the backend so its CORS allow-list
  matches the Vite dev port (defaulted to 5173).
- Add globalSetup that polls both servers with real requests, since
  the webServer url check is a one-shot probe and a page's own fetches
  can race a still-warming service.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Prevents leaked projects from failing the freshState assertion when tests
run serially against a shared backend.
Make fixtures really put the app into clean states, and validate each
fixture's UI goal in a browser so fixture tests exercise the same paths
(CORS, Vite readiness, ui_state redirect) that real UI tests hit.

- Centralize frontend/backend ports + URLs in tests/e2e/ports.ts.
- Rename freshState -> cleanBackend; actively delete projects and clear
  settings instead of only asserting empty state.
- Have registeredUser and seededProject depend on cleanBackend so every
  test starts from a reset.
- Roll ui_state localStorage priming into seededProjectWithTask; drop
  the primeUiState helper from act_edit_task.spec.ts.
- Rewrite act_fixtures.spec.ts to open a browser and assert the
  redirect/heading each fixture is supposed to produce.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
- Drop unused @playwright/mcp devDep (was pulling playwright 1.60.0-alpha
  transitively; .mcp.json uses npx @latest, so no manifest pin is needed)
- Pin @types/node to ^22.0.0 to match CI's Node LTS runtime
- Playwright workflow: add paths filter, cancel-in-progress concurrency,
  drop master branch, align setup-uv to @V3 with enable-cache (repo convention)

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Lets Playwright tests exercise Kiln's litellm-backed inference path
(POST /api/projects/:pid/tasks/:tid/run and friends) without making
real provider calls. The mock runs as a third webServer alongside
backend+frontend, registered in Kiln via /api/provider/openai_compatible.

Two new fixtures: mockInferenceProvider (queue/reset against the mock
admin endpoints) and connectedMockProvider (idempotent registration as
a Kiln openai_compatible provider). Sanity test runs a seeded task end
to end and asserts the mock saw exactly one chat-completion call.

Mock server is a small node:http app run via `node` directly — Node
22.6+ strips TS types natively, so no extra dev dep is needed.

Also extends .prettierignore to skip .act_right/ working state so the
pre-commit format hook stops tripping on it (mirrors the .act_right
ignore that already exists in root .gitignore).

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Stands in for api.kiln.tech (the hosted Kiln server that backs copilot,
specs, prompt-optimizer, chat, entitlements, etc.) so Playwright tests
can exercise the authenticated kiln_ai_server_client path without
hitting the real hosted backend.

Runs as a fourth webServer alongside backend + frontend + mock provider.
KILN_SERVER_BASE_URL is set on the backend webServer so the generated
client's base_url points at the mock; its existing env-var override
means no Python code changes are needed.

Two new fixtures: mockKilnServer (admin client with reset before/after)
and seededCopilotKey (POST /api/settings to persist kiln_copilot_api_key,
bypassing the real Kinde/OAuth signup flow for tests that just need the
backend to treat the user as copilot-connected).

Phase 2 scope is intentionally narrow: auth + a single probe
(/v1/check_entitlements). The server file documents that future test
authors extend the mock with the endpoints their tests need. The
sanity test asserts the mock received the call with
Authorization: Bearer <seeded key> and returned the default
entitlements payload.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Phase B/C of /act discover run. Authored and green against the live web UI:
- Setup wizard (flow, project creation, provider connection)
- Run, Dataset, Docs library, RAG configs, Extractors, Fine-tune export
- Data generation, Models page, Optimize run configs
- Prompts management and detail views
- Settings (projects, task import, providers, intro)
- Skills, Tools (management, Kiln task servers, assignment/editing)

174/174 passing. Fine-tune remote-provider detail page intentionally skipped
(needs paid external providers). Chat, Prompt optimization, and Specs/Evals
groups deferred — they require extending the kiln-server mock and must be
authored serially to avoid mock-code conflicts.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 21, 2026

📝 Walkthrough

Walkthrough

Adds Playwright end-to-end test infrastructure and ~30 E2E specs for the web UI, plus test fixtures, global setup, ports, mock provider and kiln-server implementations, Playwright config, CI workflow, and supporting npm/package changes to run and report tests.

Changes

Cohort / File(s) Summary
CI / project ignore
/.github/workflows/playwright.yml, /.gitignore, app/web_ui/.gitignore, app/web_ui/.prettierignore, app/web_ui/package.json
New GitHub Actions workflow for Playwright tests, npm test script and dev deps added, and Playwright/test artifacts and local act state ignored.
Playwright config & entry
app/web_ui/playwright.config.ts, app/web_ui/tests/e2e/global-setup.ts, app/web_ui/tests/e2e/ports.ts
Added Playwright config (single-worker CI, HTML report, chromium), multi-process webServer readiness gating, global readiness polling, and fixed local ports/URLs.
Fixtures & test harness
app/web_ui/tests/e2e/fixtures.ts
New Playwright fixtures for APIRequestContext, backend reset, user/project/task seeding, mock provider/kiln clients, localStorage bootstrapping, and exported typed fixtures.
Mock provider & kiln-server
app/web_ui/tests/e2e/mock_provider/*, app/web_ui/tests/e2e/mock_kiln_server/*
Added test-only mock HTTP servers (OpenAI-compatible and kiln-server), admin endpoints (__queue, __reset, __state), request recording/queueing, and client wrappers for test use.
Test bootstrap & utilities
app/web_ui/tests/e2e/mock_provider/client.ts, app/web_ui/tests/e2e/mock_kiln_server/client.ts, app/web_ui/tests/e2e/ports.ts
Client modules and port constants exported for tests to control and inspect mock servers.
Test global/specs (core)
app/web_ui/tests/e2e/act_app_loads.spec.ts, act_sanity.spec.ts, act_fixtures.spec.ts, act_edit_task.spec.ts, act_mock_inference.spec.ts, act_mock_kiln_server.spec.ts, act_mock_inference.spec.ts
Basic sanity, app-load, fixture-redirect, edit-task, and mock-server integration tests.
Feature E2E suites
app/web_ui/tests/e2e/act/... (many files, e.g., discover/*.spec.ts, setup-*.spec.ts, settings-*.spec.ts, tools-*.spec.ts, skills-management.spec.ts)
Large set of new Playwright specs (~30) covering data generation, datasets, docs library, extractors, fine-tuning, models, optimize, prompts, RAG, run page, settings, setup/onboarding, tools, skills, navigation regressions and more. (Files grouped under app/web_ui/tests/e2e/act/....)

Sequence Diagram(s)

(Skipped — changes primarily add test infra, mock servers, and specs; not introducing new runtime control-flow between production components.)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

Poem

🐰 Hopped in with fixtures, mocks in tow,

Playwright clicks where tests must go.
Servers queued and routes aligned,
Green checks hop in—CI’s kind.
A rabbit cheers: "E2E, let’s go!" 🥕

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The pull request description is entirely missing. The required template specifies sections for 'What does this PR do?', 'Related Issues', and 'Contributor License Agreement', but none are provided. Add a complete pull request description including: What does this PR do? (summary of UI test coverage), Related Issues (link if applicable), CLA confirmation, and completion of the checklists.
Docstring Coverage ⚠️ Warning Docstring coverage is 7.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Add UI Automation Tests to All Features' directly and clearly summarizes the main change: adding comprehensive Playwright E2E tests across all UI features.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch scosman/ui_test_all

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@socket-security
Copy link
Copy Markdown

socket-security Bot commented Apr 21, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatednpm/​@​types/​node@​20.12.12 ⏵ 22.19.17100 +110081 +195100
Addednpm/​@​playwright/​test@​1.59.110010010099100

View full report

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 21, 2026

📊 Coverage Report

Overall Coverage: 92%

Diff: origin/main...HEAD

No lines with coverage information in this diff.


Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a comprehensive suite of end-to-end tests for the Kiln web UI, covering various features such as dataset management, prompt engineering, skill management, and project setup. The tests are well-structured and use fixtures to manage state. My feedback focuses on improving the robustness of Playwright locators by recommending the use of semantic roles, aria-labels, or data-testids instead of fragile CSS/XPath selectors or index-based selection, and avoiding the use of { force: true } to ensure tests accurately reflect user interactions.

await page.locator("table tbody tr").first().click()

// Click the delete icon button
await page.locator('button:has(img[src$="/images/delete.svg"])').click()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This locator is fragile as it relies on the image's src attribute. This can easily break if asset paths are changed. This pattern is used in a few other new test files as well.

A more robust approach would be to add a data-testid or an aria-label to the button. For example, with aria-label="Delete", you could use page.getByRole("button", { name: "Delete" }).

Comment on lines +160 to +163
await advancedToggle
.locator("xpath=ancestor::div[contains(@class,'collapse')]")
.locator("input[type='checkbox']")
.check({ force: true })
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This locator is complex and uses XPath, which can be brittle and hard to read. Using { force: true } is also a code smell, as it can mask issues where an element isn't visible or interactive to a user.

Consider using a more user-facing locator. If the checkbox is properly associated with its label, you could use page.getByRole('checkbox', { name: 'Advanced Options' }). This would be more resilient to markup changes.

Comment on lines +471 to +479
await expect(
page.getByText("Finetune", { exact: false }).first(),
).toBeVisible()
await expect(
page.getByText("Evals", { exact: false }).first(),
).toBeVisible()
await expect(
page.getByText("Data Gen", { exact: false }).first(),
).toBeVisible()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using .first() on a generic page.getByText() locator can lead to flaky tests if the page content or layout changes. It's more robust to scope your assertions to the specific model card you're interested in.

Since you've already located the gpt4oCard, you can chain your assertions from there to ensure you're checking for badges within that specific card.

Suggested change
await expect(
page.getByText("Finetune", { exact: false }).first(),
).toBeVisible()
await expect(
page.getByText("Evals", { exact: false }).first(),
).toBeVisible()
await expect(
page.getByText("Data Gen", { exact: false }).first(),
).toBeVisible()
await expect(gpt4oCard.getByText("Finetune", { exact: false })).toBeVisible()
await expect(gpt4oCard.getByText("Evals", { exact: false })).toBeVisible()
await expect(gpt4oCard.getByText("Data Gen", { exact: false })).toBeVisible()

Comment on lines +229 to +232
const collapseCheckbox = page
.locator(".collapse", { hasText: "Advanced Options" })
.locator('input[type="checkbox"]')
await collapseCheckbox.click({ force: true })
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This locator is a bit complex and the use of { force: true } can hide potential issues with element visibility or interactivity. It's better to use locators that mimic user behavior.

If the checkbox is associated with the 'Advanced Options' text via a <label>, you could use a more robust locator like page.getByRole('checkbox', { name: 'Advanced Options' }) and remove the need for force: true.

Comment on lines +77 to +78
const buttons = page.locator("button").filter({ has: page.locator("svg") })
await buttons.nth(1).click()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This locator is brittle because it relies on the button containing an SVG and its order on the page (nth(1)). This can easily break with small UI changes. This pattern appears in a few places.

It would be more robust to add a unique attribute like an aria-label (e.g., "Next slide") to the button and select it with page.getByRole('button', { name: 'Next slide' }).

Comment on lines +151 to +154
const openRouterRow = page
.getByRole("heading", { name: /OpenRouter\.ai/ })
.locator("../..")
.getByRole("button", { name: "Connect" })
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using locator("../..") to traverse up the DOM is generally considered an anti-pattern in tests as it makes them fragile and dependent on a specific HTML structure.

A better approach is to add a data-testid to the container element for each provider row. For example: data-testid="provider-row-openrouter". Then you can write a much more stable locator like page.getByTestId('provider-row-openrouter').getByRole('button', { name: 'Connect' }).

Comment on lines +80 to +85
await page
.locator("button")
.filter({ has: page.locator("svg") })
.last()
.click()
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This locator is brittle as it relies on the button containing an SVG and being the last one on the page. This is likely to break if other buttons are added.

For better maintainability, add a unique identifier like an aria-label (e.g., "Next slide") to the button and use a more specific selector like page.getByRole('button', { name: 'Next slide' }).

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
app/web_ui/package.json (1)

20-29: ⚠️ Potential issue | 🟠 Major

Add engines field to enforce Node runtime version.

app/web_ui/playwright.config.ts runs mock servers directly via plain node on .ts files (lines 118, 127), which requires Node v22.18+ (for default type stripping in the v22 line) or v22.6+ with --experimental-strip-types. While the config includes a comment acknowledging this requirement, @types/node and the current package.json do not formally declare this constraint. This creates risk of silent failures when the package is used with incompatible Node versions.

Add the constraint to package.json:

Suggested fix
   "type": "module",
+  "engines": {
+    "node": ">=22.18.0"
+  },
   "dependencies": {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/package.json` around lines 20 - 29, Add a Node engine constraint
to app/web_ui's package.json to enforce the minimum Node version required by the
Playwright config's runtime .ts execution (see playwright.config.ts use of plain
node at the mock server invocations around lines 118 and 127); update
package.json to include an "engines" field specifying the minimum Node version
(e.g., "node": ">=22.18.0" or ">=22.6.0" if you plan to require
--experimental-strip-types) so package consumers and CI will fail fast on
incompatible runtimes.
🧹 Nitpick comments (2)
app/web_ui/tests/e2e/act/discover/settings-intro.spec.ts (1)

76-78: Fragile next-button selector.

page.locator("button").filter({ has: page.locator("svg") }).nth(1) depends on the "next arrow" being the second SVG-containing button on the whole page. Any future layout addition (navbar icon, close button, menu toggle, etc.) would silently shift the index and break these three tests in non-obvious ways. Consider adding a stable hook (e.g., aria-label="Next" or data-testid) on the carousel arrow buttons in the Svelte component and selecting by that here and in lines 118-121.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/tests/e2e/act/discover/settings-intro.spec.ts` around lines 76 -
78, The current fragile selector page.locator("button").filter({ has:
page.locator("svg") }).nth(1).click() relies on the next arrow being the second
SVG button and will break if layout changes; add a stable hook in the Svelte
carousel (e.g., aria-label="Next" or data-testid="carousel-next") on the
next-arrow button, then replace that locator (and the other similar SVG-button
locators in this test) with a stable selector referencing the new attribute
(e.g., getByRole/getByLabel or locator('[data-testid="carousel-next"]')) so the
tests target the explicit "Next" control instead of an index-based SVG button.
app/web_ui/tests/e2e/act/discover/prompts-management.spec.ts (1)

98-100: Use the seeded fixture value instead of duplicating it.

This keeps the test aligned with seededProjectWithTask if the fixture instruction changes.

Suggested cleanup
-    await expect(
-      page.getByText("ActRight fixture task instruction."),
-    ).toBeVisible()
+    await expect(page.getByText(task.instruction)).toBeVisible()
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/tests/e2e/act/discover/prompts-management.spec.ts` around lines 98
- 100, The test is asserting a hardcoded instruction string; change it to use
the seeded fixture value so the test follows seededProjectWithTask. Replace the
hardcoded text passed to page.getByText(...) with the
seededProjectWithTask.task.instruction (or the exact property name used in the
test fixture) so the expect(page.getByText(...)).toBeVisible() assertion
references the fixture value rather than duplicating the string.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@app/web_ui/tests/e2e/act/discover/rag-configuration.spec.ts`:
- Around line 150-269: The tests navigate to the wrong create route
(/create_rag_config) while the UI CTA uses /add_search_tool; update every
page.goto call in this spec (search for
page.goto(`/docs/rag_configs/${project.id}/create_rag_config`) occurrences
inside tests like "renders search configuration section with sub-config
selectors", "shows breadcrumbs with correct links", and "shows submit button and
advanced options section") to use
`/docs/rag_configs/${project.id}/add_search_tool` so the tests follow the actual
UI route.

In `@app/web_ui/tests/e2e/act/discover/setup-provider-connection.spec.ts`:
- Around line 57-77: The test is querying provider rows as headings and nested
"Connect" buttons, but connect_providers.svelte renders each provider row as a
button with the provider name text; update the assertions in
discover/setup-provider-connection.spec.ts to query provider rows by role
"button" (e.g., page.getByRole("button", { name: /OpenAI/ })) or by text, and
replace negative heading checks (e.g., Kiln Copilot, Weights & Biases) with
checks that the provider buttons are not visible or do not exist (use
getByRole/getByText or queryByRole/queryByText accordingly); make the same
updates for the other occurrences noted (the block around lines 151-155) so all
provider assertions match the button-based UI.

In `@app/web_ui/tests/e2e/act/discover/setup-wizard-flow.spec.ts`:
- Around line 154-162: Update the failing assertions to match the select-account
page UI: replace checks for "How do you want to use Kiln?", "Start Personal",
"Start Work" and the choice headings with assertions that the "Account Type"
heading is visible and that buttons or elements labeled "Personal" and "For
Work" are visible; locate the assertions using the existing helpers
(page.getByRole and page.getByText) shown in the diff and apply the same change
to the second occurrence mentioned (near lines 192-194).
- Around line 69-72: The test uses page.getByRole("link", { name: "Skip
Tutorial" }) but the rendered element on /setup/intro is a <button>, so update
all occurrences of the locator in this spec (any calls to page.getByRole with
name "Skip Tutorial") to use getByRole("button", { name: "Skip Tutorial" })
instead; search for page.getByRole(...) usages that reference "Skip Tutorial"
(including the repeated instances later in the file) and change the role string
from "link" to "button" so the locator matches the rendered element and the flow
proceeds.

In `@app/web_ui/tests/e2e/act/discover/tools-management.spec.ts`:
- Around line 133-138: Update the heading assertions to match the routed pages:
replace the expected name in the page.getByRole("heading", { name: ... }) calls
that currently assert "Connect Local MCP Server" and "Connect Remote MCP Server"
with "Add Local MCP Tool" and "Add Remote MCP Tool" respectively (preserve
exact: true). Locate those assertions (the getByRole("heading", ...) calls
around the two failing blocks) and update the strings so the tests expect the
actual <h1> rendered by the Add Local/Remote MCP Tool pages.

---

Outside diff comments:
In `@app/web_ui/package.json`:
- Around line 20-29: Add a Node engine constraint to app/web_ui's package.json
to enforce the minimum Node version required by the Playwright config's runtime
.ts execution (see playwright.config.ts use of plain node at the mock server
invocations around lines 118 and 127); update package.json to include an
"engines" field specifying the minimum Node version (e.g., "node": ">=22.18.0"
or ">=22.6.0" if you plan to require --experimental-strip-types) so package
consumers and CI will fail fast on incompatible runtimes.

---

Nitpick comments:
In `@app/web_ui/tests/e2e/act/discover/prompts-management.spec.ts`:
- Around line 98-100: The test is asserting a hardcoded instruction string;
change it to use the seeded fixture value so the test follows
seededProjectWithTask. Replace the hardcoded text passed to page.getByText(...)
with the seededProjectWithTask.task.instruction (or the exact property name used
in the test fixture) so the expect(page.getByText(...)).toBeVisible() assertion
references the fixture value rather than duplicating the string.

In `@app/web_ui/tests/e2e/act/discover/settings-intro.spec.ts`:
- Around line 76-78: The current fragile selector
page.locator("button").filter({ has: page.locator("svg") }).nth(1).click()
relies on the next arrow being the second SVG button and will break if layout
changes; add a stable hook in the Svelte carousel (e.g., aria-label="Next" or
data-testid="carousel-next") on the next-arrow button, then replace that locator
(and the other similar SVG-button locators in this test) with a stable selector
referencing the new attribute (e.g., getByRole/getByLabel or
locator('[data-testid="carousel-next"]')) so the tests target the explicit
"Next" control instead of an index-based SVG button.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3e3e4e9e-672e-4c9e-b908-6e835db7b97f

📥 Commits

Reviewing files that changed from the base of the PR and between ad997b3 and 797cfb2.

⛔ Files ignored due to path filters (1)
  • app/web_ui/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (41)
  • .github/workflows/playwright.yml
  • .gitignore
  • app/web_ui/.gitignore
  • app/web_ui/.prettierignore
  • app/web_ui/package.json
  • app/web_ui/playwright.config.ts
  • app/web_ui/tests/e2e/act/discover/data-generation.spec.ts
  • app/web_ui/tests/e2e/act/discover/dataset.spec.ts
  • app/web_ui/tests/e2e/act/discover/docs-library.spec.ts
  • app/web_ui/tests/e2e/act/discover/extractors.spec.ts
  • app/web_ui/tests/e2e/act/discover/fine-tuning.spec.ts
  • app/web_ui/tests/e2e/act/discover/models-page.spec.ts
  • app/web_ui/tests/e2e/act/discover/optimize-run-configs.spec.ts
  • app/web_ui/tests/e2e/act/discover/prompt-detail-views.spec.ts
  • app/web_ui/tests/e2e/act/discover/prompts-management.spec.ts
  • app/web_ui/tests/e2e/act/discover/rag-configuration.spec.ts
  • app/web_ui/tests/e2e/act/discover/run-page.spec.ts
  • app/web_ui/tests/e2e/act/discover/settings-intro.spec.ts
  • app/web_ui/tests/e2e/act/discover/settings-project-management.spec.ts
  • app/web_ui/tests/e2e/act/discover/settings-providers.spec.ts
  • app/web_ui/tests/e2e/act/discover/settings-task-import.spec.ts
  • app/web_ui/tests/e2e/act/discover/setup-project-creation.spec.ts
  • app/web_ui/tests/e2e/act/discover/setup-provider-connection.spec.ts
  • app/web_ui/tests/e2e/act/discover/setup-wizard-flow.spec.ts
  • app/web_ui/tests/e2e/act/discover/skills-management.spec.ts
  • app/web_ui/tests/e2e/act/discover/tools-assignment-editing.spec.ts
  • app/web_ui/tests/e2e/act/discover/tools-kiln-task-servers.spec.ts
  • app/web_ui/tests/e2e/act/discover/tools-management.spec.ts
  • app/web_ui/tests/e2e/act_app_loads.spec.ts
  • app/web_ui/tests/e2e/act_edit_task.spec.ts
  • app/web_ui/tests/e2e/act_fixtures.spec.ts
  • app/web_ui/tests/e2e/act_mock_inference.spec.ts
  • app/web_ui/tests/e2e/act_mock_kiln_server.spec.ts
  • app/web_ui/tests/e2e/act_sanity.spec.ts
  • app/web_ui/tests/e2e/fixtures.ts
  • app/web_ui/tests/e2e/global-setup.ts
  • app/web_ui/tests/e2e/mock_kiln_server/client.ts
  • app/web_ui/tests/e2e/mock_kiln_server/server.ts
  • app/web_ui/tests/e2e/mock_provider/client.ts
  • app/web_ui/tests/e2e/mock_provider/server.ts
  • app/web_ui/tests/e2e/ports.ts

Comment on lines +150 to +269
await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)

await expect(
page.getByRole("heading", { name: "Create Search Tool (RAG)" }),
).toBeVisible()

await expect(page.getByText("Part 1: Tool Properties")).toBeVisible()

await expect(page.locator("#tool_name")).toBeVisible()
await expect(page.locator("#tool_description")).toBeVisible()
})

/* @act
## Goals
The create RAG config form renders Part 2: Search Configuration with
sub-config selector dropdowns for Extractor, Chunker, Embedding Model,
Search Index, and Reranker.

## Fixtures
- registeredUser
- seededProjectWithTask

## Hints
- Part 2 heading reads "Part 2: Search Configuration".
- Selectors have ids: extractor_select, chunker_select, embedding_select,
vector_store_select, reranker_select.

## Assertions
- The "Part 2: Search Configuration" section heading is visible.
- Labels for Extractor, Chunker, Embedding Model, Search Index, and Reranker are visible.
*/
test("renders search configuration section with sub-config selectors", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask

await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)

await expect(page.getByText("Part 2: Search Configuration")).toBeVisible()

for (const label of [
"Extractor",
"Chunker",
"Embedding Model",
"Search Index",
"Reranker",
]) {
await expect(page.getByText(label, { exact: true }).first()).toBeVisible()
}
})

/* @act
## Goals
The create RAG config page shows correct breadcrumbs linking back through
the navigation hierarchy.

## Fixtures
- registeredUser
- seededProjectWithTask

## Hints
- Breadcrumbs: Optimize, Docs & Search, Search Tools, Add Search Tool.
- "Search Tools" links to /docs/rag_configs/{project_id}.
- "Docs & Search" links to /docs/{project_id}.

## Assertions
- The "Search Tools" breadcrumb links to /docs/rag_configs/{project_id}.
- The "Docs & Search" breadcrumb links to /docs/{project_id}.
*/
test("shows breadcrumbs with correct links", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask

await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)

const breadcrumbs = page.locator(".breadcrumbs")

await expect(
breadcrumbs.getByRole("link", { name: "Search Tools" }),
).toHaveAttribute("href", `/docs/rag_configs/${project.id}`)

await expect(
breadcrumbs.getByRole("link", { name: "Docs & Search" }),
).toHaveAttribute("href", `/docs/${project.id}`)
})

/* @act
## Goals
The form has a "Create Search Tool" submit button that is visible and
the form shows an Advanced Options collapsible section with Reference Name
and Reference Description fields.

## Fixtures
- registeredUser
- seededProjectWithTask

## Hints
- Submit button label is "Create Search Tool" from FormContainer.
- Advanced Options is a Collapse component with fields rag_config_name and rag_config_description.

## Assertions
- A "Create Search Tool" button is visible.
- An "Advanced Options" section is present on the page.
*/
test("shows submit button and advanced options section", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask

await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Use the same create route that the UI links to.

This spec asserts the empty-state CTA points to /add_search_tool, but the create-page tests navigate to /create_rag_config. That can either 404 or leave the actual CTA route untested.

Proposed fix
-    await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
+    await page.goto(`/docs/rag_configs/${project.id}/add_search_tool`)
@@
-    await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
+    await page.goto(`/docs/rag_configs/${project.id}/add_search_tool`)
@@
-    await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
+    await page.goto(`/docs/rag_configs/${project.id}/add_search_tool`)
@@
-    await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
+    await page.goto(`/docs/rag_configs/${project.id}/add_search_tool`)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
await expect(
page.getByRole("heading", { name: "Create Search Tool (RAG)" }),
).toBeVisible()
await expect(page.getByText("Part 1: Tool Properties")).toBeVisible()
await expect(page.locator("#tool_name")).toBeVisible()
await expect(page.locator("#tool_description")).toBeVisible()
})
/* @act
## Goals
The create RAG config form renders Part 2: Search Configuration with
sub-config selector dropdowns for Extractor, Chunker, Embedding Model,
Search Index, and Reranker.
## Fixtures
- registeredUser
- seededProjectWithTask
## Hints
- Part 2 heading reads "Part 2: Search Configuration".
- Selectors have ids: extractor_select, chunker_select, embedding_select,
vector_store_select, reranker_select.
## Assertions
- The "Part 2: Search Configuration" section heading is visible.
- Labels for Extractor, Chunker, Embedding Model, Search Index, and Reranker are visible.
*/
test("renders search configuration section with sub-config selectors", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask
await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
await expect(page.getByText("Part 2: Search Configuration")).toBeVisible()
for (const label of [
"Extractor",
"Chunker",
"Embedding Model",
"Search Index",
"Reranker",
]) {
await expect(page.getByText(label, { exact: true }).first()).toBeVisible()
}
})
/* @act
## Goals
The create RAG config page shows correct breadcrumbs linking back through
the navigation hierarchy.
## Fixtures
- registeredUser
- seededProjectWithTask
## Hints
- Breadcrumbs: Optimize, Docs & Search, Search Tools, Add Search Tool.
- "Search Tools" links to /docs/rag_configs/{project_id}.
- "Docs & Search" links to /docs/{project_id}.
## Assertions
- The "Search Tools" breadcrumb links to /docs/rag_configs/{project_id}.
- The "Docs & Search" breadcrumb links to /docs/{project_id}.
*/
test("shows breadcrumbs with correct links", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask
await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
const breadcrumbs = page.locator(".breadcrumbs")
await expect(
breadcrumbs.getByRole("link", { name: "Search Tools" }),
).toHaveAttribute("href", `/docs/rag_configs/${project.id}`)
await expect(
breadcrumbs.getByRole("link", { name: "Docs & Search" }),
).toHaveAttribute("href", `/docs/${project.id}`)
})
/* @act
## Goals
The form has a "Create Search Tool" submit button that is visible and
the form shows an Advanced Options collapsible section with Reference Name
and Reference Description fields.
## Fixtures
- registeredUser
- seededProjectWithTask
## Hints
- Submit button label is "Create Search Tool" from FormContainer.
- Advanced Options is a Collapse component with fields rag_config_name and rag_config_description.
## Assertions
- A "Create Search Tool" button is visible.
- An "Advanced Options" section is present on the page.
*/
test("shows submit button and advanced options section", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask
await page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
await page.goto(`/docs/rag_configs/${project.id}/add_search_tool`)
await expect(
page.getByRole("heading", { name: "Create Search Tool (RAG)" }),
).toBeVisible()
await expect(page.getByText("Part 1: Tool Properties")).toBeVisible()
await expect(page.locator("#tool_name")).toBeVisible()
await expect(page.locator("#tool_description")).toBeVisible()
})
/* `@act`
## Goals
The create RAG config form renders Part 2: Search Configuration with
sub-config selector dropdowns for Extractor, Chunker, Embedding Model,
Search Index, and Reranker.
## Fixtures
- registeredUser
- seededProjectWithTask
## Hints
- Part 2 heading reads "Part 2: Search Configuration".
- Selectors have ids: extractor_select, chunker_select, embedding_select,
vector_store_select, reranker_select.
## Assertions
- The "Part 2: Search Configuration" section heading is visible.
- Labels for Extractor, Chunker, Embedding Model, Search Index, and Reranker are visible.
*/
test("renders search configuration section with sub-config selectors", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask
await page.goto(`/docs/rag_configs/${project.id}/add_search_tool`)
await expect(page.getByText("Part 2: Search Configuration")).toBeVisible()
for (const label of [
"Extractor",
"Chunker",
"Embedding Model",
"Search Index",
"Reranker",
]) {
await expect(page.getByText(label, { exact: true }).first()).toBeVisible()
}
})
/* `@act`
## Goals
The create RAG config page shows correct breadcrumbs linking back through
the navigation hierarchy.
## Fixtures
- registeredUser
- seededProjectWithTask
## Hints
- Breadcrumbs: Optimize, Docs & Search, Search Tools, Add Search Tool.
- "Search Tools" links to /docs/rag_configs/{project_id}.
- "Docs & Search" links to /docs/{project_id}.
## Assertions
- The "Search Tools" breadcrumb links to /docs/rag_configs/{project_id}.
- The "Docs & Search" breadcrumb links to /docs/{project_id}.
*/
test("shows breadcrumbs with correct links", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask
await page.goto(`/docs/rag_configs/${project.id}/add_search_tool`)
const breadcrumbs = page.locator(".breadcrumbs")
await expect(
breadcrumbs.getByRole("link", { name: "Search Tools" }),
).toHaveAttribute("href", `/docs/rag_configs/${project.id}`)
await expect(
breadcrumbs.getByRole("link", { name: "Docs & Search" }),
).toHaveAttribute("href", `/docs/${project.id}`)
})
/* `@act`
## Goals
The form has a "Create Search Tool" submit button that is visible and
the form shows an Advanced Options collapsible section with Reference Name
and Reference Description fields.
## Fixtures
- registeredUser
- seededProjectWithTask
## Hints
- Submit button label is "Create Search Tool" from FormContainer.
- Advanced Options is a Collapse component with fields rag_config_name and rag_config_description.
## Assertions
- A "Create Search Tool" button is visible.
- An "Advanced Options" section is present on the page.
*/
test("shows submit button and advanced options section", async ({
page,
registeredUser,
seededProjectWithTask,
}) => {
void registeredUser
const { project } = seededProjectWithTask
await page.goto(`/docs/rag_configs/${project.id}/add_search_tool`)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/tests/e2e/act/discover/rag-configuration.spec.ts` around lines 150
- 269, The tests navigate to the wrong create route (/create_rag_config) while
the UI CTA uses /add_search_tool; update every page.goto call in this spec
(search for page.goto(`/docs/rag_configs/${project.id}/create_rag_config`)
occurrences inside tests like "renders search configuration section with
sub-config selectors", "shows breadcrumbs with correct links", and "shows submit
button and advanced options section") to use
`/docs/rag_configs/${project.id}/add_search_tool` so the tests follow the actual
UI route.

Comment on lines +57 to +77
await expect(
page.getByRole("heading", { name: /OpenRouter\.ai/ }),
).toBeVisible()
await expect(page.getByText("Recommended").first()).toBeVisible()
await expect(
page.getByRole("heading", { name: "OpenAI", exact: true }),
).toBeVisible()
await expect(
page.getByRole("heading", { name: "Ollama", exact: true }),
).toBeVisible()

await expect(
page.getByRole("button", { name: "Connect" }).first(),
).toBeVisible()

await expect(
page.getByRole("heading", { name: "Kiln Copilot" }),
).not.toBeVisible()
await expect(
page.getByRole("heading", { name: "Weights & Biases" }),
).not.toBeVisible()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Match the provider row roles used by the UI.

connect_providers.svelte renders provider rows as buttons containing text, not headings with nested Connect buttons. These selectors will fail for visible providers, and the negative heading checks can miss providers rendered as non-headings.

Suggested selector update
-    await expect(
-      page.getByRole("heading", { name: /OpenRouter\.ai/ }),
-    ).toBeVisible()
+    await expect(
+      page.getByRole("button", { name: /OpenRouter\.ai/ }),
+    ).toBeVisible()
     await expect(page.getByText("Recommended").first()).toBeVisible()
     await expect(
-      page.getByRole("heading", { name: "OpenAI", exact: true }),
+      page.getByRole("button", { name: /OpenAI/ }),
     ).toBeVisible()
     await expect(
-      page.getByRole("heading", { name: "Ollama", exact: true }),
+      page.getByRole("button", { name: /Ollama/ }),
     ).toBeVisible()
 
     await expect(
-      page.getByRole("button", { name: "Connect" }).first(),
+      page.getByRole("button", { name: /OpenRouter\.ai/ }),
     ).toBeVisible()
 
-    await expect(
-      page.getByRole("heading", { name: "Kiln Copilot" }),
-    ).not.toBeVisible()
-    await expect(
-      page.getByRole("heading", { name: "Weights & Biases" }),
-    ).not.toBeVisible()
+    await expect(page.getByText("Kiln Copilot")).not.toBeVisible()
+    await expect(page.getByText("Weights & Biases")).not.toBeVisible()
-    const openRouterRow = page
-      .getByRole("heading", { name: /OpenRouter\.ai/ })
-      .locator("../..")
-      .getByRole("button", { name: "Connect" })
+    const openRouterRow = page.getByRole("button", {
+      name: /OpenRouter\.ai/,
+    })

Also applies to: 151-155

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/tests/e2e/act/discover/setup-provider-connection.spec.ts` around
lines 57 - 77, The test is querying provider rows as headings and nested
"Connect" buttons, but connect_providers.svelte renders each provider row as a
button with the provider name text; update the assertions in
discover/setup-provider-connection.spec.ts to query provider rows by role
"button" (e.g., page.getByRole("button", { name: /OpenAI/ })) or by text, and
replace negative heading checks (e.g., Kiln Copilot, Weights & Biases) with
checks that the provider buttons are not visible or do not exist (use
getByRole/getByText or queryByRole/queryByText accordingly); make the same
updates for the other occurrences noted (the block around lines 151-155) so all
provider assertions match the button-based UI.

Comment on lines +69 to +72
// Should show Skip Tutorial initially, not Continue
await expect(
page.getByRole("link", { name: "Skip Tutorial" }),
).toBeVisible()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Use the rendered button role for Skip Tutorial.

/setup/intro renders Skip Tutorial as a <button>, so these getByRole("link") locators will not match and the setup-flow tests will time out before reaching the next step.

Proposed fix
-      page.getByRole("link", { name: "Skip Tutorial" }),
+      page.getByRole("button", { name: "Skip Tutorial" }),
-    await page.getByRole("link", { name: "Skip Tutorial" }).click()
+    await page.getByRole("button", { name: "Skip Tutorial" }).click()

Also applies to: 116-120, 151-152, 189-190, 234-235

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/tests/e2e/act/discover/setup-wizard-flow.spec.ts` around lines 69
- 72, The test uses page.getByRole("link", { name: "Skip Tutorial" }) but the
rendered element on /setup/intro is a <button>, so update all occurrences of the
locator in this spec (any calls to page.getByRole with name "Skip Tutorial") to
use getByRole("button", { name: "Skip Tutorial" }) instead; search for
page.getByRole(...) usages that reference "Skip Tutorial" (including the
repeated instances later in the file) and change the role string from "link" to
"button" so the locator matches the rendered element and the flow proceeds.

Comment on lines +154 to +162
await expect(
page.getByRole("heading", { name: "How do you want to use Kiln?" }),
).toBeVisible()

await expect(page.getByRole("heading", { name: "Personal" })).toBeVisible()
await expect(page.getByText("Start Personal")).toBeVisible()

await expect(page.getByRole("heading", { name: "For Work" })).toBeVisible()
await expect(page.getByText("Start Work")).toBeVisible()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Align select-account assertions with the current page UI.

The select-account page renders Account Type plus Personal / For Work buttons; it does not render How do you want to use Kiln?, Start Personal, Start Work, or choice headings. These assertions will fail after navigation succeeds.

Proposed fix
-    await expect(
-      page.getByRole("heading", { name: "How do you want to use Kiln?" }),
-    ).toBeVisible()
+    await expect(
+      page.getByRole("heading", { name: "Account Type", exact: true }),
+    ).toBeVisible()

-    await expect(page.getByRole("heading", { name: "Personal" })).toBeVisible()
-    await expect(page.getByText("Start Personal")).toBeVisible()
+    await expect(
+      page.getByRole("button", { name: "Personal", exact: true }),
+    ).toBeVisible()

-    await expect(page.getByRole("heading", { name: "For Work" })).toBeVisible()
-    await expect(page.getByText("Start Work")).toBeVisible()
+    await expect(
+      page.getByRole("button", { name: "For Work", exact: true }),
+    ).toBeVisible()
-    await expect(
-      page.getByRole("heading", { name: "How do you want to use Kiln?" }),
-    ).toBeVisible()
+    await expect(
+      page.getByRole("heading", { name: "Account Type", exact: true }),
+    ).toBeVisible()

Also applies to: 192-194

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/tests/e2e/act/discover/setup-wizard-flow.spec.ts` around lines 154
- 162, Update the failing assertions to match the select-account page UI:
replace checks for "How do you want to use Kiln?", "Start Personal", "Start
Work" and the choice headings with assertions that the "Account Type" heading is
visible and that buttons or elements labeled "Personal" and "For Work" are
visible; locate the assertions using the existing helpers (page.getByRole and
page.getByText) shown in the diff and apply the same change to the second
occurrence mentioned (near lines 192-194).

Comment on lines +133 to +138
await expect(
page.getByRole("heading", {
name: "Connect Local MCP Server",
exact: true,
}),
).toBeVisible()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Align the MCP page heading assertions with the routed pages.

The provided route implementations render Add Local MCP Tool and Add Remote MCP Tool as the page <h1>, but these tests assert Connect Local MCP Server / Connect Remote MCP Server, so the new E2E suite can fail on these pages.

Proposed fix
     await expect(
       page.getByRole("heading", {
-        name: "Connect Local MCP Server",
+        name: "Add Local MCP Tool",
         exact: true,
       }),
     ).toBeVisible()
@@
     await expect(
       page.getByRole("heading", {
-        name: "Connect Remote MCP Server",
+        name: "Add Remote MCP Tool",
         exact: true,
       }),
     ).toBeVisible()

Also applies to: 187-192

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/tests/e2e/act/discover/tools-management.spec.ts` around lines 133
- 138, Update the heading assertions to match the routed pages: replace the
expected name in the page.getByRole("heading", { name: ... }) calls that
currently assert "Connect Local MCP Server" and "Connect Remote MCP Server" with
"Add Local MCP Tool" and "Add Remote MCP Tool" respectively (preserve exact:
true). Locate those assertions (the getByRole("heading", ...) calls around the
two failing blocks) and update the strings so the tests expect the actual <h1>
rendered by the Add Local/Remote MCP Tool pages.

Three Playwright tests that reproduce known navigation bugs via
in-app anchor clicks (page.goto would reload and mask each bug):
- KIL-521: deeplink to prompt created after parent /prompts page load
  shows "Prompt not found." because the saved detail route never
  force-refreshes current_task_prompts.
- KIL-516 spec-to-spec: specB heading never appears after same-route
  nav from specA because the spec detail page only loads in onMount.
- KIL-516 compare columns: grid-template-columns stays at repeat(2,1fr)
  after nav to ?columns=4 because initializeFromURL runs only in onMount.

All three fail today and will pass once the pages react to \$page.params
changes instead of fetching only in onMount.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
app/web_ui/tests/e2e/act_navigation_bugs.spec.ts (1)

205-222: .first() on a broad style selector may bind to a stale/unintended element.

[style*="grid-template-columns"] matches any element whose inline style mentions that property (including non-row containers), and .filter({ hasText: /.*/ }) is effectively a no-op (matches anything, even empty text). After the client-side ?columns=4 navigation the component re-renders, and relying on .first() to point at the header grid row is brittle: if element order changes, or another grid appears while loading, the assertion on line 222 could check the wrong node (or still observe the old repeat(2, 1fr) during a transient render and flake).

Consider tightening the selector to the header row specifically (e.g., via a test id / role / a unique style fragment like 200px repeat), and dropping the hasText filter:

Proposed tightening
-  const gridRow = page
-    .locator('[style*="grid-template-columns"]')
-    .filter({ hasText: /.*/ })
-    .first()
+  const gridRow = page.locator('[style*="200px repeat"]').first()
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/web_ui/tests/e2e/act_navigation_bugs.spec.ts` around lines 205 - 222, The
test's locator gridRow currently uses a broad selector
'[style*="grid-template-columns"]' with .filter(...).first(), which can bind to
a stale or wrong element after navigation; replace it with a specific selector
for the header row (e.g., a dedicated data-testid or role or a unique
inline-style fragment such as 'grid-template-columns: 200px repeat' instead of
the generic substring), remove the .filter and .first usage, and update the
subsequent expects (the expect(gridRow)... checks) to use the new specific
locator so the assertion always targets the header grid element (refer to the
gridRow variable, the '[style*="grid-template-columns"]' selector, and the
.filter/.first calls to locate the change).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@app/web_ui/tests/e2e/act_navigation_bugs.spec.ts`:
- Around line 205-222: The test's locator gridRow currently uses a broad
selector '[style*="grid-template-columns"]' with .filter(...).first(), which can
bind to a stale or wrong element after navigation; replace it with a specific
selector for the header row (e.g., a dedicated data-testid or role or a unique
inline-style fragment such as 'grid-template-columns: 200px repeat' instead of
the generic substring), remove the .filter and .first usage, and update the
subsequent expects (the expect(gridRow)... checks) to use the new specific
locator so the assertion always targets the header grid element (refer to the
gridRow variable, the '[style*="grid-template-columns"]' selector, and the
.filter/.first calls to locate the change).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7500866b-5498-4f32-9c1c-ff02b67b0d85

📥 Commits

Reviewing files that changed from the base of the PR and between 797cfb2 and 01d484f.

📒 Files selected for processing (1)
  • app/web_ui/tests/e2e/act_navigation_bugs.spec.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant