Skip to content

[Image] | (DX) | Add delay to flakey Graphie image regression test#3506

Draft
nishasy wants to merge 5 commits intomainfrom
image-flakey-test
Draft

[Image] | (DX) | Add delay to flakey Graphie image regression test#3506
nishasy wants to merge 5 commits intomainfrom
image-flakey-test

Conversation

@nishasy
Copy link
Copy Markdown
Contributor

@nishasy nishasy commented Apr 16, 2026

Summary:

The ZoomClickedWithGraphieImage regression story keeps being reported by Chromatic as
having a ~2px difference from base.

This seems to be happening due to a timing issue. Adding a delay to see if it stops
getting flagged by Chromatic.

Issue: https://khanacademy.atlassian.net/browse/LEMS-4053

Test plan:

Wait and see if anything gets reported in this PR. Continue monitoring after.


In case anyone is curious, here's what Claude says about the cause:

margin-top: -18.5px → -16px (diff: 2.5px)
margin-left: -20px → -20px (stable)

Since margin-left is identical in both runs, the scale and normalizedWidth values are
consistent. Only normalizedHeight changed. Working backwards from setLabelMargins
(graphie.ts:1842):

marginTop = normalizedHeight × multipliers[1] × scale

With font-size: 100% implying scale = 1, the span's measured height itself differs by
~2.5px between runs (assuming the "above" direction with multiplier -1.0). Math.round()
on the margin output would just give you -19px vs -16px — the underlying measurement
is genuinely different, not a precision artifact.

What's actually varying: font loading timing

The formula is C\hphantom{C} — a KaTeX math expression. The label height is measured
via span.scrollHeight in the label() method (graphie.ts:~1016). If KaTeX's math fonts
haven't fully loaded when scrollHeight is first sampled, the browser measures the span
using a fallback system font, which can easily produce a 2–5px different line height
than the actual KaTeX font.

That first measurement is also stored as originalLabelSize (graphie.ts:1713-1715):

if ($span.data("originalLabelSize") == null) {
$span.data("originalLabelSize", [width, height]);
}

Once stored with the wrong height, every subsequent _recalculateLabels() call
(including the one triggered by the zoom click) reuses this baked-in wrong value.

★ Insight ─────────────────────────────────────
This is a classic FOUT (Flash of Unstyled Text) timing bug in visual tests. The
scrollHeight of a KaTeX-rendered element depends on whether math fonts have loaded.
Chromatic takes its screenshot non-deterministically relative to FontFace resolution,
so some runs measure with the fallback font and some with the real one — producing
genuinely different pixel values, not floating-point noise.
──────────────────────────────────────────

Better fixes ranked by invasiveness:

  1. Chromatic delay parameter (zero code change to widget logic) — add delay: 500 or
    pauseAnimationAtEnd: true to the story's chromatic parameters to let fonts settle
    before the screenshot.
  2. document.fonts.ready before measuring — in graphie.ts's label() method, defer the
    setLabelMargins call until after await document.fonts.ready, so scrollHeight is always
    measured with the correct font metrics.
  3. Re-measure on font load — use a FontFaceObserver or a ResizeObserver on the label
    itself to detect when KaTeX fonts cause a layout change and update
    originalLabelSize accordingly.

Option 1 is the right starting point since it confirms the diagnosis cheaply. If it
stops the flakiness, option 2 or 3 would be the proper long-term fix in the widget
code.

nishasy added 3 commits April 16, 2026 15:17
…e regression test

The `ZoomClickedWithGraphieImage` regression story keeps being reported by Chromatic as
having a ~2px difference from base.

This seems to be happening due to a timing issue. Adding a delay to see if it stops
getting flagged by Chromatic.

Issue: https://khanacademy.atlassian.net/browse/LEMS-4053

Test plan:
Wait and see if anything gets reported in this PR. Continue monitoring after.
@nishasy nishasy self-assigned this Apr 16, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

🗄️ Schema Change: No Changes ✅

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

Size Change: +18 B (0%)

Total Size: 499 kB

📦 View Changed
Filename Size Change
packages/perseus/dist/es/index.js 196 kB +18 B (+0.01%)
ℹ️ View Unchanged
Filename Size
packages/kas/dist/es/index.js 20.5 kB
packages/keypad-context/dist/es/index.js 1 kB
packages/kmath/dist/es/index.js 6.36 kB
packages/math-input/dist/es/index.js 98.5 kB
packages/math-input/dist/es/strings.js 1.61 kB
packages/perseus-core/dist/es/index.item-splitting.js 11.9 kB
packages/perseus-core/dist/es/index.js 25.1 kB
packages/perseus-editor/dist/es/index.js 102 kB
packages/perseus-linter/dist/es/index.js 9.3 kB
packages/perseus-score/dist/es/index.js 9.7 kB
packages/perseus-utils/dist/es/index.js 403 B
packages/perseus/dist/es/strings.js 8.27 kB
packages/pure-markdown/dist/es/index.js 1.39 kB
packages/simple-markdown/dist/es/index.js 6.71 kB

compressed-size-action

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

🛠️ Item Splitting: No Changes ✅

@nishasy nishasy requested a review from mark-fitzgerald April 16, 2026 22:25
@nishasy nishasy marked this pull request as ready for review April 16, 2026 22:25
Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

npm Snapshot: Published

Good news!! We've packaged up the latest commit from this PR (b76b3fa) and published it to npm. You
can install it using the tag PR3506.

Example:

pnpm add @khanacademy/perseus@PR3506

If you are working in Khan Academy's frontend, you can run the below command.

./dev/tools/bump_perseus_version.ts -t PR3506

If you are working in Khan Academy's webapp, you can run the below command.

./dev/tools/bump_perseus_version.js -t PR3506

Copy link
Copy Markdown
Contributor

@mark-fitzgerald mark-fitzgerald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fingers crossed.

@nishasy
Copy link
Copy Markdown
Contributor Author

nishasy commented Apr 16, 2026

Nope 😭 Back to the drawing board

image

@nishasy nishasy marked this pull request as draft April 16, 2026 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants