Playwright as a measuring instrument — the i18n defect no static check can reach

Introduction

I asked why there was a browser in my blog’s toolchain. It is a static site with no tests, and a 95 MB Chromium download had just appeared in it, so the question was reasonable.

The answer turned out to be more interesting than the feature that prompted it. Diagrams on this site are components with their text in a per-locale record, so one drawing renders in English, Japanese and Traditional Chinese. The geometry is written once. A Japanese label that is wider than the box it was sized for overflows, on the Japanese site only, and every check the project had was blind to it — not because the checks were weak, but because the fact they would need does not exist until something renders.

This article is about that gap: which defects a type checker and a linter can reach, which ones only a layout engine can, and the two bugs that let my render-time check report 37 pages clean while measuring nothing at all.

What the existing checks could already see

The project was not short of checks. pnpm check runs astro check, tsc --noEmit, and a cross-locale consistency script. A figure’s labels look like this:

const LABELS = {
	en:      { flexItem: 'flex item' },
	ja:      { flexItem: 'フレックスアイテム' },
	'zh-tw': { flexItem: 'flex 項目' },
} as const satisfies Record<Locale, unknown>;

That satisfies is doing real work. Delete the ja key and astro check fails, which is exactly what you want, because a missing locale renders a blank diagram. The consistency script catches its own class of problem: an article whose locales disagree on tags, or a figure referenced from one language file and not another.

Now the geometry, in the same file:

<rect class="pill" x="316" width="124" />
<text x="378">{t.flexItem}</text>

Every check above passes. The record has all three locales, the types line up, the reference resolves. And 124 was chosen while looking at flex item, which is the shortest of the three strings. Nothing in the toolchain has any opinion about whether フレックスアイテム fits inside it.

The type checker can see that a locale exists. Whether its string fits is not a property of the source at all.

This is not a gap in the tools. A static check reads what you wrote; this defect is a property of what the browser draws. The two are separated by font loading, glyph shaping and layout, none of which have run yet.

Measuring a string means resolving its font

The tempting shortcut is to skip rendering and estimate: count the characters, treat CJK as double-width, compare against the box. It is wrong here, for a reason specific to this site.

html[lang^='ja'] { --font-sans: … 'Hiragino Sans', 'Noto Sans JP', … }
html[lang^='zh'] { --font-sans: … 'PingFang TC',  'Noto Sans TC', … }

Those stacks differ on purpose. Han characters are unified in Unicode but drawn differently by language, so a single merged stack renders Traditional Chinese pages with Japanese glyph shapes. The consequence for measurement is that the same character has different metrics in ja and zh-TW. Width is not a function of the string. It is a function of the string and whichever font actually resolved, on the machine doing the drawing.

SVG already exposes the answer. getBBox() on a <text> node returns its real rendered box, in user units, after shaping. It just does not exist outside a layout engine, and jsdom does not lay out or shape text.

So the browser is not there to drive the UI. It is there as a ruler. That reframing is what made the dependency easy to justify: Playwright opens the page, and the only thing asked of it is arithmetic on boxes.

const box = (textEl as SVGGraphicsElement).getBBox();

const spillX = Math.max(bounds.x - box.x, box.x + box.width - (bounds.x + bounds.width));
const spillY = Math.max(bounds.y - box.y, box.y + box.height - (bounds.y + bounds.height));

The rest of the script is bookkeeping: walk every article with figures, visit each locale that actually has a file, and compare every <text> against the <rect> it belongs to. A two-unit tolerance absorbs rounding, which is well under a character at these sizes.

It found nothing. 37 pages, all clean. Which was the correct answer, and also, at that moment, the only answer the script was capable of giving.

Two bugs that made the check pass everything

Both were written by the agent building the check, and both had the same shape: the comparison quietly stopped happening, and silence was indistinguishable from success.

The container it lost

Each label is measured against the box it sits in, so the script has to decide which <rect> that is. The first version picked the smallest rect containing the label’s centre, which sounds right and reads right.

Containment is only true while the label still fits. Once it escapes, the box it belonged to is no longer a candidate, and the comparison falls back to the whole figure.

A label that has escaped its card is no longer inside it. The container disappeared from the candidate list, the comparison fell back to the SVG viewBox, and a label sitting well outside its box was compared against the figure’s outer bounds, which it comfortably fits. The failure mode was inverted: the worse the overflow, the less likely it was to be reported.

The fix is to stop asking about containment and ask about overlap. Whichever rect the label overlaps most is the one it belongs to, and that holds while any part of the label is still over the box:

const holder = rects
	.map((r) => {
		const ox = Math.min(box.x + box.width, r.box.x + r.box.width) - Math.max(box.x, r.box.x);
		const oy = Math.min(box.y + box.height, r.box.y + r.box.height) - Math.max(box.y, r.box.y);
		return { ...r, area: ox > 0 && oy > 0 ? ox * oy : 0 };
	})
	.filter((r) => r.area > 0)
	.sort((a, b) => b.area - a.area || a.box.width * a.box.height - b.box.width * b.box.height)[0];

The tie-break matters more than it looks. A label inside a chip is also inside the card holding the chip and the panel holding the card, and all three overlaps equal the label’s own area. Sorting ties by the smaller rect is what picks the chip.

The spread that returned nothing

Refactoring to add a vertical check, the agent writing it replaced an explicit read of the container’s fields with a spread:

// Before: worked.
const bounds = { x: holder.box.x, width: holder.box.width };

// After: silently returns an empty object.
const bounds = { ...holder.box };

getBBox() returns an SVGRect, and its x, y, width and height are prototype accessors rather than own properties. Object spread copies own enumerable properties, so it copies nothing. Every bound became undefined.

Four steps from a spread to a green result, none of which raise anything. The comparison at the end is the only place it could have surfaced, and NaN makes it false.

undefined in the arithmetic produced NaN. And NaN > 2 is false, as is every other comparison you could write against it, so the overflow branch was unreachable. No exception, no warning. The script visited all 37 pages, measured each one into NaN, compared it to the tolerance, concluded everything fitted, and exited zero.

Both bugs failed open. That is the property they share and the reason either one was dangerous: a check that fails closed produces noise you will investigate, and a check that fails open produces a green result you will trust.

Breaking it on purpose

What caught both was the same discipline, applied late: make the check fail before believing it can. A deliberately overlong label was pasted into a figure and the script run against it.

over  playwright-i18n-svg-label-overflow-render-check [en]
      "Right CSS, wrong box, and a deliberately overlong heading to prove it fires"
      overflows its box horizontally by 176u

That is the horizontal case. The vertical one needed its own deliberate break, four extra lines pushed out the bottom of a card, and reported overflows its box vertically by 7u. Each axis needed its own negative test, because the spread bug disabled both while the earlier containment bug affected only one, and a passing run cannot tell you which of those you are looking at.

The order is what matters. Both clean sweeps came before the negative tests, and both were meaningless. A green run from a check whose failure path has never executed is not evidence about the code, only about itself.

What it is wired into

Deliberately not pnpm check. That job gates production deploys, and a figure changes on a small fraction of commits, so putting a browser download in front of every release trades a lot of reliability for very little coverage. The check runs as pnpm figures:fit, and the publish step blocks on it, which is the moment a broken Japanese label would actually go live.

The tolerance for a defect like this is asymmetric. It is invisible to everyone who reviews the article and visible to everyone who reads it, because review happens in the source language and the source language is the one that fits.

Summary

A type checker can prove a translation exists. It cannot prove it fits. One is a property of the source; the other only exists after fonts resolve and text is shaped.
Text width is not a function of the string. With per-language font stacks, the same character has different metrics in ja and zh-TW, so there is no arithmetic shortcut around rendering.
A browser is usable as a measuring instrument, not only as a test runner. The check is getBBox() and some subtraction.
{ ...someSVGRect } is an empty object, because those fields are prototype accessors. The arithmetic then yields NaN, and every comparison against NaN is false, so the failure path becomes unreachable.
Verification that fails open is worse than none, since it converts an unknown into a false assurance. Two unrelated bugs here produced the same clean output.

Both clean sweeps looked exactly like the real one that came after the fixes. The only thing that ever distinguished them was making the check fail on purpose.