Passpoint Docs QA — Evidence Report (ENGW-208 / ENGW-211)

Verdict at a glance

Pass

Fail

Blocked

Needs dev confirm

The docs v2.0 redesign is solid: branding, typography, self-hosted fonts, theme toggle, soft navigation, accessible names, SEO metadata and the admin sign-in gate all behave as described. Four things do not match the ticket: the CMS content path returns 404s on every page; the Transaction Simulator page has no simulator; the modal close rule is not followed (and the assistant overlay lacks dialog semantics); and the docs footer has no contact-email link.

Continuation result. Two items left open by yesterday's filesystem stall are now settled and are not defects: the reading-progress bar (Defect 5) and the sandbox focus mode (Defect 4) both work — the earlier readings were measurement artifacts (programmatic window scroll, and only header/sidebar geometry measured). Both verdicts move to PASS.

Acceptance criteria

#	Criterion	Verdict	Evidence
1	All public routes load; unknown URL handled	PASS	96/96 routes HTTP 200 (`scripts/.route-status.txt`); unknown route returns a real 404
2	Landing: hero + live code sample, by-the-numbers, 4-step path, cards, coverage, FAQ, CTA, footer	PASS	All sections present; 5 code blocks in hero
3	Typography: Anton / Urbanist / IBM Plex Mono	PASS	Computed families confirmed
4	Self-hosted fonts	PASS	0 external font-service links
5	Dark mode keeps brand red; toggle works	PASS	`html.light → html.dark`; toggle `aria-label`
6	Docs chrome: badge, numbered sidebar, current-page highlight, prev/next	PASS	Badge present; active page painted `rgb(255,0,61)`; NEXT card present
7	Settings: per-page title + meta description	PASS	Confirmed on all sampled routes
8	Settings: contact email as a Contact link in docs footer	FAIL	No `mailto:`, no "Contact" text — footer is Playground / Dashboard / Website
9	Media: images have alt text, videos accessible titles	PASS	2 images, 0 missing alt; 0 videos present
10	Icon-only buttons have screen-reader names	PASS	23 buttons, 0 unnamed
11	Small red text contrast ≥ 4.5:1	PASS	0 reddish small-text nodes below threshold
12	Navigation: current page stays on screen, no half-loaded content	PASS	Soft nav confirmed; content retained, no full reload
13	Admin: sign-in screen, excluded from indexing	PASS	`/admin → /admin/login`; `robots: noindex, nofollow, nocache`; `robots.txt` Disallow
14	Docs assistant opens almost instantly	PASS	96 ms click → overlay
15	Every page loads CMS content without errors	FAIL	Defect 1
16	Transaction Simulator module	FAIL	Defect 2
17	Modals close only via Close or Cancel	FAIL	Defect 3
18	Sandbox Playground: focus mode, Escape to close	PASS	Re-tested 13 Oct — focus mode engages (2 full-screen overlays, body locked), Esc exits
19	Reading progress bar fills as you scroll	PASS	Re-tested 13 Oct — 0% → 93% → 100% on real wheel scroll
20	Sandbox signed-in flows (tabs, environment, send)	BLOCKED	No sandbox Merchant ID / API Key supplied
21	Image/video upload, Cloudinary, nightly cleanup, admin CMS	BLOCKED	Admin credentials not supplied

Split by ticket

Attribution follows the ticket descriptions. ENGW-208 names most behaviors verbatim (modals, progress bar, contact email, alt text, contrast, icon labels, settings); ENGW-211 is the migration plus the Transaction Simulation Module.

ENGW-208 pass

ENGW-208 fail

ENGW-208 blocked

ENGW-211 pass

ENGW-211 fail

ENGW-211 blocked

Ticket	Pass	Fail	Blocked	Criteria
ENGW-208 — Docs v2.0 + admin CMS	14	3	1	1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 19 pass · 8, 15, 17 fail · 21 blocked
ENGW-211 — Next.js migration + Transaction Simulation Module	1	1	1	18 pass · 16 fail · 20 blocked
Totals	15	4	2	21 criteria

Caveats. Migration internals were deliberately out of scope per the ticket, so ENGW-211's column here is effectively just the simulator module. Criteria 18 and 20 are an inference — neither ticket description names the "Sandbox Playground"; they are assigned to ENGW-211 because the simulator module points at it. If the Playground predates this work, they belong to ENGW-208 or neither.

Defects

Defect 1 — Every page fires 404s against the CMS API OPEN · MEDIUM

Sampled routes request CMS resources that do not exist. Pages still render, so it is invisible to a reader, but it is 2–4 failed requests on every page load.

404 .../public/cms-pages/id/ui-labels          (every sampled page)
404 .../public/cms-pages/id/site-announcement  (nearly every page)
404 .../public/cms-pages/id/sandbox-playground
404 .../public/cms-pages/id/home
404 .../public/cms-pages/id/transaction-simulator

The ticket states "website builds now include the latest content changes made through the admin area." These 404s mean ui-labels and site-announcement are not served at all, so admin-set labels/announcements cannot be reaching the public site.

Defect 2 — Transaction Simulator page has no simulator OPEN · HIGH

/transaction-simulator renders a heading, one paragraph and a link to the Sandbox Playground. Nothing to interact with.

0 inputs / selects / textareas in <main>
5 buttons — all page furniture (HELPFUL / NOT HELPFUL, prev/next)
0 iframes, 0 canvases
No Live/Simulated switch, no product flows (Collection / Payout / Virtual Cards), no activity log

The copy promises "every click runs the Passpoint API calls, simulated in your browser or live on your sandbox," but no module is present.

Defect 3 — Modal close rule not followed; assistant lacks dialog semantics OPEN · LOW–MED

Modal	Backdrop click	Escape	Close button
Search (`⌘K`)	stays open ✔	closes ✘	—
Docs assistant	stays open ✔	stays open ✔	closes ✔

Stray-click protection works on both. But Escape closes search (contradicts "close only via their Close or Cancel button"), and the assistant overlay has no role="dialog", no aria-modal, no accessible name — a screen-reader user gets no announcement. Search does this correctly (role="dialog" aria-modal="true" aria-label="Search documentation"). Recommend confirming the intended rule, applying it consistently, and adding dialog semantics to the assistant either way (a focus-trapping dialog that ignores Escape is its own WCAG 2.1 keyboard-trap problem).

Defect 4 — Sandbox focus mode CLOSED · NOT A DEFECT

Yesterday this was left blocked because clicking Focus mode produced no measurable change. It does work — the previous check measured only header height and sidebar width, which by design do not change. Measured today on /sandbox-playground:

before     { fixedFullOverlays: 0, bodyOverflow: "hidden auto" }
focus mode { fixedFullOverlays: 2, bodyOverflow: "hidden" }   → toast "Focus mode is on. Press Esc or the exit button to return to the page."
Escape     { fixedFullOverlays: 0, bodyOverflow: "hidden auto" }

An "Exit focus mode" control appears and Shift+Alt+M is shown as the shortcut. Verdict: PASS.

Defect 5 — Reading progress bar reads `width:0%` CLOSED · NOT A DEFECT

The bar was reported as stuck at 0%. It tracks correctly — the docs layout scrolls an inner container (<main class="flex-1 min-w-0 min-h-0 overflow-y-auto">, scrollHeight 8544 / clientHeight 834), not the window. Yesterday's window.scrollTo scrolled the window (which has nothing to scroll), so the bar never moved. Real wheel scrolling drives it:

scrollTop 0     →  width:0%
scrollTop 7200  →  width:93.4%
scrollTop 7710  →  width:100%

Verdict: PASS. Worth a note to devs: because the page uses an inner scroll container, anything that scrolls only the window (anchor jumps, some test tooling) will not move the bar.

Evidence

Click any image to open the full-size screenshot in a new tab.

Progress bar filled mid-scroll — Defect 5 closed — progress bar at ~93% during real wheel scroll.

Sandbox focus mode active — Defect 4 closed — focus mode is on; Esc / exit returns to the page.

Landing page hero — AC2 — landing hero with live code sample.

Docs chrome — AC6 — "Docs v2.0" badge, numbered sidebar, red active page.

Dark mode — AC5 — dark mode keeps brand red.

Search modal — Defect 3 — the search modal has correct dialog semantics (the assistant overlay does not).

Admin sign-in — AC13 — admin sign-in gate, noindex.

Transaction simulator page — Defect 2 — simulator page with no simulator.

Docs assistant overlay — Defect 3 — assistant overlay with no dialog role.

404 page — AC1 — unknown route returns a real 404.

Not tested

Admin CMS — image/video upload, Cloudinary signed uploads, drag-and-drop, size limits, nightly media cleanup, Settings permissions. Needs admin credentials.
Sandbox Playground authenticated flows — tabs, environment, history, send. Needs a sandbox Merchant ID + API Key.
Legacy URL redirects — /docs, /documentation, /virtual-card return 404 with no redirect. The ticket says "redirected two legacy page URLs" without naming them, so this is not a fail until the two real URLs are named.
Image format negotiation / lazy loading — only 2 images on sampled pages.