Passpoint Docs QA — Evidence Report (ENGW-208 / ENGW-211)

Verdict at a glance

Pass

Fail

Blocked

Needs dev confirm

The docs v2.0 redesign is solid: branding, typography, self-hosted fonts, theme toggle, soft navigation, accessible names, SEO metadata and the admin sign-in gate all behave as described. Five things do not match the ticket: the CMS content path returns 404s on every page; the Transaction Simulator page has no simulator; the modal close rule is not followed (and the assistant overlay lacks dialog semantics); the docs footer has no contact-email link; and the Sandbox Playground sign-in rejects a valid sandbox key.

Continuation result. The two items left open by the 2026-10-12 filesystem stall are settled and are not defects: the reading-progress bar (Defect 5) and the sandbox focus mode (Defect 4) both work. On 2026-10-14 fresh credentials were supplied: the admin CMS now works (AC21 PASS), and the sandbox key is valid (the sandbox accepts it directly) but the playground still rejects it (AC20 FAIL — Defect 6).

Acceptance criteria

#	Criterion	Verdict	Evidence
1	All public routes load; unknown URL handled	PASS	96/96 routes HTTP 200 (`scripts/.route-status.txt`); unknown route returns a real 404
2	Landing: hero + live code sample, by-the-numbers, 4-step path, cards, coverage, FAQ, CTA, footer	PASS	All sections present; 5 code blocks in hero
3	Typography: Anton / Urbanist / IBM Plex Mono	PASS	Computed families confirmed
4	Self-hosted fonts	PASS	0 external font-service links
5	Dark mode keeps brand red; toggle works	PASS	`html.light → html.dark`; toggle `aria-label`
6	Docs chrome: badge, numbered sidebar, current-page highlight, prev/next	PASS	Badge present; active page painted `rgb(255,0,61)`; NEXT card present
7	Settings: per-page title + meta description	PASS	Confirmed on all sampled routes
8	Settings: contact email as a Contact link in docs footer	FAIL	No `mailto:`, no "Contact" text — footer is Playground / Dashboard / Website
9	Media: images have alt text, videos accessible titles	PASS	2 images, 0 missing alt; 0 videos present
10	Icon-only buttons have screen-reader names	PASS	23 buttons, 0 unnamed
11	Small red text contrast ≥ 4.5:1	PASS	0 reddish small-text nodes below threshold
12	Navigation: current page stays on screen, no half-loaded content	PASS	Soft nav confirmed; content retained, no full reload
13	Admin: sign-in screen, excluded from indexing	PASS	`/admin → /admin/login`; `robots: noindex, nofollow, nocache`; `robots.txt` Disallow
14	Docs assistant opens almost instantly	PASS	96 ms click → overlay
15	Every page loads CMS content without errors	FAIL	Defect 1
16	Transaction Simulator module	FAIL	Defect 2
17	Modals close only via Close or Cancel	FAIL	Defect 3
18	Sandbox Playground: focus mode, Escape to close	PASS	Re-tested 13 Oct — focus mode engages (2 full-screen overlays, body locked), Esc exits
19	Reading progress bar fills as you scroll	PASS	Re-tested 13 Oct — 0% → 93% → 100% on real wheel scroll
20	Sandbox signed-in flows (tabs, environment, send)	FAIL	Sign-in rejects a valid key: sandbox accepts it directly (200) but `POST /api/sandbox/session` → 401. Defect 6
21	Image/video upload, Cloudinary, nightly cleanup, admin CMS	PASS	Admin CMS reachable (role `editor`). Image upload end to end: signature → Cloudinary → confirm; `f_auto,c_limit,w_1600`; 10 MB limit enforced. Video control present. Not exercised: video upload, 100 MB limit, nightly cleanup

Split by ticket

Attribution follows the ticket descriptions. ENGW-208 names most behaviors verbatim (modals, progress bar, contact email, alt text, contrast, icon labels, settings); ENGW-211 is the migration plus the Transaction Simulation Module.

ENGW-208 pass

ENGW-208 fail

ENGW-211 pass

ENGW-211 fail

Ticket	Pass	Fail	Criteria
ENGW-208 — Docs v2.0 + admin CMS	15	3	1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 19, 21 pass · 8, 15, 17 fail
ENGW-211 — Next.js migration + Transaction Simulation Module	1	2	18 pass · 16, 20 fail
Totals	16	5	21 criteria

Caveats. Migration internals were deliberately out of scope per the ticket, so ENGW-211's column here is effectively just the simulator module. Criteria 18 and 20 are an inference — neither ticket description names the "Sandbox Playground"; they are assigned to ENGW-211 because the simulator module points at it. If the Playground predates this work, they belong to ENGW-208 or neither.

Defects

Defect 1 — Every page fires 404s against the CMS API OPEN · MEDIUM

Sampled routes request CMS resources that do not exist. Pages still render, so it is invisible to a reader, but it is 2–4 failed requests on every page load.

404 .../public/cms-pages/id/ui-labels          (every sampled page)
404 .../public/cms-pages/id/site-announcement  (nearly every page)
404 .../public/cms-pages/id/sandbox-playground
404 .../public/cms-pages/id/home
404 .../public/cms-pages/id/transaction-simulator

The ticket states "website builds now include the latest content changes made through the admin area." These 404s mean ui-labels and site-announcement are not served at all, so admin-set labels/announcements cannot be reaching the public site.

Defect 2 — Transaction Simulator page has no simulator OPEN · HIGH

/transaction-simulator renders a heading, one paragraph and a link to the Sandbox Playground. Nothing to interact with.

0 inputs / selects / textareas in <main>
5 buttons — all page furniture (HELPFUL / NOT HELPFUL, prev/next)
0 iframes, 0 canvases
No Live/Simulated switch, no product flows (Collection / Payout / Virtual Cards), no activity log

The copy promises "every click runs the Passpoint API calls, simulated in your browser or live on your sandbox," but no module is present.

Defect 3 — Modal close rule not followed; assistant lacks dialog semantics OPEN · LOW–MED

Modal	Backdrop click	Escape	Close button
Search (`⌘K`)	stays open ✔	closes ✘	—
Docs assistant	stays open ✔	stays open ✔	closes ✔

Stray-click protection works on both. But Escape closes search (contradicts "close only via their Close or Cancel button"), and the assistant overlay has no role="dialog", no aria-modal, no accessible name — a screen-reader user gets no announcement. Search does this correctly (role="dialog" aria-modal="true" aria-label="Search documentation"). Recommend confirming the intended rule, applying it consistently, and adding dialog semantics to the assistant either way (a focus-trapping dialog that ignores Escape is its own WCAG 2.1 keyboard-trap problem).

Defect 4 — Sandbox focus mode CLOSED · NOT A DEFECT

Yesterday this was left blocked because clicking Focus mode produced no measurable change. It does work — the previous check measured only header height and sidebar width, which by design do not change. Measured today on /sandbox-playground:

before     { fixedFullOverlays: 0, bodyOverflow: "hidden auto" }
focus mode { fixedFullOverlays: 2, bodyOverflow: "hidden" }   → toast "Focus mode is on. Press Esc or the exit button to return to the page."
Escape     { fixedFullOverlays: 0, bodyOverflow: "hidden auto" }

An "Exit focus mode" control appears and Shift+Alt+M is shown as the shortcut. Verdict: PASS.

Defect 5 — Reading progress bar reads `width:0%` CLOSED · NOT A DEFECT

The bar was reported as stuck at 0%. It tracks correctly — the docs layout scrolls an inner container (<main class="flex-1 min-w-0 min-h-0 overflow-y-auto">, scrollHeight 8544 / clientHeight 834), not the window. Yesterday's window.scrollTo scrolled the window (which has nothing to scroll), so the bar never moved. Real wheel scrolling drives it:

scrollTop 0     →  width:0%
scrollTop 7200  →  width:93.4%
scrollTop 7710  →  width:100%

Verdict: PASS. Worth a note to devs: because the page uses an inner scroll container, anything that scrolls only the window (anchor jumps, some test tooling) will not move the bar.

Defect 6 — Sandbox Playground sign-in rejects a valid sandbox key OPEN · HIGH

A key the sandbox accepts directly is rejected through the playground:

direct     POST https://dev.mypasspoint.com/userapp/merchant-app/get-auth-token
           {"merchantId":"0000e50b-…","apiKey":"497762…"}  →  HTTP 200 "Client token obtained successful"

playground POST https://main-passpoint-documentation-api.vercel.app/api/sandbox/session
           {"merchantId":"0000e50b-…","apiKey":"497762…","formMs":6596}  →  HTTP 401
           {"message":"The Passpoint sandbox did not accept this merchant ID and API key."}

Reproduced in a real headed browser with a 5-second dwell, so it is not a speed or automation artefact. The playground endpoint also refuses non-browser callers (403 "The sandbox playground only accepts requests from the Passpoint docs in a browser."), which a real browser passes before hitting the 401. Likely cause: the sandbox requires an identifiable caller IP, and the playground proxies through its own server, whose IP is not on this merchant's sandbox allow-list.

Playground sign-in rejects a valid key — Playground sign-in rejects a key the sandbox accepts directly — 401.

Evidence

Click any image to open the full-size screenshot in a new tab.

Progress bar filled mid-scroll — Defect 5 closed — progress bar at ~93% during real wheel scroll.

Sandbox focus mode active — Defect 4 closed — focus mode is on; Esc / exit returns to the page.

Landing page hero — AC2 — landing hero with live code sample.

Docs chrome — AC6 — "Docs v2.0" badge, numbered sidebar, red active page.

Dark mode — AC5 — dark mode keeps brand red.

Search modal — Defect 3 — the search modal has correct dialog semantics (the assistant overlay does not).

Admin sign-in — AC13 — admin sign-in gate, noindex.

Transaction simulator page — Defect 2 — simulator page with no simulator.

Docs assistant overlay — Defect 3 — assistant overlay with no dialog role.

404 page — AC1 — unknown route returns a real 404.

Image uploaded to Cloudinary — AC21 — image uploaded through the CMS (signature → Cloudinary → confirm).

Oversize image rejected — AC21 — an 11 MB file is rejected: "larger than the 10 MB limit".

Admin Settings — AC8/AC21 — Settings has a Contact email (support@passpoint.com) the footer is not showing.

Not tested

Admin CMS — remaining bits. Image upload, Cloudinary signed uploads and the 10 MB image limit are verified (AC21). Still not exercised: a real video upload, the 100 MB video limit, nightly media cleanup, and changing Settings (the editor role can view but not change).
Sandbox Playground authenticated flows — tabs, environment, history, send. Blocked by Defect 6: the playground sign-in rejects a valid sandbox key.
Legacy URL redirects — /docs, /documentation, /virtual-card return 404 with no redirect. The ticket says "redirected two legacy page URLs" without naming them, so this is not a fail until the two real URLs are named.
Image format negotiation / lazy loading — only 2 images on sampled pages.

Access attempts — credentials re-tried 2026-10-14

Fresh credentials were supplied on 2026-10-14 and tried in a real headed browser. The admin CMS now works; the sandbox key is valid but the playground rejects it (Defect 6).

Sandbox Playground — valid key, rejected by the playground

The key is accepted by the sandbox directly:

POST https://dev.mypasspoint.com/userapp/merchant-app/get-auth-token  →  HTTP 200
{"responseCode":"00","responseMessage":"Client token obtained successful"}

The playground rejects the same key:

POST https://main-passpoint-documentation-api.vercel.app/api/sandbox/session  →  HTTP 401
{"message":"The Passpoint sandbox did not accept this merchant ID and API key.","error":"Unauthorized","statusCode":401}

Admin CMS — works

chibuzor@mypasspoint.com / password → HTTP 200, role editor; dashboard loads (93 pages, 14 feedback). Image upload verified end to end. See AC21.

Admin CMS image upload — Admin CMS reachable; image uploaded through Cloudinary.