test: live interaction coverage gate for the chat surface

[sakibsadmanshajib]

Enumerates the chat app's controls from the rendered DOM on every run and requires each one to produce an observable effect against a running deployment. Nothing here carries a hand written list of controls, so a control added upstream or by a Hive patch is enumerated the first time it renders and has to earn a proof or fail the gate.

Reworked after an adversarial review returned BLOCK on thirteen findings. What changed in that rework is in the last section; the rest of this description already reflects it.

No coverage figure is claimed

The 144 of 165 headline is withdrawn, and the script that derived it is
deleted. It came from a run that predates this rework, whose proven count
includes destructive controls clicked with the write aborted and controls
proven by a request no server ever answered, neither of which is proof any
more. It was derived by a script with none of the guards the floor updater
has, from a ledger carrying five surface errors that also violates four of the
floors this pull request restores. A number no shipped code path can produce
is not a measurement.

The figure returns when a sweep runs under the rules in this pull request,
writes its own summary, and passes its own floor and error checks. That run
was deliberately not taken here: the demo box is single tenant and another
agent is on it.

The ledger in docs/proof/chat-interaction-coverage-2026-08-10/ is kept for
what it records, with the summary the run itself wrote.

What has been measured, live, by the shipped code path

Run from a worktree against https://chat-hive.scubed.co, signed in through
the shared live-auth helper, 2026-08-12T20:06:47Z. A slice, so it reports no
percentage, which is the partial-run rule working:

| surface        | proven | not fired | total |
| settings:About |      2 |         0 |     5 |

Unproven:
- settings:About::button::See what's new   no effect (mutations=0)
- settings:About::a::Hive                  no effect (mutations=26)
- settings:About::a::All rights reserved.  no effect (mutations=28)

Five controls enumerated against a floor of 5, so the denominator guard was
satisfied exactly. Zero surface errors. Two controls proven by navigation,
three found dead, and the gate went red on them.

Two of those three are the rework changing verdicts rather than wording:
mutations=26 and mutations=28 would both have been proven under the old
mutations > 8 fallback. Under the rules in this pull request they are
unproven, because a mutation count nobody ever measured against an idle page is
not evidence of anything.

A full sweep has been attempted twice and does not complete against the box
today. It fails after the setup project signs in successfully, with the
coverage project finding itself on /auth:

could not get back to the chat home. url=https://chat-hive.scubed.co/auth
visible=["Continue with Hive","Open the Hive agent workspace"]

That is the #782 class, a chat session dying under a token refresh, and it is
not caused by this branch. Stated rather than papered over: until it completes,
this gate is worth a real slice measurement plus a detector proven against nine
fixture controls, and it is not worth a whole-app number. No whole-app number
is claimed anywhere in the branch.

Destructive controls are checked, never fired, and never counted as proof

The first version clicked them with the write aborted at the network layer and then forced proven: true on the strength of the aborted request. That is worse than no coverage: an aborted request never reaches the server, so a button wired to an endpoint that is gone came back green, which is exactly the #846 defect class this gate exists to catch, and it was invisible.

Now a control whose label says it destroys data is not clicked at all. It is asserted present, visible, enabled (or disabled with a stated reason) and carrying an accessible name, and recorded as not-fired, a category that is neither proof nor failure and is reported on its own line. The network level abort survives only as a data safety net for a control whose label says nothing about what it does, and any control that trips it is recorded not-fired for the same reason: there is no server verdict to read.

The recorded run shows 0 not fired because the category did not exist when it was taken. The next live run will show a non-zero count and a correspondingly lower proven count. That is a more honest number, not a regression.

Floors no longer ratchet downward

COV_FLOORS=update rewrote surface-floors.json from the run in progress and skipped checking the floors in the same pass. A bar the measured run can move is not a bar, and this branch shipped the proof: three surfaces that had genuinely degraded had the smaller numbers written in as the new baseline, with no reason recorded anywhere.

surface	earlier run the same day	floor this PR shipped	floor now
settings:Interface	56	48	56
settings:General	17	16	17
settings:Audio	8	7	8

The sweep now only ever reads the floors. Changing one is scripts/update-chat-coverage-floors.mjs, run against a recorded ledger, in its own commit: it refuses a partial ledger, refuses a ledger with surface errors, and refuses to lower any floor without --allow-lower. A unit test holds the committed file at or above what a recorded live run enumerated, so this specific regression cannot come back quietly.

An unswept floor key is now a hard failure too. The guard used to iterate only what was swept, so a surface that stopped opening took its own floor with it and left the denominator smaller with nothing said. Settings and workspace discovery throw instead of returning an empty list for the same reason, and the throw is recorded and the sweep continues, so the run still produces a ledger.

Proof is tied harder to the click

• Each surface is sampled while idle first, and every request signature the page produces on its own is excluded from proof on that surface. Only socket.io was excluded before, by name, so any REST poll landing in a control's 1200ms settle window counted as that control's proof. The self-test fixture now includes a dead button on a page that polls underneath it, and it must come back unproven.
• The mutations > 8 fallback is deleted. Nothing ever measured what an idle page emits in the same window, so it was a number that could pass a control on background churn. The rendered-text signature is the measured test; the count survives only in the failure detail.
• The in-test retry now only retries failures that say nothing about the control (surface would not re-open, control gone, no verdict within budget). A clean verdict of "does nothing" is final on the first attempt, instead of quietly reinstating the retries the config sets to zero.

Still not done, and deliberately: proof is not tied to the specific request a given control should issue. That needs a per-control expectation, which is a hand written inventory, which is the thing this gate exists to avoid. The idle sample is the general form of the same guard.

Partial runs no longer report a total

COV_SURFACES slices a run so it can finish inside the session lifetime (#782). A sliced run now says so, prints identity counts for the swept surfaces only, and prints no percentage at all. Its floor check is scoped to the same slice, so a slice cannot fail on the floors it was never asked to sweep.

Credential hygiene

• Traces and videos are off across the whole config, and unconditionally off for the project that walks the OAuth hop. A trace records request headers, so it carries this account's session cookies, and a trace of the sign-in hop carries the callback's code and state. tools/lint-no-token-in-proof-captures.mjs skips binaries by design and cannot see any of it. COV_TRACE=1 turns them on for local debugging, with the artefact never to be attached anywhere.
• Every URL the suite writes to a message, a ledger or a log goes through redactUrl, which scrubs the credential parameters in the fragment as well as the query string. That covers the "could not get back to the chat home" error, which used to interpolate a raw page.url().
• live-auth.ts keeps the only implementation of the chat hop, which is where its only caller imports it from; the duplicate added to live-auth.mjs had no caller and is deleted. The mint itself stays in the .mjs child process so the service-role key never enters a Playwright worker.

Wired into CI

The gate previously ran nowhere: no workflow and no npm script referenced its config, and its specs were invisible to scripts/verify-spec-collection.mjs, so a rebase onto main would have turned the required Web console job red on two undeclared spec files.

• .github/workflows/chat-coverage.yml, job self-check: the prover break-proof, the inert-registry and exclusion validity check, and the exclusion-expiry check. Needs no deployment and no credentials, runs on every pull request. Green on this one: 3 passed.
• Same workflow, job live-sweep: the real sweep, on workflow_dispatch or a run-chat-coverage label, uploading the ledger as an artifact. Same gating shape as owui-nightly, for the same reason.
• All three spec files are registered in playwright-spec-manifest.json, and the chat-coverage config is listed in the collection guard with placeholder env so its collection is deterministic everywhere.

Exclusion expiry can actually fire now

It used to test.skip on a missing token and continue past any non-ok GitHub response, so both paths reported green and an exclusion could outlive its issue forever. Both now fail, and the CI job passes GITHUB_TOKEN.

The removed-surfaces test no longer passes against a broken app

A settings modal that failed to open produced an empty list and counted as "nothing forbidden is present". It now asserts that each surface enumerated something before concluding anything is absent, and a route that redirects to /auth is a dead session rather than a removed route.

Two controls the old predicate called proven are broken

Both were being reported as coverage until this run.

• Chat Settings > General: Admin Settings fires a request to a 404 and does nothing #846, Settings General, Admin Settings. Fires a request that answers 404. Was counted as network proof.
• Chat composer: Upload Files renders "File not found." and nothing else #847, composer, Upload Files. Renders File not found. and nothing else. Was counted as dom proof.

Both are defects in the surface, not gate artefacts.

Five surface errors, and the gate is red on all of them

#834 merged and deployed and was expected to unblock the first four. It did not.

surface	state
sidebar	enumerated zero controls
chat-item-menu	could not open, click timed out after 20s
chat-message-actions	could not open, same
workspace:knowledge	enumerated zero controls
user-menu	click pass aborted, the execution context was destroyed by a navigation partway through

These stay as surface errors that fail the gate rather than becoming exclusions. An exclusion is a decision that a surface will not be measured, and that is not the decision here: the gate should stay red until they sweep.

Verification

• npx tsc --noEmit: clean.
• npx vitest run: 509 passed, 44 files, including 18 new tests over the gate's pure logic.
• npm run e2e:verify-collection: OK, 39 Playwright test files across 3 configs.
• npm run e2e:chat-coverage:self-check: 3 passed, including the new polling case. Without a token the expiry check fails as designed.
• node tools/lint-no-token-in-proof-captures.mjs: ok, 57 files scanned.
• Rebased onto current main; the branch is mergeable again.

The live sweep was deliberately not re-run: the demo box is single tenant and another agent is walking it. The headline figure is therefore derived rather than re-measured, and says so in both the README and above.

Second review pass, and what it changed

A second independent review returned BLOCK. Everything below is in the branch.

Proof now needs a server verdict. requests.length > 0 proved a control
pointing at an endpoint that hangs, for ever, which is the same missing verdict
that made an aborted write look like proof. Proof is a 2xx or 3xx response
inside the settle window; a request nothing answered is reported as exactly
that. My earlier rejection of this finding was wrong and is withdrawn: the rule
is generic and needs no per-control inventory.

not-fired no longer launders a dead control. Two problems, both fixed. It
matched label, which falls back to up to 140 characters of ancestor text, so
one "Delete All Chats" heading made every button in that panel destructive:
two controls this repository's own ledger proves dead would have gone green and
three genuinely proven ones would have been thrown away. It now matches the
control's own accessible name. And it was filtered out of the failure list
entirely; it now fails the gate like any other unfireable control unless
inert-registry.json justifies the key, while still being reported in its own
column so it can never be read as coverage.

A disabled control is no longer proof. disabled-with-reason counted as
coverage, so #846 could have been "fixed" by disabling Admin Settings with a
title attribute and the ratio would have risen. Disabled is now not-fired, and
aria-describedby is no longer accepted as a reason because it holds an
element id.

Destructive stateful controls are checked rather than flipped. flip()
fills every text input and clicks Save, and only the click pass consulted the
classifier.

The artifact cannot carry the OAuth callback. The upload took a whole
directory, and Playwright's JSON report was written into it; that report
carries raw error text, and a sign-in that outran its ninety second wait quotes
the callback URL with a live code and state. Three changes: the framework
reports are written outside the uploaded directory, the upload names two files
rather than a directory, and signInToChat redacts the framework's own failure
text where the credential is still inside a string this repository owns.
Redaction now lives in one place, tests/e2e/support/redact.ts, matches
parameter names by pattern rather than an exact list that missed id_token and
provider_token, and covers the hash-router shape.

The floor pin test actually pins. It compared against the morning ledger
only, where search was 50 against a committed floor of 106, so a fifth of the
denominator could have been signed away with the test still green. It now takes
the highest across every recorded run, and a missing floor key is a failure
rather than a skip.

Deleting an entry point no longer turns red into green. sidebar,
chat-item-menu, chat-message-actions and workspace:knowledge carry a floor of

1. They do not sweep today, so removing the link that reaches one of them now
   fails on the unswept floor key instead of quietly shrinking the denominator.

The chat list carries no floor at all. It is one row per chat, so it
enumerated 50 controls one run and 106 the next with no product change. A floor
there reds the gate whenever someone grooms the account, and the fix everyone
reaches for is to lower it. It is listed under dataDriven, still swept, still
required to prove every control.

Delta enumeration subtracts as a multiset. This is the finding I rejected
on the first pass and was wrong about: the point was ordinal SHIFT, not plain
stripping. An overlay that injects or hides one identically-keyed element moved
every later ordinal, so a base control could survive subtraction and be clicked
in the wrong context.

The service-role key claim is corrected. It is ordinary job-level
environment, the config reads it, and the spawn forwards all of process.env.
Only its use is isolated, and both comments that claimed otherwise now say so.

New cover for every branch above: a control whose request nothing answers,
one whose request dies in transport, a destructive one and a disabled one in
the browser fixture; exclusion bookkeeping asserted against deliberately broken
input rather than only against the valid committed file; the floor guard's
data-driven exemption and its missing-key failure.

Stage 6 review streams

Full verdicts are in a comment on this pull request. In short: CodeRabbit CLI SKIPPED (403, organization membership), the Codex adversarial stream SKIPPED (usage limit, produced no findings), the plain adversarial pass RAN and found two blocking issues that are fixed here (a response proving a control that did not cause it, and three strings reaching the ledger and CI log unredacted), and the three agent streams could not be dispatched from the process running this stage. Two blocked streams are reported as blocked rather than as passes.

Known gap, not fixed here

Neither new job is in .github/branch-protection-main.json, so neither can
block a merge. That file is owner territory rather than something a builder
should quietly extend, and there is a trap worth recording before anyone does:
a job-level if: skip SATISFIES a required context in GitHub, so making
Chat coverage live sweep required would create a check that is permanently
and silently green, since it skips on every ordinary pull request. If either
becomes required it should be Chat coverage self-check, which has no
job-level if: and runs everywhere.

Buglog entry

To be appended to .wolf/buglog.jsonl on main in a separate buglog-only pull request once this merges:

{"id":"bug-2026-08-11-coverage-gate-forced-proof","date":"2026-08-11","area":"testing","error_message":"chat interaction coverage reported controls as proven that were never shown to work: destructive controls were clicked with their write aborted and then forced proven:true, background REST polling inside a control's settle window counted as that control's network proof, and COV_FLOORS=update rewrote the surface floors from the same run that was supposed to check them","root_cause":"three separate ways for the gate to manufacture a green: an aborted request has no server verdict but was treated as one, only socket.io was excluded from the request evidence channel while Open WebUI also polls REST, and setting a threshold and checking it happened in one pass so a degraded run ratified itself as the new baseline (Interface 56 to 48, General 17 to 16, Audio 8 to 7)","fix":"destructive controls are asserted present, enabled and named and never fired, in a not-fired category that is neither proof nor failure; each surface is sampled while idle and its own background request signatures are excluded from proof; floors are read-only during a sweep and changed by scripts/update-chat-coverage-floors.mjs against a recorded ledger, which refuses to lower one without --allow-lower, with a unit test holding the committed floors at or above a recorded live run","tags":["e2e","playwright","coverage","false-green","chat"]}

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[coderabbitai]

Warning

Review limit reached

@sakibsadmanshajib, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 116 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8b24aa0b-a8f7-4aca-89d6-72e2659e861e

📥 Commits

Reviewing files that changed from the base of the PR and between 5f5426d and 3a75aa8.

📒 Files selected for processing (8)

• .github/workflows/chat-coverage.yml
• apps/web-console/e2e/chat-coverage/break-proof.spec.ts
• apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts
• apps/web-console/e2e/chat-coverage/lib.ts
• apps/web-console/scripts/update-chat-coverage-floors.mjs
• apps/web-console/scripts/verify-spec-collection.mjs
• apps/web-console/tests/e2e/support/live-auth.ts
• docs/proof/chat-interaction-coverage-2026-08-10/README.md

📝 Walkthrough

Walkthrough

Adds a live, authenticated Playwright chat-coverage harness. It discovers application surfaces, proves control effects, validates exclusions and coverage floors, generates reports, and runs self-checks and live sweeps in CI.

Changes

Live chat coverage

Layer / File(s)	Summary
Authentication and test configuration apps/web-console/e2e/chat-coverage/*, apps/web-console/tests/e2e/support/*, .gitignore	Adds environment-gated Playwright projects, OAuth authentication, storage-state handling, credential redaction, and ignored test artifacts.
Surface discovery and relocation apps/web-console/e2e/chat-coverage/surfaces.ts	Defines static surfaces, discovers settings and workspace tabs, normalizes navigation state, and attributes controls without duplicate baseline controls.
Control enumeration and proof validation apps/web-console/e2e/chat-coverage/lib.ts, data.ts, *.json	Enumerates controls, records state, proves clicks and state changes, filters background requests, aggregates identities, and validates floors, exclusions, registry entries, and removed surfaces.
Live sweep and reporting apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts, docs/proof/..., results/*	Runs click, persistence, transient, destructive-control, and removed-surface checks, then writes coverage ledgers and documentation.
Break-proof self-checks apps/web-console/e2e/chat-coverage/break-proof.spec.ts, apps/web-console/tests/unit/chat-coverage-lib.test.ts	Tests successful, inert, failing, deferred, destructive, and disabled controls, plus parser, floor, exclusion, redaction, and aggregation behavior.
CI integration and floor maintenance .github/workflows/chat-coverage.yml, apps/web-console/scripts/*, apps/web-console/package.json, apps/web-console/playwright-spec-manifest.json	Adds self-check and gated live-sweep jobs, collection verification, coverage commands, artifact upload, and floor-update automation.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

• Audit: tests that execute in CI and cannot fail, sweeping Go, vitest, Playwright and the scripts self-checks #797 — Adds live coverage checks that fail on ineffective controls and verify state persistence.

Possibly related PRs

• sakibsadmanshajib/hive#811 — Extends its interaction-coverage gate with chat-specific authentication, proof, exclusions, floors, and reporting.
• sakibsadmanshajib/hive#779 — The removed-surface manifest checks controls and routes removed by this PR.
• sakibsadmanshajib/hive#843 — Shares Playwright collection verification changes for the web-console test projects.

Sequence Diagram(s)

sequenceDiagram
  participant CI
  participant Playwright
  participant Auth
  participant Chat
  participant CoverageReports
  CI->>Playwright: run self-check or gated live sweep
  Playwright->>Auth: create authenticated browser state
  Auth->>Chat: complete console and Hive sign-in flow
  Playwright->>Chat: discover surfaces and interact with controls
  Chat-->>Playwright: return UI, navigation, and network evidence
  Playwright->>CoverageReports: write coverage ledgers and summaries
  CI-->>CoverageReports: upload coverage artifacts

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 42.55% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the new live interaction coverage gate for the chat surface.

✨ Finishing Touches 💡 2

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

⚔️ Resolve merge conflicts 💡

• Resolve merge conflict in branch test/chat-interaction-coverage

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch test/chat-interaction-coverage

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sakibsadmanshajib]

Findings filed from this run, deduped against open issues:

• Settings > About: "See what's new" is a dead button, the changelog it opened was removed #830 Settings > About, "See what's new" is a dead button (the changelog it opened was removed by Open WebUI chat surface exposes infra model aliases and duplicate/unscoped product surfaces (models, playground, automations, calendar, notes) #772)
• Four chat settings flip in the UI and revert after a reload, including Allow User Location #831 Four chat settings flip in the UI and revert after a reload, including Allow User Location
• Workspace > Knowledge: Create Knowledge does nothing and explains nothing on an empty form #832 Workspace > Knowledge, Create Knowledge does nothing and explains nothing
• Chat sidebar expand toggle has no accessible name, leaving four chat surfaces untestable #833 The chat sidebar expand toggle has no accessible name, which is why four surfaces are outside this run's denominator

Not filed, expected and already tracked: the model picker still listing the embedding, speech-to-text and text-to-speech aliases (#792, fix unmerged in #814).

[coderabbitai]

Actionable comments posted: 8

🧹 Nitpick comments (3)

apps/web-console/tests/e2e/support/live-auth.ts (1)

85-129: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

signInToChat exists twice with identical logic. Both files implement the same chat sign-in recipe: the same console probe, the same continue with hive and approve selectors, the same 4000/8000/8000 waits, and the same new chat readiness check. Two copies drift when an Open WebUI button name or timing changes, and only one of the two callers would then break.

• apps/web-console/tests/e2e/support/live-auth.ts#L85-L129: keep this as the single implementation, since apps/web-console/e2e/chat-coverage/auth.setup.ts imports it.
• apps/web-console/tests/e2e/support/live-auth.mjs#L313-L350: remove the duplicated body and delegate to the shared implementation, or extract the shared steps into one module that both entry points import.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/web-console/tests/e2e/support/live-auth.ts` around lines 85 - 129, Keep
signInToChat in apps/web-console/tests/e2e/support/live-auth.ts lines 85-129 as
the single implementation, preserving its console probe, selectors, waits, and
New Chat readiness check. Update
apps/web-console/tests/e2e/support/live-auth.mjs lines 313-350 to remove the
duplicated sign-in body and delegate to or import the shared implementation so
both callers use the same logic.

apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts (1)

618-627: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

The removed-route check accepts any page that contains the text "404".

body.includes("404") matches a version string, a request id, or a chat title that contains those digits. A route that still renders would then pass as removed. Tighten the check, for example by asserting on the HTTP status returned by page.goto.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts` around lines 618 -
627, Tighten the removed-route validation in the REMOVED.forbiddenRoutes loop so
incidental “404” text cannot mark a still-rendered page as gone. Capture the
response returned by page.goto and use its HTTP status, together with the
existing not-found text or redirect checks, to determine whether the route is
removed.

apps/web-console/e2e/chat-coverage/lib.ts (1)

421-439: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

readState and stateOf already disagree on input[type=file].

stateOf at lines 134-139 returns null for a file input. readState returns input.value for it. The two readers describe the same concept, so a control read through one path and compared through the other can produce a false difference. Extract one shared in-page reader and call it from both places.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/web-console/e2e/chat-coverage/lib.ts` around lines 421 - 439, Unify the
duplicated control-state logic used by readState and stateOf by extracting a
shared in-page reader, ensuring input[type=file] returns null in both paths.
Update both functions to call the shared reader while preserving the existing
handling for all other control types.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts`:
- Around line 479-525: Guard the existing coverage.json read and parse in the
tracker initialization before merging results: wrap fs.readFileSync and
JSON.parse in try/catch, and use an empty tracker when either operation fails.
Preserve the current controls extraction for valid files and leave the
subsequent cleanup, assertions, and report generation unchanged.

In `@apps/web-console/e2e/chat-coverage/lib.ts`:
- Around line 94-178: Update ENUMERATE to remove existing data-cov-id attributes
from all elements in the scoped root before assigning new IDs to the current
visible elements. Preserve the existing covId generation and locate behavior
while ensuring each current surface has no stale duplicate identifiers.
- Around line 261-288: Update proveByClickInner to perform all request,
download, filechooser, context page-listener, and MutationObserver cleanup in a
finally block. In proveByClick, signal cancellation when the budget timer wins
so the still-running inner operation stops handling stale events and cannot
affect later controls, while preserving the existing timeout Result.

In `@apps/web-console/e2e/chat-coverage/removed-surfaces.json`:
- Around line 5-14: Add /workspace/skills to the forbiddenRoutes array alongside
the existing workspace routes, keeping issue `#772`, unless the $comment
explicitly documents why Skills has no route. Ensure the forbidden control and
direct-navigation route checks remain aligned.

In `@apps/web-console/e2e/chat-coverage/results/2026-08-08-live-run.json`:
- Around line 5-9: The recorded coverage results violate the assertions in
chat-coverage.spec.ts because errors and unproven controls remain. Before
merging, update the affected surfaces to eliminate the failures, or add each
accepted control with a justification to inert-registry.json; alternatively gate
the suite so it is not a required check, ensuring no required test remains
failing.
- Around line 1-4: Rename the live-run results file to match its generatedAt
date, 2026-08-10. Align the JSON schema with the output written by
chat-coverage.spec.ts—using excused, surfaceErrors, and results—or explicitly
mark the artifact as hand-curated in note; preferably preserve the raw
coverage.run.json so tooling can compare future runs.

In `@apps/web-console/e2e/chat-coverage/surfaces.ts`:
- Around line 94-102: Update ensureSidebar to verify the sidebar reaches the
requested state after any toggle click, rather than silently succeeding when the
expected named toggle is absent. Try the available open/close toggle names,
assert the resulting sidebar state, and throw a distinct error when neither
toggle is found; preserve the existing settle delay after successful
interaction.
- Around line 256-284: Update the signature helper in enumerateSurface so
baseline and surface comparisons ignore enumerate’s trailing duplicate suffixes
such as `#2` and `#3`, while still removing the surface prefix. Use normalized
signatures for both baseline construction and delta filtering so repeated
controls are not incorrectly counted as new.

---

Nitpick comments:
In `@apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts`:
- Around line 618-627: Tighten the removed-route validation in the
REMOVED.forbiddenRoutes loop so incidental “404” text cannot mark a
still-rendered page as gone. Capture the response returned by page.goto and use
its HTTP status, together with the existing not-found text or redirect checks,
to determine whether the route is removed.

In `@apps/web-console/e2e/chat-coverage/lib.ts`:
- Around line 421-439: Unify the duplicated control-state logic used by
readState and stateOf by extracting a shared in-page reader, ensuring
input[type=file] returns null in both paths. Update both functions to call the
shared reader while preserving the existing handling for all other control
types.

In `@apps/web-console/tests/e2e/support/live-auth.ts`:
- Around line 85-129: Keep signInToChat in
apps/web-console/tests/e2e/support/live-auth.ts lines 85-129 as the single
implementation, preserving its console probe, selectors, waits, and New Chat
readiness check. Update apps/web-console/tests/e2e/support/live-auth.mjs lines
313-350 to remove the duplicated sign-in body and delegate to or import the
shared implementation so both callers use the same logic.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f088e664-7848-4426-a442-b4adfa9c992e

📥 Commits

Reviewing files that changed from the base of the PR and between f7d9293 and dbcb9f9.

📒 Files selected for processing (11)

• .gitignore
• apps/web-console/e2e/chat-coverage/auth.setup.ts
• apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts
• apps/web-console/e2e/chat-coverage/inert-registry.json
• apps/web-console/e2e/chat-coverage/lib.ts
• apps/web-console/e2e/chat-coverage/playwright.chat-coverage.config.ts
• apps/web-console/e2e/chat-coverage/removed-surfaces.json
• apps/web-console/e2e/chat-coverage/results/2026-08-08-live-run.json
• apps/web-console/e2e/chat-coverage/surfaces.ts
• apps/web-console/tests/e2e/support/live-auth.mjs
• apps/web-console/tests/e2e/support/live-auth.ts

[sakibsadmanshajib]

Body updated and read back from the API to confirm it landed: the 196 of 222 figure, the per-surface table, how the denominator is derived, the four excluded surfaces and the fixture result are all present.

Three of the four excluded surfaces (sidebar, chat-item-menu, chat-message-actions) become sweepable once #834 lands, which gives the Open WebUI sidebar toggle an accessible name. composer-controls has a separate cause and stays excluded.

The earlier body write went missing because gh pr edit fails on a projects-classic GraphQL error without writing anything. Filed as #842 with the gh api -X PATCH workaround and the read-back rule.

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts`:
- Around line 716-731: Update the test "no excluded surface waits on an issue
that already closed" to collect issue lookups whose GitHub responses are not
ok(), then fail the test with those failures instead of continuing to
expiredExclusions(EXCLUSIONS, closed). Preserve the existing missing-token skip
and closed-state collection, while ensuring 401, 403, 404, and rate-limit
responses cannot allow the expiry gate to pass silently.

In `@apps/web-console/e2e/chat-coverage/lib.ts`:
- Around line 615-623: Update the validation around the exclusion entry checks
near the issue/permanent rules so an issue is considered present only when it is
a positive safe integer. Reject or report zero, negative, non-integer, and
unsafe numeric issue identifiers before downstream GitHub validation, while
preserving the existing mutual-exclusion and non-permanent checks for valid
issue values.
- Around line 344-357: Update the response-listener flow around onResponse to
register a requestfailed listener that records meaningful transport failures
using Request.url(), method(), and failure(). Ensure this listener is removed
alongside the existing listeners, and move all listener cleanup into a finally
block covering settling and state inspection so handlers are always released
before returning proof.

In `@docs/proof/chat-interaction-coverage-2026-08-10/README.md`:
- Around line 31-42: Update the section heading to state five surfaces, add the
missing user-menu entry with its “click pass aborted” error, and reword the line
beginning with “#834” so it no longer starts with a hash while preserving the
deployment reference.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fb3fe458-57ff-4c4e-b61e-101c62d3508a

📥 Commits

Reviewing files that changed from the base of the PR and between dbcb9f9 and 22a7d3d.

📒 Files selected for processing (6)

• apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts
• apps/web-console/e2e/chat-coverage/lib.ts
• apps/web-console/e2e/chat-coverage/surface-exclusions.json
• apps/web-console/e2e/chat-coverage/surface-floors.json
• docs/proof/chat-interaction-coverage-2026-08-10/README.md
• docs/proof/chat-interaction-coverage-2026-08-10/coverage.run.json

[sakibsadmanshajib]

Stage 6, adversarial review, pipeline mode

Streams selected by diff shape: this diff carries a live OAuth flow and a CI artifact upload, so the security stream is mandatory and it earns the Codex adversarial stream as well.

stream	verdict
CodeRabbit CLI (coderabbit review --agent --base main)	SKIPPED, not a clean pass. Review failed: You are not a member of the requested organization (FORBIDDEN, HTTP 403, vsCode.requestFullReview). The GitHub App bot's own check on this pull request is a separate producer and is explicitly not being read as this stream's result.
/codex:adversarial-review (codex exec, gpt-5.5, read-only sandbox)	SKIPPED, not a clean pass. You've hit your usage limit ... try again at Sep 10th. The run reached the model and stopped there, so it produced no findings at all.
Plain adversarial pass, ticket-first	RAN, two findings, both fixed in 047789a8 (below).
ecc:code-review, typescript-reviewer, security-reviewer	NOT RUN, capability gap. These are agent streams, and the process running this stage has no agent-dispatch tool: Read, Write, Edit and Bash only. They cannot be run from here and are not being reported as passed. This is the correct escalation rather than a workaround: they need to be dispatched by whoever owns stage 6 dispatch, against head 047789a8.

Two of the four available streams were blocked by their own infrastructure, so the coverage of this review is thinner than the pipeline calls for. That is stated plainly rather than being allowed to read as three green streams.

Plain adversarial pass, findings

1. A response proved a control that did not cause it. Blocking, fixed.
lib.ts, proveByClickInner. Network proof was rewritten in this branch to require a 2xx or 3xx response inside the settle window, which closes the hanging-endpoint hole. It did not close the neighbouring one: any qualifying response arriving in the window counted, including the answer to a request the page had already started before the click. On a surface that is still finishing its own loads, that is precisely the coincidence the idle chatter sample exists to remove, and the sample cannot see it because the request predates the sample's window too. The request objects observed during a control's own window are now held, and a response has to belong to one of them.

2. Three strings reached the ledger and the CI log unredacted. Blocking, fixed.
The artifact-leak work in this branch redacted every URL the suite writes through redactUrl and moved Playwright's own reports out of the uploaded directory, but three paths still carried raw text: the intercepted-write list (blockedWrites, built from req.url()), the surfaceErrors list, and the failing assertion's own message. The third is the worst, because a failing assertion's message lands in the CI log, the JSON report and the HTML report at once, and its contents are mostly Playwright error text, which enumerates the URLs a failed navigation walked through. On a suite whose sign-in walks an OAuth callback that is the same class of leak the artifact change was written to prevent, one layer further in. All three now go through redactText.

Both were caught by asking the two questions the earlier reviews established for this file: where can a credential reach something durable, and where can a control be recorded as proven by something it did not cause.

[coderabbitai]

Actionable comments posted: 9

🧹 Nitpick comments (3)

apps/web-console/e2e/chat-coverage/lib.ts (1)

105-111: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

The doc comment above NOT_MOUNTED describes a different rule.

The comment explains asset and socket.io request filtering, which isMeaningfulRequest implements at lines 122-127. NOT_MOUNTED is a set of HTTP status codes that mark an endpoint as absent. A reader who edits the set based on this comment changes the wrong rule.

♻️ Proposed fix

-/**
- * Assets and background chatter never count as evidence that a control did
- * something. socket.io is the important exclusion: Open WebUI long-polls it
- * every few seconds, so counting it would hand a free pass to every control
- * whose click happened to land near a poll.
- */
+/**
+ * Statuses that say the endpoint a control called is not there, or failed
+ * outright. A click that produces one of these is not proof: it is a defect in
+ * the surface, reported ahead of every positive channel.
+ */
 const NOT_MOUNTED = new Set([404, 405, 410, 501]);

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/web-console/e2e/chat-coverage/lib.ts` around lines 105 - 111, Update the
doc comment above NOT_MOUNTED to describe its actual purpose: the HTTP status
codes that indicate an endpoint is absent or not mounted. Remove the unrelated
asset and socket.io request-filtering explanation, which belongs with
isMeaningfulRequest.

apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts (1)

605-611: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Build a new tracker record instead of deleting keys in place.

Lines 605-609 delete keys from tracker, and line 611 assigns into the same object. The repository guideline requires new objects instead of mutation, and append-only ledgers. coverage.json is a cross-slice ledger. A rebuilt record keeps the pruning rule explicit and keeps the read result immutable.

♻️ Proposed refactor

-    const sweptSurfaces = new Set(results.map((r) => r.surface));
-    const seenKeys = new Set(results.map((r) => r.key));
-    for (const key of Object.keys(tracker)) {
-      if (sweptSurfaces.has(tracker[key].surface) && !seenKeys.has(key)) {
-        delete tracker[key];
-      }
-    }
+    const sweptSurfaces = new Set(results.map((r) => r.surface));
+    const seenKeys = new Set(results.map((r) => r.key));
+    // A surface swept in this run is fully described by this run, so its stale
+    // keys are dropped by rebuilding rather than by mutating what was read.
+    const kept: Record<string, TrackedResult> = Object.fromEntries(
+      Object.entries(tracker).filter(
+        ([key, entry]) => !(sweptSurfaces.has(entry.surface) && !seenKeys.has(key)),
+      ),
+    );
     if (!partial) {
-      for (const r of results) tracker[r.key] = { ...r, lastSeen: generatedAt };
+      const merged: Record<string, TrackedResult> = {
+        ...kept,
+        ...Object.fromEntries(results.map((r) => [r.key, { ...r, lastSeen: generatedAt }])),
+      };
       fs.writeFileSync(
         trackerPath,
         JSON.stringify(
           {
             generatedAt,
             target: report.target,
-            summary: summarise(Object.values(tracker)),
-            controls: tracker,
+            summary: summarise(Object.values(merged)),
+            controls: merged,
           },
           null,
           2,
         ),
       );
     }

As per coding guidelines: "Create new objects instead of mutating existing ones; keep ledgers append-only."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts` around lines 605 -
611, Refactor the tracker update flow around the existing tracker-pruning loop
to avoid deleting keys or assigning records in place. Build a new tracker object
that retains unseen entries not matching swept surfaces, then merge new results
into that object when partial is false, and use the rebuilt record for
subsequent reads while preserving the existing pruning rule and append-only
ledger behavior.

Source: Coding guidelines

apps/web-console/e2e/chat-coverage/break-proof.spec.ts (1)

197-214: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Avoid in-place accumulator mutation in these TypeScript tests.

• apps/web-console/e2e/chat-coverage/break-proof.spec.ts#L197-L214: replace closed.add(...) and unreadable.push(...) with immutable accumulator updates.
• apps/web-console/tests/unit/chat-coverage-lib.test.ts#L181-L200: replace seen.set(...) with a newly constructed Map.
• apps/web-console/tests/unit/chat-coverage-lib.test.ts#L207-L220: replace below.push(...) with a newly constructed array.

As per coding guidelines, “Create new objects instead of mutating existing ones; keep ledgers append-only.”

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/web-console/e2e/chat-coverage/break-proof.spec.ts` around lines 197 -
214, Replace in-place accumulator mutations in the indicated sites: in
apps/web-console/e2e/chat-coverage/break-proof.spec.ts lines 197-214, update
closed and unreadable immutably; in
apps/web-console/tests/unit/chat-coverage-lib.test.ts lines 181-200, replace
seen.set with a newly constructed Map; and in lines 207-220, replace below.push
with a newly constructed array. Preserve the existing accumulation behavior and
append-only ledger semantics.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/chat-coverage.yml:
- Around line 17-19: Restrict the workflow-level token permissions in
.github/workflows/chat-coverage.yml to contents: read and issues: read,
preserving the self-check’s issue access. Update both checkout steps at the
referenced locations to set persist-credentials: false.
- Around line 62-69: The live-sweep job is currently skipped for ordinary pull
requests, so live coverage is not an enforced merge gate. Update the workflow
around live-sweep to run an always-present approved live sweep status for every
pull request, and configure the corresponding check as required branch
protection for the main branch so merges are blocked when it fails or is
missing.

In `@apps/web-console/e2e/chat-coverage/break-proof.spec.ts`:
- Around line 153-159: Update the SELF_TEST_HTML fixture to expose a click
counter or data-* marker for the “Delete Everything” control, then use that
marker around the checkWithoutFiring call and assert it remains unchanged
afterward. Keep the existing destructive.proven, destructive.proof, and
isDeferred assertions intact.

In `@apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts`:
- Around line 551-561: Update the excused filtering logic around results and
registryHits to require exact control-key identity instead of substring
matching: replace r.key.includes(key) with an exact comparison, or compare
identityOf(r) to identityOf the registry entry if that is the established key
model. Add identityOf to the existing ./lib imports only if using that approach,
while preserving the existing justification-length requirement and failure
filtering.

In `@apps/web-console/e2e/chat-coverage/lib.ts`:
- Around line 481-494: Add the same request-identity guard used by onResponse to
onRequestFailed: after the existing meaningful-request check, return when
mine.has(r) is false, so only failures belonging to the clicked request are
added to transportFailed.

In `@apps/web-console/e2e/chat-coverage/playwright.chat-coverage.config.ts`:
- Around line 77-82: Update the “chat-coverage-setup” project configuration to
set its runnable condition using the existing runnable symbol, so auth.setup.ts
is scheduled only when explicitly enabled. Preserve the project’s OAuth trace,
video, and screenshot settings and keep the credential-free break-proof project
runnable independently.

In `@apps/web-console/scripts/update-chat-coverage-floors.mjs`:
- Around line 38-45: Update parseArgs to validate every flag against
--allow-lower and --dry-run, and reject any unknown flag before returning. Also
reject more than one positional path, while preserving the existing default
ledger and recognized-flag behavior so main cannot read or write files with
invalid arguments.

In `@apps/web-console/tests/e2e/support/live-auth.ts`:
- Around line 163-165: Update the page.waitForURL predicate in the live
authentication flow to require both the Chat URL origin and a pathname other
than /auth. Keep the existing timeout and subsequent “New Chat” wait unchanged.

In `@docs/proof/chat-interaction-coverage-2026-08-10/README.md`:
- Around line 124-128: Update the shell command code fence in the README near
the e2e:chat-coverage:self-check example to declare the sh language after the
opening fence, preserving the commands and output unchanged.

---

Nitpick comments:
In `@apps/web-console/e2e/chat-coverage/break-proof.spec.ts`:
- Around line 197-214: Replace in-place accumulator mutations in the indicated
sites: in apps/web-console/e2e/chat-coverage/break-proof.spec.ts lines 197-214,
update closed and unreadable immutably; in
apps/web-console/tests/unit/chat-coverage-lib.test.ts lines 181-200, replace
seen.set with a newly constructed Map; and in lines 207-220, replace below.push
with a newly constructed array. Preserve the existing accumulation behavior and
append-only ledger semantics.

In `@apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts`:
- Around line 605-611: Refactor the tracker update flow around the existing
tracker-pruning loop to avoid deleting keys or assigning records in place. Build
a new tracker object that retains unseen entries not matching swept surfaces,
then merge new results into that object when partial is false, and use the
rebuilt record for subsequent reads while preserving the existing pruning rule
and append-only ledger behavior.

In `@apps/web-console/e2e/chat-coverage/lib.ts`:
- Around line 105-111: Update the doc comment above NOT_MOUNTED to describe its
actual purpose: the HTTP status codes that indicate an endpoint is absent or not
mounted. Remove the unrelated asset and socket.io request-filtering explanation,
which belongs with isMeaningfulRequest.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: bc64bafe-56f2-42d6-a000-43e491a24823

📥 Commits

Reviewing files that changed from the base of the PR and between 22a7d3d and 5f5426d.

📒 Files selected for processing (20)

• .github/workflows/chat-coverage.yml
• .gitignore
• apps/web-console/e2e/chat-coverage/auth.setup.ts
• apps/web-console/e2e/chat-coverage/break-proof.spec.ts
• apps/web-console/e2e/chat-coverage/chat-coverage.spec.ts
• apps/web-console/e2e/chat-coverage/data.ts
• apps/web-console/e2e/chat-coverage/lib.ts
• apps/web-console/e2e/chat-coverage/playwright.chat-coverage.config.ts
• apps/web-console/e2e/chat-coverage/removed-surfaces.json
• apps/web-console/e2e/chat-coverage/results/2026-08-10-morning-live-run.json
• apps/web-console/e2e/chat-coverage/surface-floors.json
• apps/web-console/e2e/chat-coverage/surfaces.ts
• apps/web-console/package.json
• apps/web-console/playwright-spec-manifest.json
• apps/web-console/scripts/update-chat-coverage-floors.mjs
• apps/web-console/scripts/verify-spec-collection.mjs
• apps/web-console/tests/e2e/support/live-auth.ts
• apps/web-console/tests/e2e/support/redact.ts
• apps/web-console/tests/unit/chat-coverage-lib.test.ts
• docs/proof/chat-interaction-coverage-2026-08-10/README.md

🚧 Files skipped from review as they are similar to previous changes (4)

• apps/web-console/e2e/chat-coverage/removed-surfaces.json
• .gitignore
• apps/web-console/e2e/chat-coverage/auth.setup.ts
• apps/web-console/e2e/chat-coverage/surfaces.ts

[sakibsadmanshajib]

Stage 6 adversarial review, pipeline mode, dynamic selector. Diff shape: a live OAuth flow plus a CI artifact upload, so the security stream is mandatory and the Codex adversarial stream is earned.

stream	verdict
CodeRabbit CLI	SKIPPED, see inline comment
Codex adversarial	SKIPPED, see inline comment
Plain adversarial, ticket first	RAN, two blocking findings, both fixed, see inline comments
ecc:code-review, typescript-reviewer, security-reviewer	NOT RUN, capability gap, see inline comment

Two of the four available streams were blocked by their own infrastructure and one group could not be dispatched from here. That is stated as blocked rather than allowed to read as green.

[sakibsadmanshajib]

Live measurement against the deployed surface, and proof the gate can go red

Run from a worktree at head 3a75aa83 against https://chat-hive.scubed.co, signed in through the shared live-auth helper (mint plus the real OIDC hop, no password read or written). Credentials came from the shared .env and none was printed. This is the shipped code path, not a mock.

Measured, settings:About slice

Partial run: surfaces matching ^settings:About$ only.
Proven 2 of the 5 distinct control identities on those surfaces.
No percentage is reported for a slice, because its denominator is not the app.

| surface        | proven | not fired | total |
| settings:About |      2 |         0 |     5 |

Unproven
- settings:About::button::See what's new   no effect (mutations=0)
- settings:About::a::Hive                  no effect (mutations=30)
- settings:About::a::All rights reserved.  no effect (mutations=28)

Five controls enumerated against a committed floor of 5, so the denominator guard was satisfied exactly, with zero surface errors. Two controls proven by navigation. Three found dead, and the run failed, which is the gate working rather than the gate being unhappy.

Two of those three are the rework changing verdicts rather than wording: mutations=30 and mutations=28 would both have been counted as proven under the old mutations > 8 fallback. Under the rules in this pull request they are unproven, because a mutation count nobody ever measured against an idle page is not evidence.

Can it go red on a control that works? Yes, demonstrated

COV_SABOTAGE neuters one named control in the page, in the capture phase, so a control that genuinely works stops working and nothing else changes. Same slice, same head, sabotaging Twemoji, which the baseline above proved by navigation:

[control] UNPROVEN :: settings:About::a::Twemoji
[coverage] settings:About: 1/5 proven
[stage] sabotage "Twemoji" intercepted 2 events on the last page
1 failed

Proven count moved 2 to 1 and the suite failed on the control that was deliberately broken. The assertion tracks the deployment rather than always failing on the same three.

One honest wrinkle from the same pair of runs, reported rather than smoothed over: CC-BY 4.0 and All rights reserved. swapped verdicts between the two runs (one proven in the baseline and unproven under sabotage, the other the reverse). Both are external links in one paragraph whose whole effect is a new tab, so a popup that arrives outside its own settle window is attributed to neither control cleanly. That is a real limitation of popup-based proof on that surface, it affects two controls on one surface, and it is not what either of the two runs above turns on.

The full sweep does not complete against the box today

Attempted three times. It fails after the setup project signs in, with the coverage project finding itself on /auth:

could not get back to the chat home. url=https://chat-hive.scubed.co/auth
visible=["Continue with Hive","Open the Hive agent workspace"]

Part of that is now fixed in this branch: waitForURL matched on origin alone, and the sign-in page is on the chat origin too, so a sign-in that had not completed satisfied it (CodeRabbit found this; fixed in 3a75aa83). The remainder is the #782 class, a chat session dying under a token refresh, which this branch does not cause and does not claim to fix.

So, plainly: Chat coverage live sweep cannot yet report a whole-app number in CI, and a skipped job must not be read as a passing one. What the gate is worth today:

• A real slice measurement against the live deployment, reproducible with COV_SURFACES, which found three dead controls on the first surface it looked at.
• A detector proven against nine fixture controls (chat-coverage-break-proof), running in CI on every pull request with no deployment and no secrets, covering: a working control, a dead one, one whose endpoint answers 500, one that only raises a toast, one that is dead while the page polls underneath it, one whose request nothing answers, one whose request dies in transport, a destructive one that must never be clicked, and a disabled one that must never count as proof.
• A denominator that cannot shrink quietly: floors are read-only during a sweep, an unswept floor key fails, and four surfaces that do not sweep today hold a floor of 1 so deleting their entry point fails rather than emptying the denominator.

No whole-app coverage figure is claimed anywhere in this branch, and the previous 144/165 headline is withdrawn precisely because no shipped code path can reproduce it.

Floors, and the direction they moved

surface	earlier commit on this branch	now	direction
settings:Interface	48	56	raised
settings:General	16	17	raised
settings:Audio	7	8	raised

Nothing was ratcheted downward to make a run pass. The 48, 16 and 7 were the defect: COV_FLOORS=update rewrote the floors from the run in progress and skipped checking them in the same pass, so a degraded run ratified itself. That path is deleted, the sweep only reads floors now, scripts/update-chat-coverage-floors.mjs refuses a partial ledger, refuses a ledger with surface errors and refuses to lower any floor without --allow-lower, and a unit test holds the committed file at or above the highest count any recorded run enumerated.

One floor was deliberately reduced and it is not a ratchet: search went from 106 to 1, a presence bar. Its control count is one row per chat, so it enumerated 50 on one run and 106 on the next with no product change; pinning that number reds the gate whenever the account is groomed, which is what teaches people to lower floors. The surface must still exist and render something, and dataDriven in the file records the reason.