feat: add proof worker — AI-powered browser testing - #2
Merged
Conversation
proof is an iii worker that scans code changes and verifies them in a real Chromium browser using snapshot-driven accessibility testing. 25 registered functions: - 14 browser tools (navigate, snapshot, click, type, screenshot, console logs, network requests, performance metrics, raw Playwright exec, assertions, CDP discovery, cookie injection) - 11 pipeline functions (scan, coverage, execute, report, run, replay, flows, history, enqueue, cleanup) 8 HTTP endpoints for REST access. Uses iii primitives throughout: - All inter-function calls via iii.trigger() - State for reports and saved flows - Streams for real-time progress - Queue + DLQ for CI runs with auto-retry - Logger with OTel tracing Default mode: Claude Code or Codex as the agent (no API key). Automated mode: Anthropic API for headless CI (needs ANTHROPIC_API_KEY). 1,506 lines across 8 TypeScript files.
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 23 minutes and 7 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (7)
📝 WalkthroughWalkthroughA new "proof" package is introduced as an AI-powered browser testing worker that integrates with an iii engine. It enables Claude agents to drive Chromium via snapshot-based element referencing (using ARIA trees and Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Worker as iii Worker
participant Agent as Claude Agent
participant BrowserAPI as Browser/Playwright
participant State as State & Stream
User->>Worker: POST /proof (run request)
Worker->>Worker: proof::scan (analyze diffs)
Worker->>Worker: proof::coverage (test coverage)
Worker->>Worker: proof::execute
Worker->>BrowserAPI: launch browser session
Worker->>BrowserAPI: inject cookies (optional)
Worker->>Agent: initialize with system prompt + user prompt
loop Agent Loop (max 50 iterations)
Agent->>Agent: call Claude API with tools
alt Tool Use Response
Agent->>Worker: trigger (function_id, payload)
Worker->>BrowserAPI: execute browser action<br/>(navigate, click, type, etc.)
BrowserAPI->>BrowserAPI: perform action on page
BrowserAPI-->>BrowserAPI: build ARIA snapshot
BrowserAPI-->>Worker: return result + snapshot
Worker-->>Agent: return tool_result
else End Turn
Agent-->>Worker: stop_reason = end_turn
Worker->>Worker: exit loop
end
Worker->>State: push step progress (best-effort)
end
Worker->>BrowserAPI: close browser
Worker->>Worker: proof::report (store results)
Worker->>State: save flow (if pass)
Worker->>State: push stream update
Worker-->>User: return RunReport
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 8
🧹 Nitpick comments (6)
proof/README.md (2)
113-127: Add language identifier to fenced code block.The static analysis tool flagged this ASCII flow diagram as missing a language specification. Use
textorplaintextto satisfy linters while preserving the diagram formatting.📝 Suggested fix
-``` +```text proof::scan git diff → changed files, commits🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@proof/README.md` around lines 113 - 127, The fenced ASCII diagram block containing the flow lines starting with "proof::scan ... proof::report" is missing a language identifier; update that code fence to include a plaintext specifier (e.g., ```text or ```plaintext) so linters accept the block while preserving the diagram formatting.
230-249: Add language identifier to architecture diagram code block.Same as above — add
textorplaintextlanguage identifier.📝 Suggested fix
-``` +```text ┌──────────────────────────────────────────┐🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@proof/README.md` around lines 230 - 249, The README's ASCII architecture diagram code block (the triple-backtick block containing the "iii Engine" diagram) lacks a language identifier; update the opening fence from ``` to ```text (or ```plaintext) so the diagram is rendered as plain text—locate the block that starts with the ASCII art including "iii Engine" and change its opening backticks accordingly.proof/src/context.ts (2)
169-180:resolveImportPathreturns the unresolved path as fallback.When no file matches, the function returns the normalized path anyway (line 179). This could lead to false coverage associations if the test imports a file that doesn't exist or is generated at build time.
♻️ Consider returning null for unresolved imports
for (const ext of ["", ".ts", ".tsx", ".js", ".jsx", "/index.ts", "/index.js"]) { const full = path.join(root, resolved + ext); try { if (fs.statSync(full).isFile()) return resolved + ext; } catch { /* not found */ } } - return resolved; + return null;The caller at line 104 already handles
nullreturns appropriately.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@proof/src/context.ts` around lines 169 - 180, resolveImportPath currently returns the normalized path as a fallback even when no matching file exists, which can create false associations; change the function to return null when no file was found (instead of returning resolved at the end), keeping the existing file-check loop and error handling in resolveImportPath so callers (e.g., the caller at line 104) receive null for unresolved imports and can handle them appropriately.
129-148: Async function uses synchronous filesystem operations.
findTestFilesis markedasyncbut usesfs.readdirSync. Similarly,extractImports(line 150) usesfs.readFileSync. While functionally correct, this blocks the event loop during file scanning. For large repositories, consider using async variants (fs.promises.readdir,fs.promises.readFile).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@proof/src/context.ts` around lines 129 - 148, The functions findTestFiles and extractImports use synchronous fs calls; change them to non-blocking async calls by replacing fs.readdirSync with await fs.promises.readdir(fullDir, { withFileTypes: true }) in findTestFiles (keep the recursive await findTestFiles logic) and replace fs.readFileSync in extractImports with await fs.promises.readFile(filePath, "utf8"); ensure you catch and handle errors (use catch(err) to propagate or return results as before) and preserve checks like { withFileTypes: true } and TEST_PATTERN/SKIP_DIRS behavior so the public APIs (findTestFiles, extractImports) remain async and non-blocking.proof/src/agent.ts (1)
38-43: Consider typing the messages array more strictly.Using
any[]for the messages array loses type safety. The Anthropic SDK provides message types that could be used here.♻️ Suggested improvement
+import type { MessageParam } from "@anthropic-ai/sdk/resources/messages"; + - const messages: any[] = [ + const messages: MessageParam[] = [ { role: "user", content: buildUserPrompt(diff, files, baseUrl, instruction, commits, coverage), }, ];🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@proof/src/agent.ts` around lines 38 - 43, The messages array is typed as any[] which loses type safety; change it to the Anthropic SDK message type (e.g., use the exported message interface/type from the SDK such as AnthropicMessage or ChatCompletionRequestMessage depending on your SDK version) and update the array declaration for messages to that type, ensuring the object created with buildUserPrompt conforms to that message type; add the appropriate import of the message type at the top of the file and adjust buildUserPrompt's return type if needed so the element matches the SDK message interface.proof/src/worker.ts (1)
175-181: Edge case: empty slug base when title has no alphanumeric characters.If
report.titlecontains only non-alphanumeric characters (e.g., "..."), thebasebecomes an empty string and the slug is just the timestamp. Consider adding a fallback.♻️ Suggested improvement
const base = report.title .toLowerCase() .replace(/[^a-z0-9]+/g, "-") .replace(/^-|-$/g, "") - .slice(0, 50); + .slice(0, 50) || "flow"; const slug = `${base}-${Date.now().toString(36)}`;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@proof/src/worker.ts` around lines 175 - 181, When building the slug in the pass-report branch (where report.status, report.steps, base, slug are used), handle the edge case where base becomes empty by providing a fallback token (e.g., "report" or "result" plus a short unique suffix) before composing slug; update the slug construction to use that fallback when base === "" so the final slug is not just the timestamp and remains readable/consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@proof/README.md`:
- Around line 167-202: The section headers miscount the functions: update
"Browser Tools (12)" to "Browser Tools (14)" (table lists
proof::browser::launch, close, navigate, snapshot, click, type, select, press,
screenshot, console_logs, network, performance, exec, assert) and update
"Pipeline (10)" to "Pipeline (12)" (table lists proof::scan, coverage, execute,
report, run, replay, flows, history, enqueue, cleanup, cookies::inject,
cdp::discover); also update any overall total function count references
elsewhere in the README to reflect the correct combined total (26) so all counts
stay consistent.
In `@proof/src/agent.ts`:
- Around line 45-113: The agent loop can silently exit when it reaches
MAX_ITERATIONS; after the for-loop that iterates up to MAX_ITERATIONS, detect if
iteration completed due to the limit (i.e., never broke out early) and then set
runStatus = "error" (or another sentinel) and log or record a warning (using
existing logging or pushStepProgress/runId) so the caller can surface the stuck
state; update any final pushStepProgress call to flush this status and/or append
a message to messages so the run is clearly marked as failed due to iteration
limit.
In `@proof/src/browser.ts`:
- Around line 237-250: handlePlaywrightExec currently runs the dynamically
constructed AsyncFunction (created via AsyncFunction and invoked as fn(page,
context, browser, ref)) with no timeout, so a hung/infinite script will block
the worker; wrap the invocation in a Promise.race between the fn(...) call and a
reject-after-timeout promise (use a configurable constant like EXEC_TIMEOUT_MS
or default to an appropriate value) so the race rejects when the timeout
elapses, and propagate that rejection; ensure the timeout is created and cleared
properly around the call to fn to avoid resource leaks and reference the
existing symbols handlePlaywrightExec, AsyncFunction, fn, and ref when locating
where to add the timeout logic.
In `@proof/src/context.ts`:
- Around line 36-43: When handling the "commit" case, guard against the
initial-commit edge where `${hash}^` has no parent; detect whether the parent
exists (e.g., call git.revparse([`${hash}^`]) or attempt a lightweight probe in
a try/catch) and if the parent does not exist, use a fallback pattern: compute
the diff against the empty tree or just diff the commit itself (use git.diff
with a single hash or diff against the empty tree), build files from that result
(still apply slice(0, MAX_FILES)), and set commits to the single commit from
git.log({ to: hash, maxCount: 1 }) or the log result you already have; update
uses of git.diff, git.diffSummary, and git.log in the "commit" branch to branch
on whether the parent exists and avoid referencing `${hash}^` when it does not.
In `@proof/src/cookies.ts`:
- Around line 129-134: The Firefox cookie extraction constructs an SQL query by
directly interpolating the domain into the execFileAsync call (the SELECT from
moz_cookies in the try block), which mirrors the injection risk in the Chrome
path; fix it by applying the same sanitation/validation used for Chrome: either
use a safe escape function on the domain value before interpolation or switch to
a parameterized query approach so the domain isn't injected raw into the SQL
string, and ensure the revised logic is applied to the execFileAsync call
constructing the moz_cookies SELECT statement.
- Around line 64-69: The current execFileAsync call builds a SQL string using
domain interpolation (see execFileAsync invocation and cookieDbPath) which risks
SQL injection; replace this by using a proper SQLite binding with parameterized
queries (e.g., better-sqlite3 or sqlite3) and bind the pattern `'%'+domain`
instead of interpolating, or strictly validate/whitelist the domain input before
usage; also add a short comment near cookieDbPath/execFileAsync noting that
Chrome cookie values are encrypted on macOS/Windows and may be unusable so
callers should expect empty values or document platform limitations.
In `@proof/src/worker.ts`:
- Around line 39-51: The run lock acquired by acquireRun(runId) in the
iii.registerFunction handler ("proof::browser::launch") isn't released if
launchBrowser throws; wrap the launchBrowser call in a try/catch and on error
call the corresponding release function (e.g., releaseRun(runId) or clear the
activeRunId) then rethrow the error so the lock is freed for subsequent runs; if
no releaseRun exists, add a function to clear activeRunId and call it in the
catch for the launchBrowser call.
- Around line 270-276: The current loop builds function IDs by stripping
"browser_" from action.tool before calling iii.trigger, which can produce
invalid IDs for non-browser actions; update the code in the flow processing loop
(the block iterating flow.actions and calling iii.trigger) to use the existing
mapping utility toolNameToFunctionId to derive the function_id from action.tool
(and handle undefined/missing mappings by logging an error or marking the result
as "fail" instead of calling iii.trigger), so iii.trigger receives a correct
function_id for every tool and results is updated appropriately.
---
Nitpick comments:
In `@proof/README.md`:
- Around line 113-127: The fenced ASCII diagram block containing the flow lines
starting with "proof::scan ... proof::report" is missing a language identifier;
update that code fence to include a plaintext specifier (e.g., ```text or
```plaintext) so linters accept the block while preserving the diagram
formatting.
- Around line 230-249: The README's ASCII architecture diagram code block (the
triple-backtick block containing the "iii Engine" diagram) lacks a language
identifier; update the opening fence from ``` to ```text (or ```plaintext) so
the diagram is rendered as plain text—locate the block that starts with the
ASCII art including "iii Engine" and change its opening backticks accordingly.
In `@proof/src/agent.ts`:
- Around line 38-43: The messages array is typed as any[] which loses type
safety; change it to the Anthropic SDK message type (e.g., use the exported
message interface/type from the SDK such as AnthropicMessage or
ChatCompletionRequestMessage depending on your SDK version) and update the array
declaration for messages to that type, ensuring the object created with
buildUserPrompt conforms to that message type; add the appropriate import of the
message type at the top of the file and adjust buildUserPrompt's return type if
needed so the element matches the SDK message interface.
In `@proof/src/context.ts`:
- Around line 169-180: resolveImportPath currently returns the normalized path
as a fallback even when no matching file exists, which can create false
associations; change the function to return null when no file was found (instead
of returning resolved at the end), keeping the existing file-check loop and
error handling in resolveImportPath so callers (e.g., the caller at line 104)
receive null for unresolved imports and can handle them appropriately.
- Around line 129-148: The functions findTestFiles and extractImports use
synchronous fs calls; change them to non-blocking async calls by replacing
fs.readdirSync with await fs.promises.readdir(fullDir, { withFileTypes: true })
in findTestFiles (keep the recursive await findTestFiles logic) and replace
fs.readFileSync in extractImports with await fs.promises.readFile(filePath,
"utf8"); ensure you catch and handle errors (use catch(err) to propagate or
return results as before) and preserve checks like { withFileTypes: true } and
TEST_PATTERN/SKIP_DIRS behavior so the public APIs (findTestFiles,
extractImports) remain async and non-blocking.
In `@proof/src/worker.ts`:
- Around line 175-181: When building the slug in the pass-report branch (where
report.status, report.steps, base, slug are used), handle the edge case where
base becomes empty by providing a fallback token (e.g., "report" or "result"
plus a short unique suffix) before composing slug; update the slug construction
to use that fallback when base === "" so the final slug is not just the
timestamp and remains readable/consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4a8a06b6-9aeb-4bb2-9112-c98e1023e2fa
📒 Files selected for processing (11)
proof/README.mdproof/package.jsonproof/src/agent.tsproof/src/browser.tsproof/src/context.tsproof/src/cookies.tsproof/src/prompt.tsproof/src/tools.tsproof/src/types.tsproof/src/worker.tsproof/tsconfig.json
Comment on lines
+167
to
+202
| ### Browser Tools (12) | ||
|
|
||
| | Function | Description | | ||
| |----------|-------------| | ||
| | `proof::browser::launch` | Launch Chromium (headed or headless, CDP optional) | | ||
| | `proof::browser::close` | Close browser session | | ||
| | `proof::browser::navigate` | Navigate to URL, return snapshot | | ||
| | `proof::browser::snapshot` | ARIA accessibility tree with `[ref=eN]` markers | | ||
| | `proof::browser::click` | Click element by ref | | ||
| | `proof::browser::type` | Type text into input by ref | | ||
| | `proof::browser::select` | Select dropdown option by ref | | ||
| | `proof::browser::press` | Press keyboard key on element | | ||
| | `proof::browser::screenshot` | Capture page as base64 PNG | | ||
| | `proof::browser::console_logs` | Read browser console messages | | ||
| | `proof::browser::network` | Read network request log | | ||
| | `proof::browser::performance` | Core Web Vitals (FCP, TTFB, CLS) | | ||
| | `proof::browser::exec` | Execute raw Playwright code | | ||
| | `proof::browser::assert` | Record a pass/fail assertion | | ||
|
|
||
| ### Pipeline (10) | ||
|
|
||
| | Function | Description | | ||
| |----------|-------------| | ||
| | `proof::scan` | Git diff scanning (4 target modes) | | ||
| | `proof::coverage` | Import graph analysis → test coverage | | ||
| | `proof::execute` | Agent loop with Claude API | | ||
| | `proof::report` | Results → iii State + Stream | | ||
| | `proof::run` | Full pipeline orchestration | | ||
| | `proof::replay` | Replay a saved flow without AI | | ||
| | `proof::flows` | List saved flows | | ||
| | `proof::history` | Run history with trends | | ||
| | `proof::enqueue` | Queue-based run with retries + DLQ | | ||
| | `proof::cleanup` | Close all browser sessions | | ||
| | `proof::cookies::inject` | Extract local browser cookies | | ||
| | `proof::cdp::discover` | Find running Chrome CDP endpoint | | ||
|
|
There was a problem hiding this comment.
Function count headers don't match table contents.
The section header says "Browser Tools (12)" but the table lists 14 functions. Similarly, "Pipeline (10)" but the table lists 12 functions. Consider updating the headers to match the actual counts.
📝 Suggested fix
-### Browser Tools (12)
+### Browser Tools (14)-### Pipeline (10)
+### Pipeline (12)Also update line 19 and 239-240 to reflect the actual total (26 functions if all are counted).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@proof/README.md` around lines 167 - 202, The section headers miscount the
functions: update "Browser Tools (12)" to "Browser Tools (14)" (table lists
proof::browser::launch, close, navigate, snapshot, click, type, select, press,
screenshot, console_logs, network, performance, exec, assert) and update
"Pipeline (10)" to "Pipeline (12)" (table lists proof::scan, coverage, execute,
report, run, replay, flows, history, enqueue, cleanup, cookies::inject,
cdp::discover); also update any overall total function count references
elsewhere in the README to reflect the correct combined total (26) so all counts
stay consistent.
The test was asserting "0.1.0" but Cargo.toml is at 0.1.2, causing CI failure on every PR.
Contributor
Author
|
Heads up — main is being bumped to Node SDK 0.11.x changes: typed Release notes: https://github.com/iii-hq/iii/releases/tag/iii/v0.11.3 |
…, count drift
- worker.ts: release run lock if launchBrowser throws; replace hand-rolled
action.tool.replace('browser_','') with toolNameToFunctionId so every
action (including browser_assert) routes to the correct function id.
Also migrate all 26 registerFunction({ id: '...' }, ...) sites to the
two-arg SDK signature required by iii-sdk 0.11.3.
- browser.ts: race handlePlaywrightExec against a 30s timeout so runaway
user code can't wedge the worker event loop.
- agent.ts: surface MAX_ITERATIONS without end_turn as runStatus='error'
and a console.warn instead of exiting silently.
- context.ts: fall back to the empty-tree hash when diffing the initial
commit (no parent) so 'target=commit' works on first-commit repos.
- cookies.ts: validate hostname against [A-Za-z0-9.-] allow-list before
interpolating into the sqlite3 LIKE clause. Document that Chrome/Firefox
cookies on macOS and Windows are keychain-encrypted and return empty.
- README: Browser Tools (12)→(14), Pipeline (10)→(12).
Merged
andersonleal
added a commit
that referenced
this pull request
Jul 24, 2026
rctest5 attempt 4 exposed the registration-vs-completion race: a rejected sibling (the MOT-4210 divergent-spec check, working as designed) forced a re-registration round, the watched writer sessions finished inside that window, and the re-armed edge-triggered turn-completed join predecessors starved forever — finalizer never spawned, report never written, orchestrator parked. A completion BARRIER is only correct level-triggered: registering a join predecessor on a turn-completed binding whose filtered session is ALREADY terminally completed (durable status completed, no armed wake) now delivers a catch-up fire shaped like the real completion event, stamped __late_subscription_replay. Joins only — their per-key accumulator makes a rare double-delivery idempotent. Fail-open: a status lookup failure keeps edge semantics. The registration response carries a note naming the replay. E2E-009 late-join-predecessor-replay gates it end to end: a worker session completes (proved past via a call-mode completion witness — recorder call #1), then a probe-steered turn registers the late join; the joined downstream's recorder call #2 exists only through the replay. Gate check: disabling the replay times the scenario out. Runner: ProbeAction gains after_target_calls (an untracked session's only observable milestone is a controlled-function call). Verified live: rctest5 attempt 5 completed fully green — report row self-verifying pass, totals == GROUP BY (5/5/5), 15/15 events, cited trigger-spawned reactor, zero leaked subscriptions, no polling.
andersonleal
added a commit
that referenced
this pull request
Jul 24, 2026
rctest5 attempt 4 exposed the registration-vs-completion race: a rejected sibling (the MOT-4210 divergent-spec check, working as designed) forced a re-registration round, the watched writer sessions finished inside that window, and the re-armed edge-triggered turn-completed join predecessors starved forever — finalizer never spawned, report never written, orchestrator parked. A completion BARRIER is only correct level-triggered: registering a join predecessor on a turn-completed binding whose filtered session is ALREADY terminally completed (durable status completed, no armed wake) now delivers a catch-up fire shaped like the real completion event, stamped __late_subscription_replay. Joins only — their per-key accumulator makes a rare double-delivery idempotent. Fail-open: a status lookup failure keeps edge semantics. The registration response carries a note naming the replay. E2E-009 late-join-predecessor-replay gates it end to end: a worker session completes (proved past via a call-mode completion witness — recorder call #1), then a probe-steered turn registers the late join; the joined downstream's recorder call #2 exists only through the replay. Gate check: disabling the replay times the scenario out. Runner: ProbeAction gains after_target_calls (an untracked session's only observable milestone is a controlled-function call). Verified live: rctest5 attempt 5 completed fully green — report row self-verifying pass, totals == GROUP BY (5/5/5), 15/15 events, cited trigger-spawned reactor, zero leaked subscriptions, no polling.
andersonleal
added a commit
that referenced
this pull request
Jul 24, 2026
rctest5 attempt 4 exposed the registration-vs-completion race: a rejected sibling (the MOT-4210 divergent-spec check, working as designed) forced a re-registration round, the watched writer sessions finished inside that window, and the re-armed edge-triggered turn-completed join predecessors starved forever — finalizer never spawned, report never written, orchestrator parked. A completion BARRIER is only correct level-triggered: registering a join predecessor on a turn-completed binding whose filtered session is ALREADY terminally completed (durable status completed, no armed wake) now delivers a catch-up fire shaped like the real completion event, stamped __late_subscription_replay. Joins only — their per-key accumulator makes a rare double-delivery idempotent. Fail-open: a status lookup failure keeps edge semantics. The registration response carries a note naming the replay. E2E-009 late-join-predecessor-replay gates it end to end: a worker session completes (proved past via a call-mode completion witness — recorder call #1), then a probe-steered turn registers the late join; the joined downstream's recorder call #2 exists only through the replay. Gate check: disabling the replay times the scenario out. Runner: ProbeAction gains after_target_calls (an untracked session's only observable milestone is a controlled-function call). Verified live: rctest5 attempt 5 completed fully green — report row self-verifying pass, totals == GROUP BY (5/5/5), 15/15 events, cited trigger-spawned reactor, zero leaked subscriptions, no polling.
andersonleal
added a commit
that referenced
this pull request
Jul 24, 2026
rctest5 attempt 4 exposed the registration-vs-completion race: a rejected sibling (the MOT-4210 divergent-spec check, working as designed) forced a re-registration round, the watched writer sessions finished inside that window, and the re-armed edge-triggered turn-completed join predecessors starved forever — finalizer never spawned, report never written, orchestrator parked. A completion BARRIER is only correct level-triggered: registering a join predecessor on a turn-completed binding whose filtered session is ALREADY terminally completed (durable status completed, no armed wake) now delivers a catch-up fire shaped like the real completion event, stamped __late_subscription_replay. Joins only — their per-key accumulator makes a rare double-delivery idempotent. Fail-open: a status lookup failure keeps edge semantics. The registration response carries a note naming the replay. E2E-009 late-join-predecessor-replay gates it end to end: a worker session completes (proved past via a call-mode completion witness — recorder call #1), then a probe-steered turn registers the late join; the joined downstream's recorder call #2 exists only through the replay. Gate check: disabling the replay times the scenario out. Runner: ProbeAction gains after_target_calls (an untracked session's only observable milestone is a controlled-function call). Verified live: rctest5 attempt 5 completed fully green — report row self-verifying pass, totals == GROUP BY (5/5/5), 15/15 events, cited trigger-spawned reactor, zero leaked subscriptions, no polling.
andersonleal
added a commit
that referenced
this pull request
Jul 24, 2026
… subscriptions (#586) * (MOT-4217) fix(harness): let reaction sessions unregister their run's subscriptions rctest5-K7mQ ended deadlocked: the repair reactor hit `subscription belongs to a different session` trying to clean up the run, because unregistration is owner-session-scoped and the owner — the orchestrator — was parked waiting on a report notification its children could no longer satisfy. Three armed subscriptions leaked. Reaction spawns now record child → registrant lineage in the ephemeral subscription registry, and the unregister ownership check accepts any session whose lineage chain reaches the owner (transitively, hop-bounded against re-targeted-session cycles). Lineage entries are purged on session::deleted along with the session's subscriptions. * (MOT-4217) test(harness): E2E-008 gates lineage unregister; serve-time captures; armed-wake advisory Three pieces from the rctest5 live-run iteration: * Router serve-time captures: a generation can capture a runtime value (regex over its matched request) and later frames echo it via [[cap:name]] — the only way a static fixture can call engine::unregister_trigger with a runtime-generated sub_… id. Validation requires declared-before-referenced and a capture group. * E2E-008 reaction-unregisters-run: a STANDING binding's reaction, pinned to a separate session, unregisters the registrant's subscription. The gate is gen4's matcher demanding the unregister function_result with is_error:false AND removed:true (once:false is load-bearing — a one-shot binding retires itself first and the check never runs). The await phase now also drains the script after the awaited target call, so collection can't race an untracked session's turn tail. Gate check: reverting the ownership check to pre-lineage semantics times the scenario out. * armed_wake_advisory: a one-shot state-key wake registration now warns that nothing fires it automatically and the session sleeps forever unless a registered task explicitly sets that scope/key — the exact wiring gap that left the rctest5 orchestrator parked with its cleanup pending after every row landed correctly. * (MOT-4217) feat(harness): level-triggered join predecessors + E2E-009 rctest5 attempt 4 exposed the registration-vs-completion race: a rejected sibling (the MOT-4210 divergent-spec check, working as designed) forced a re-registration round, the watched writer sessions finished inside that window, and the re-armed edge-triggered turn-completed join predecessors starved forever — finalizer never spawned, report never written, orchestrator parked. A completion BARRIER is only correct level-triggered: registering a join predecessor on a turn-completed binding whose filtered session is ALREADY terminally completed (durable status completed, no armed wake) now delivers a catch-up fire shaped like the real completion event, stamped __late_subscription_replay. Joins only — their per-key accumulator makes a rare double-delivery idempotent. Fail-open: a status lookup failure keeps edge semantics. The registration response carries a note naming the replay. E2E-009 late-join-predecessor-replay gates it end to end: a worker session completes (proved past via a call-mode completion witness — recorder call #1), then a probe-steered turn registers the late join; the joined downstream's recorder call #2 exists only through the replay. Gate check: disabling the replay times the scenario out. Runner: ProbeAction gains after_target_calls (an untracked session's only observable milestone is a controlled-function call). Verified live: rctest5 attempt 5 completed fully green — report row self-verifying pass, totals == GROUP BY (5/5/5), 15/15 events, cited trigger-spawned reactor, zero leaked subscriptions, no polling. * style(harness): rustfmt across the stacked e2e additions * review(harness): validate probe-action call gates like await_target_calls Same authoring-mistake class CodeRabbit flagged on the await knob: a zero after_target_calls fails at runtime and a positive one without a controlled function waits until the deadline burns. Reject both in ScenarioFixture::validate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
proof is an iii worker that scans code changes and verifies them in a real Chromium browser using snapshot-driven accessibility testing.
Any agent connected to the engine (Claude Code, Codex) can drive the browser through iii functions — no API key needed. For CI/automated runs, proof drives Claude directly via the Anthropic API.
How it works
The snapshot-driven approach:
proof::browser::snapshotreturns an ARIA accessibility tree with[ref=eN]markers. The agent reads refs, not CSS selectors — making tests resilient to UI changes.What's included
iii primitives used
proof:reports), saved flows (proof:flows)Quick start
Then in Claude Code: "test my changes at localhost:3000"
Tested
iii.trigger())Test plan
iii trigger --function-id='proof::scan' --payload='{"target":"unstaged"}'iii trigger --function-id='proof::browser::launch' --payload='{"runId":"t1","headed":true}'iii trigger --function-id='proof::browser::navigate' --payload='{"url":"https://example.com"}'iii trigger --function-id='proof::browser::snapshot' --payload='{}'iii trigger --function-id='proof::browser::click' --payload='{"ref":"e2"}'iii trigger --function-id='proof::browser::close' --payload='{"runId":"t1"}'curl localhost:3111/proof/flowsreturns[]curl localhost:3111/proof/historyreturns[]Summary by CodeRabbit
New Features
Documentation