[lidge-jun]

GOAT's paired agent-surface campaigns do not cover small tool-use mistakes such as polling an async child or linking a PR that was only reviewed. Add seven replay cases and an offline continuation grader beside the existing eval tooling, using GOAT's real registry, served core/full listings and current runtime instructions.

Upstream commit	Upstream PR	Verdict	What GOAT got
82dc892c5a8b4879783444595c3e49f8ba8de64c	pingdotgg/t3code#24635	adapt	Review-round, async-delegation, managed-worktree, PR ownership, thread-link and preview-before-render cases; argument/order/count/text grading and negative grader coverage.

GOAT adjustments:

• Keep everything in scripts/agent-surface/eval. No workspace package, dependencies, inventory copy, paid model calls or dev server.
• Resolve tool names through listedNames.ts and obtain full/core tools through loadServedCatalog().listedAsThread; export this checkout's orchestration/development/browser instructions rather than captured upstream prompts.
• Accept direct listed tools and goat_call {path,args} consistently. Reject unknown GOAT paths, unlisted direct calls, foreign-server names used as GOAT calls, failed required calls, malformed traces and unfinished runs.
• Include a good and mistaken continuation for every case against both served listings, plus focused negative grader tests. Cover both provider-option wire forms and numeric argument bounds.
• The CLI exports harness-independent context or grades a supplied continuation. It does not call a model, simulate tool results, execute nested JavaScript or prove server-side success. This remains separate from D8's paired gate and does not change the campaign task-set version.
• Applicable surfaces: maintainer eval CLI, registry/listing consumption and operations guidance. No web, desktop, mobile, adapter, wire-schema or server behavior changes.

Not taken:

• The upstream eval workspace, lockfile/dependency changes, 1,861-line model runner, paid shell simulator, compressed harness captures, capture trimmer and thread-title relocation. GOAT already has its own eval framework; the authorized scope excludes these additions.
• Upstream orchestration/runtime and PR-reactor edits are outside this replay adaptation. GOAT's current runtime instructions remain authoritative.
• Provider-specific/native-subagent and browser policy cases that require different harness machinery or conflict with GOAT's screen-code/browser stance. The selected HTML case tests the existing html.preview then html.render rule only.

Rename residue (inspected added upstream lines with git show 82dc892c5a | rg '^\+.*\b[Tt]3'):

• No upstream source hunks were copied. All upstream identifiers below are excluded with their original hunks; the selected behaviors were rewritten using existing GOAT registry paths. No new rename rule or kept legacy name is needed.
• T3, T3 Code, t3, t3code, t3-code, t3-browser, .t3, t3-f8cdd940: upstream branding, package/readme/template/world data excluded. New case histories are invented GOAT data.
• @t3tools/evals, @t3tools/contracts, @t3tools/provider-core, @t3tools/shared: upstream workspace/import/lockfile hunks excluded; reuse the existing GOAT scripts package.
• t3_thread_send, t3_thread_read, t3_thread_wait: original names excluded; selected behaviors use thread.send, thread.read, thread.wait and listed goat_thread_* names.
• t3-thread: original link literal excluded; selected thread-link behavior uses goat-thread://v1/.
• t3_thread_interrupt, t3_thread_launch, t3_thread_list, t3_thread_search, t3_preview_close, t3_preview_list: unselected case/runner hunks excluded.
• T3_PREFIX, t3SourceTools, t3Tools, t3SystemAppend, t3ContextReport, t3Default, t3_code_, t3_code_runtime, T3_DEBUG_WS, T3_TRACE_SQL: upstream runner/template/helper/fixture hunks excluded. Existing GOAT frozen wrappers are untouched.
• node scripts/goat-rename.ts --check: exit 0, 0 lines with T3 residue in 0 files, 0 dropped paths on the staged change.

Verification:

• vp fmt scripts/agent-surface/eval/replay.ts scripts/agent-surface/eval/replay-cases.ts scripts/agent-surface/eval/replay-run.ts scripts/agent-surface/eval/replay.test.ts docs/operations/agent-surface-eval.md: exit 0 (5 files).
• git diff --cached --check, git diff --check, git diff origin/dev...HEAD --check: exit 0.
• Private-data term scan of the staged diff: no matches for personal paths, known credential variable names, bearer tokens or maintainer contact identifiers.
• Normal pre-push workflow guard passed. The vp staged commit hook was bypassed for this commit to honor the explicit local-check override.
• goat-ci is the verification authority for this PR. CI has not been awaited; no test/typecheck success is claimed.

NOT RUN: vp test run scripts/agent-surface/eval/replay.test.ts scripts/agent-surface/eval/core.test.ts scripts/agent-surface/eval/rows.test.ts, other local test suites/vitest, vp run check:changed, vp run typecheck, lockf-wrapped heavy checks, vp staged commit hook, separate lint/knip/build checks, replay CLI context/trace runtime smoke, paid model evaluations; browser captures and native mobile verification are unavailable on this machine. CI results were not awaited, per instruction.

Review rounds 1-2

Review round 1

• Finding 1 (missing Knip entry): fixed in dbc56729142e3c457b9df8c4fcef8a33715b3bd4. Register the standalone replay CLI in the scripts workspace entry graph. Add a real-Knip fixture regression in scripts/knip-schemas.test.ts that requires the CLI to stay reachable while an unrelated unused script remains reported.
• Finding 2 (Monitor bypass): fixed in 09b89fbb3c1489bc4a98d9634ad6283ca39e59de. Recognize native Monitor and functions.Monitor alongside Bash and exec_command. Add otherwise-valid review and managed-worktree continuations that fail only the forbidden-shell check; cover direct and functions-prefixed shell names.
• Finding 3 (child-created unmanaged checkout): fixed in 09b89fbb3c1489bc4a98d9634ad6283ca39e59de. Add callsMustNot with the existing partial-argument matcher and forbid child task prompts containing git worktree add or git clone. Regression continuations retain isolation: "worktree" and cover both direct calls and dispatcher calls. Raise REPLAY_VERSION to 2 and update the documented trace example.

Fresh local evidence: changed-file vp fmt exited 0; git diff --check and git diff de8fc894d75351e3d573713943a632b249faf103...HEAD --check exited 0; node scripts/goat-rename.ts --check exited 0 with 0 lines with T3 residue in 0 files, 0 dropped paths. Private-data scan of added review-fix lines found no matches. The pre-commit vp staged hook was bypassed to honor the local-execution override; the normal pre-push workflow guard remains enabled.

The regression tests were written before their respective fixes, but were not executed locally. Red/green proof and test/knip success are not claimed. goat-ci must verify the current PR head; its results were not awaited. The known inherited ComputerUseLeases.ts code-budget issue is unchanged.

NOT RUN: vp test run scripts/knip-schemas.test.ts scripts/agent-surface/eval/replay.test.ts, other local suites/vitest, red/green mutation runs, vp run check:changed, lockf -k /tmp/goat-heavy-check.lock vp run check:changed, vp run typecheck, vp staged, separate lint/knip/build checks, replay CLI context/trace smoke, paid model evaluations; browser captures and native mobile verification are unavailable. CI results were not awaited.

Review round 2

• Finding 1 (child-PR CI polling bypass): addressed in 766b71cbd84cb58e0b8a611ed671313e5abbc31f. Restore the retained upstream case's until polling of gh pr|run|api, for polling of gh pr checks, and pr-wait negatives. Add five otherwise-valid negative traces and a one-shot gh pr checks passing control in the existing replay test file. Raise REPLAY_VERSION and the documented example to 3.
• Verified all three round-1 changes remain present by reading their implementation and regression cases. No rebuttals or deferred findings.

Fresh local evidence for round 2: vp fmt scripts/agent-surface/eval/replay.test.ts scripts/agent-surface/eval/replay-cases.ts docs/operations/agent-surface-eval.md completed successfully (3 files); git diff --check, git diff --cached --check, and git diff 09b89fbb3c1489bc4a98d9634ad6283ca39e59de --check exited 0. node scripts/goat-rename.ts --check exited 0: 0 lines with T3 residue in 0 files, 0 dropped paths. Added lines were inspected and contain only invented fixture data. The commit hook (vp staged) was disabled for this commit to honor the explicit local-execution override; the normal pre-push guard remained enabled.

The regression cases were added before the rule change but were not executed locally. No red/green, test, typecheck or Knip success is claimed. goat-ci on 766b71cbd84cb58e0b8a611ed671313e5abbc31f is the verification authority; CI is not awaited. The inherited ComputerUseLeases.ts code-budget issue is unchanged. No new PR, watch, merge or review request was made.

NOT RUN: focused vp test run scripts/agent-surface/eval/replay.test.ts scripts/agent-surface/eval/core.test.ts scripts/agent-surface/eval/rows.test.ts and vp test run scripts/knip-schemas.test.ts scripts/agent-surface/eval/replay.test.ts; all other local test suites/vitest and red/green runs; vp run check:changed, lockf -k /tmp/goat-heavy-check.lock vp run check:changed, vp run typecheck, vp staged, lint/Knip/build checks, replay CLI/context/trace smoke, paid model evaluations; browser captures and native mobile verification are unavailable. CI results were not awaited.

CI fixes: 286e28fc680b00d57d5809752fb4eb034b61f03d on 766b71cbd84cb58e0b8a611ed671313e5abbc31f — match the sibling eval CLIs with a documented plain-Node globalConsole:off directive in replay-run.ts; preserve JSON output and assertions. Read failing jobs 122433685883 and 122433685972. Static checks passed: node scripts/goat-rename.ts --check, changed-file vp fmt, changed-file vp lint, and git diff --check. vp exec oxlint was attempted but the executable is absent; vp lint passed instead. Leave the inherited DesktopTargetHost.test.ts code-budget failure to #1972.

NOT RUN: local test suites (vp test run, vitest, red/green runs), vp run check:changed, vp run typecheck, lockf-wrapped checks, vp staged commit hook, separate Knip/build checks, replay CLI runtime smoke, browser/mobile verification. goat-ci verifies the pushed head; CI was not awaited.

Model: gpt-6.1-sol via GOAT delegated task (Codex harness)