Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 234 additions & 0 deletions docs/design/review-toolchain-adapters.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
# Review toolchain adapters

## Status

Accepted, implemented. This document covers the extraction of the toolchain
adapter boundary: the npm-specific `qwen review build-test` behavior moves
behind an internal contract without changing its command-line interface or
report format. The phase that adds the first second adapter appends its own
section here.

## Problem

`qwen review build-test` currently combines three responsibilities in one
module:

1. Reading the review plan and selecting changed files.
2. Deciding which repository toolchain can be verified deterministically.
3. Implementing npm workspace installation, affected-package selection,
dependency widening, build execution, test execution, and result reporting.

The command works well for npm repositories, but its public report models the
implementation directly as `toolchain: "npm" | "unsupported"`. Agent 7 falls
back to prompt-directed Maven, Gradle, Cargo, Go, or Python commands when the
npm path is unsupported. That fallback is useful, but it is not deterministic
infrastructure: module selection, command choice, result parsing, timeout
classification, and failure attribution remain agent decisions.

Adding Maven and Gradle directly to `build-test.ts` would create a growing
conditional command rather than a stable cross-language verification boundary.
It would also make the existing npm behavior harder to protect while new
languages are added.

## Goals

P0 must:

- Introduce a small internal toolchain adapter contract.
- Move npm repository detection and npm build/test execution behind the npm
adapter.
- Preserve the `qwen review build-test` CLI arguments.
- Preserve the existing `BuildTestReport` JSON shape and all npm behavior.
- Preserve the exported `runBuildTest`, `trimOutput`, `buildRunEnv`,
`spawnTimedOut`, and `unresolvedWorkspaceDeps` test seams.
- Keep unsupported repositories on the existing Agent 7 fallback path.
- Make Maven and Gradle additions possible without modifying command routing or
verdict composition.

## Non-goals

P0 does not:

- Execute Maven or Gradle.
- Support multiple toolchains in one repository.
- Define a third-party plugin API or dynamic adapter loading.
- Parse test coverage artifacts such as Istanbul, LCOV, or JaCoCo.
- Generalize `test-efficacy`, which remains npm workspace and Vitest specific.
- Change Agent 7 prompts, findings, verdicts, or coverage gates.
- Change the `BuildTestReport` JSON schema.

Multi-toolchain repositories are an expected future requirement, but P0 does
not introduce an unused aggregation model. The adapter contract is scoped to
one verification target so a later orchestrator can select multiple targets
without changing an individual adapter.

## Current behavior to preserve

The npm implementation currently:

- Treats a root package with build or test scripts as a single package.
- Supports the modeled npm workspace glob shapes.
- Selects changed workspaces from plan file paths.
- Builds affected workspaces and their reverse dependents.
- Widens or reorders the build set when the compiler names an undeclared
workspace dependency.
- Tests the affected workspaces and every workspace declared to depend on
them that defines a test script.
- Runs `npm ci` only for an npm repository with an incomplete dependency tree.
- Avoids `npm ci` for warm Yarn, pnpm, and Bun trees.
- Classifies unsupported layouts as a handoff, not a successful verification.
- Classifies timeouts, insufficient disk, and unusable installs as
infrastructure rather than PR findings.
- Removes failed intermediate widening attempts from the final evidence.
- Supports build-only verification for merge-base trees.

The existing focused test suite is the compatibility oracle for these rules.

## Design

### Adapter contract

Add an internal `ReviewToolchainAdapter` interface with:

- An `applies` method that decides whether the adapter owns the repository.
- A `run` method that receives normalized build/test arguments and changed file
paths and returns the existing report shape.

P0 registers one built-in adapter, npm. It applies when the root
`package.json` describes something npm can build — workspaces, or a root
`build`/`test` script; the adapter's existing execution logic then decides
whether the npm layout and dependency state are supported or require the
structured handoff used today. The registry is a fixed array in code. There is no extension
discovery or configuration surface.

P0 deliberately does not claim to solve mixed-toolchain selection. Static
repository detection alone cannot know whether an adapter will later decline
because of changed-file ownership or cold dependency state. The Maven phase must
design target selection from two real adapters and their module models rather
than freezing a speculative priority rule now.

### Command boundary

`build-test.ts` remains the CLI boundary and compatibility facade. It:

1. Resolves the worktree.
2. Reads and validates changed file paths from the review plan.
3. Selects the sole applicable built-in adapter, failing closed to the
unsupported report when zero or more than one apply.
Comment on lines +116 to +117

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-16: The documented command-boundary flow (and the Goals bullet "Move npm repository detection … behind the npm adapter") omits that on the zero-adapters path build-test.ts keeps its own npm-shapedness detection and calls the npm adapter directly, outside the registry: if (existsSync(join(root, 'package.json'))) return npmToolchainAdapter.run(runArgs);. — Failure scenario: the numbered contract says zero adapters → unsupported report, full stop, and claims detection moved behind the adapter → a second-adapter author implementing against it emits the generic note for zero-adapter roots and drops npm's precise handoff note for every unscopable npm root (unmodeled glob, empty glob, script-less package.json) — exactly the note-quality property the renamed test in this PR pins.

Suggested fix: add the delegation step to the command-boundary list (zero adapters + root package.json → delegate the handoff to the npm adapter for its precise note; zero adapters + no package.json → generic unsupported report), and qualify the Goals bullet to say npm applicability detection moves behind the adapter while the facade retains the npm-shaped fallback routing.

中文说明

文档记载的命令边界流程(以及 Goals 中「把 npm 仓库检测移到 npm adapter 之后」一条)遗漏了:在零 adapter 路径上,build-test.ts 保留了自己的 npm 形态检测,并在注册表之外直接调用 npm adapter:if (existsSync(join(root, 'package.json'))) return npmToolchainAdapter.run(runArgs);。— 失败场景:编号契约说零 adapter → unsupported 报告,到此为止,并声称检测已移入 adapter → 第二个 adapter 的作者照此实现时,会对零 adapter 的根输出通用 note,丢掉所有无法 scope 的 npm 根(未建模 glob、空 glob、无脚本 package.json)上 npm 的精确交接 note——正是本 PR 重命名测试所钉住的 note 质量属性。

修复建议: 在命令边界清单中补上委托步骤(零 adapter + 根有 package.json → 委托 npm adapter 产出精确 note 的交接;零 adapter + 无 package.json → 通用 unsupported 报告),并限定 Goals 的措辞:npm 适用性检测移入 adapter,facade 保留 npm 形态的兜底路由。

— qwen3.8-max via Qwen Code /review (v0.21.8)

4. Calls the adapter.
5. Emits the unchanged JSON report.

The npm-specific implementation owns package discovery, install policy,
workspace selection, build ordering, widening, tests, and npm-specific notes.

### Report compatibility

P0 deliberately keeps:

```text
toolchain: "npm" | "unsupported"
```

Changing this to a new generic schema in the same refactor would require
coordinated edits to Agent 7, base-tree, test-plan, test-delta, tests, and any
external scripts consuming the report. The adapter boundary does not require
that migration.

A later Maven/Gradle phase can widen the discriminant while adding the first
new behavior, with tests for each downstream consumer.

### Shared execution primitives

Command execution, output trimming, timeout detection, and environment shaping
remain shared exports from the command module in P0 because adjacent review
commands and existing tests consume them. The npm-specific dependency widening
Comment on lines +142 to +144

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-15: The doc states command execution "remain[s] shared exports from the command module … because adjacent review commands and existing tests consume them", but the executor run is module-private (no export, unchanged context line), nothing imports it (test-delta.ts imports only buildRunEnv, spawnTimedOut, trimOutput), and the next paragraph contradicts it: "It does not import the command's runtime executor … the command selects the adapter and passes execution in." — Failure scenario: a Maven-phase author implementing against "Shared execution primitives" expects an importable executor carrying the spawn behavior — including the fractional-deadline clamp this PR adds inside private run — and, finding none, re-implements spawn/env-shaping in the new adapter → forks the deadline-coercion and timeout-as-data semantics the doc claims are shared.

Suggested fix: rewrite as "Output trimming, timeout detection, and environment shaping remain shared exports … Command execution stays private to the command module and reaches the adapter through the injected exec argument."

中文说明

文档声称命令执行「仍是命令模块的共享导出……因为相邻 review 命令和现有测试会消费它们」,但执行器 run 是模块私有的(无 export,未改动的上下文行),没有任何模块导入它(test-delta.ts 只导入 buildRunEnvspawnTimedOuttrimOutput),且下一段与之矛盾:「It does not import the command's runtime executor … the command selects the adapter and passes execution in.」。— 失败场景:Maven 阶段的作者按「Shared execution primitives」实现时,会期待一个可导入、带 spawn 行为的执行器——包括本 PR 在私有 run 中新增的小数 deadline 取整——发现没有后便在新 adapter 里重新实现 spawn/环境构造 → 使文档声称共享的 deadline 强制与「超时即数据」语义发生分叉。

修复建议: 改写为 「Output trimming, timeout detection, and environment shaping remain shared exports … Command execution stays private to the command module and reaches the adapter through the injected exec argument.」

— qwen3.8-max via Qwen Code /review (v0.21.8)

helper moves with the npm adapter and is re-exported from the command module for
compatibility.

The adapter receives the injectable executor already used by the existing unit
tests. It does not import the command's runtime executor, so the dependency stays
one-way: the command selects the adapter and passes execution in. Type-only
imports may reference the existing report types without creating a runtime
cycle. This preserves deterministic tests without spawning npm.

## Files

P0 changes:
Comment on lines +154 to +156

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-7: The P0 changes: file list omits packages/cli/src/commands/review/lib/disk.ts, a module this PR creates (the moved free-disk floors and statfsSync preflight that npm-toolchain.ts imports from ./disk.js). — Failure scenario: the doc is the refactor's committed record — its Status section says the next adapter phase appends its own section, and its Testing section demands the disk behavior stay unchanged — yet the module owning that behavior has no entry → a future phase author auditing the extraction against this list gets no design-level trace that the disk floors left the command module.

Suggested fix: add a bullet: packages/cli/src/commands/review/lib/disk.ts — owns the free-disk floors and the statfsSync preflight shared by the install and build phases.

中文说明

P0 changes: 文件清单遗漏了 packages/cli/src/commands/review/lib/disk.ts——本 PR 新建的模块(搬过来的磁盘空间下限与 statfsSync 预检,npm-toolchain.ts./disk.js 导入)。— 失败场景:该文档是重构的存档记录——其 Status 一节说下一个 adapter 阶段会在此追加自己的章节,Testing 一节要求磁盘行为保持不变——但承载该行为的模块却没有条目 → 未来阶段的作者对照此清单审计搬运时,得不到任何设计层面的线索表明磁盘下限已离开命令模块。

修复建议: 补一条:packages/cli/src/commands/review/lib/disk.ts —— 承载安装与构建阶段共享的磁盘空间下限与 statfsSync 预检。

— qwen3.8-max via Qwen Code /review (v0.21.8)


- `packages/cli/src/commands/review/build-test.ts`
- Retains CLI routing and compatibility exports.
- Selects and invokes the built-in adapter.
- `packages/cli/src/commands/review/lib/toolchain.ts`
- Defines the internal adapter and detection contracts.
- Selects the sole applicable adapter (zero or more than one fails closed).
- `packages/cli/src/commands/review/lib/npm-toolchain.ts`
- Owns npm detection and the existing npm verification algorithm.
- `packages/cli/src/commands/review/lib/npm-toolchain.test.ts`
- Pins adapter selection and contract-level behavior.
- `packages/cli/src/commands/review/build-test.test.ts`
- Remains the end-to-end compatibility suite for the command facade.

## Testing

Focused tests must prove:

1. An npm workspace selects the npm adapter.
2. A single-root npm package selects the npm adapter.
3. A non-npm repository produces the existing unsupported report.
4. An unmodeled npm layout remains unsupported rather than returning a false
Comment on lines +176 to +178

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-13: This Testing item claims a non-npm repository "produces the existing unsupported report", but for roots with no package.json at all the PR changes the note text — runtime A/B confirmed byte-different reports: base emits 'No npm package here to scope (no workspaces, and the root has no build/test script)…'; the PR emits the newly invented 'No supported npm project here to scope…'. The new test named keeps the complete generic unsupported report when no adapter applies pins the changed text under a name that claims preservation, contradicting the Goals' "Preserve … all npm behavior". No consumer string-matches the note, so blast radius is agent-facing prose plus doc/test-name accuracy. — Failure scenario: byte-different report content for the no-package.json repo class contradicts the preservation goal → a future phase author auditing against this list sees green-plus-"keeps" and concludes nothing drifted.

Suggested fix: either emit the pre-PR note string verbatim from the generic fallback (restoring byte-compat), or amend the doc's Goals/Testing wording to say the no-package.json note was rephrased and rename the test so it no longer claims to "keep" the existing report.

中文说明

该 Testing 条目声称非 npm 仓库会「产生既有的 unsupported 报告」,但对完全没有 package.json 的根,本 PR 改变了 note 文案——运行时 A/B 确认报告字节级不同:基线输出 'No npm package here to scope (no workspaces, and the root has no build/test script)…';PR 输出新造的 'No supported npm project here to scope…'。名为 keeps the complete generic unsupported report when no adapter applies 的新测试用声称「保持」的名字钉住了改变后的文案,与 Goals 中「保留……所有 npm 行为」矛盾。没有消费方对该 note 做字符串匹配,因此影响范围是面向 agent 的文案加文档/测试名准确性。— 失败场景:无 package.json 仓库类别的报告内容字节级不同,违背保留目标 → 未来阶段作者对照此清单审计时看到全绿加「keeps」,会得出「没有漂移」的结论。

修复建议: 要么让通用兜底逐字输出 PR 前的 note 字符串(恢复字节级兼容),要么修订文档 Goals/Testing 的措辞说明无 package.json 的 note 被改写,并给测试改名使其不再声称「保持」既有报告。

— qwen3.8-max via Qwen Code /review (v0.21.8)

green result.
5. Existing build ordering, widening, install, timeout, disk, and test behavior
remains unchanged through `runBuildTest`.
6. The serialized report shape remains unchanged.

Verification commands:

```bash
cd packages/cli && npx vitest run src/commands/review/
npm run typecheck
```

## Future phases

### A second toolchain

The boundary exists so a second language lands as a registration rather than
another branch in `build-test.ts`. Whichever comes first — Maven, Gradle —
should prefer a checked-in wrapper, take its project model from the build tool
itself rather than re-deriving one from the manifests, select the projects the
diff changed, and parse the JUnit XML the run produced. A build it cannot
model must fail closed to an unsupported handoff, never to a partial green.

### Coverage artifacts

Istanbul/LCOV and JaCoCo should normalize into a language-independent
changed-line and changed-branch coverage model. Coverage numbers are evidence
for a concrete untested behavior, not an automatic Critical threshold.

### Multiple toolchains

A later orchestration layer may detect multiple verification roots and invoke
one adapter per target. It should aggregate evidence while preserving each
command's toolchain, root, module, and infrastructure status. This phase avoids
specifying that before two real adapters demonstrate the common boundary.

## Risks

- **Accidental report drift:** protected by the existing `build-test` suite and
explicit report-shape assertions.
- **Adapter abstraction without behavior:** this phase is justified only if npm
detection and execution move behind the adapter rather than adding an empty
interface around unchanged branching.
- **Premature generalization:** the contract intentionally excludes coverage,
mutation, CI discovery, and multi-toolchain aggregation.
- **False applicability:** the npm adapter applies only when the root
`package.json` can scope something — workspaces or a root build/test script.
A package.json with neither workspaces nor build/test scripts (husky, a lint
config, a script-less docs site) does not apply, so a future adapter can own
such a root alone rather than losing it to a manifest npm cannot scope.

## Open questions

Report-schema widening for a second toolchain — the `toolchain` discriminant
and any per-command classification flags — is deferred to the phase that
introduces that behavior, along with multi-toolchain aggregation.
Loading
Loading