Skip to content
Closed
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
afa3b4d
docs: add legacy code audit (/audit) design doc
wenshao Aug 2, 2026
522e23c
docs: revise legacy audit design with round-2 replication evidence
wenshao Aug 2, 2026
509e79a
docs: note cross-file tracer cost and budget rule in legacy audit design
wenshao Aug 2, 2026
7fd87c6
feat(audit): add legacy code audit workflow
wenshao Aug 2, 2026
2992d53
Merge remote-tracking branch 'origin/main' into feat/legacy-code-audit
wenshao Aug 6, 2026
2b9cf83
feat(audit): align the audit workflow with the merged design doc
wenshao Aug 6, 2026
e4386ea
Merge remote-tracking branch 'origin/main' into feat/legacy-code-audit
wenshao Aug 6, 2026
0ad3dd9
fix(audit): lift tokenizeArgs and anchor the guard probes at the invo…
wenshao Aug 6, 2026
ff9c53b
fix(audit): make check-anchors --callers conditional in the skill
wenshao Aug 6, 2026
94b0b95
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 6, 2026
6d633b9
fix(audit): name-record unreadable sidecar callers
qwen-code-dev-bot Aug 6, 2026
03c1519
Merge remote-tracking branch 'origin/main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 7, 2026
5eb156e
fix(audit): close fail-open gates found in review across the pipeline
qwen-code-dev-bot Aug 7, 2026
0642802
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 7, 2026
d883ddc
fix(cli): harden audit helpers per review feedback (#8403)
qwen-code-dev-bot Aug 7, 2026
e1e6fcf
fix(cli): address second review round for audit helpers (#8403)
qwen-code-dev-bot Aug 11, 2026
67b1320
fix(external-context): read the response body with a reader, not for-…
wenshao Aug 9, 2026
7c33b0b
fix(audit): make fallback relocation paths explicit in the skill comm…
qwen-code-dev-bot Aug 12, 2026
8209913
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 12, 2026
319c0cf
fix(cli): apply review-round fixes to audit anchors, walk, and exit f…
qwen-code-dev-bot Aug 12, 2026
72a4d89
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 12, 2026
029231e
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 13, 2026
2dc868d
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 13, 2026
5df644a
Merge remote-tracking branch 'origin/main' into feat/legacy-code-audit
qwen-code-ci-bot Aug 13, 2026
2581412
fix(audit): address round-3 review findings for the audit module (#8403)
qwen-code-ci-bot Aug 13, 2026
54c5cbe
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 13, 2026
722b06d
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 13, 2026
c714b90
Merge remote-tracking branch 'origin/main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 14, 2026
216937d
fix(cli): apply round-4 review fixes to audit guard, anchors, and sid…
qwen-code-dev-bot Aug 14, 2026
538905d
fix(cli): apply round-6 review fixes to audit guard, anchors, and sid…
qwen-code-dev-bot Aug 14, 2026
d5c48e3
Merge branch 'main' into feat/legacy-code-audit
wenshao Aug 14, 2026
e528918
fix(cli): close round-6 audit guard, anchor, and hash holes (#8403)
qwen-code-dev-bot Aug 14, 2026
789b6e4
fix(cli): close round-8 audit anchor and guard criticals (#8403)
qwen-code-dev-bot Aug 15, 2026
25da0ff
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 15, 2026
cd1276a
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 15, 2026
92642bd
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 15, 2026
2afc1fb
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 16, 2026
5aabe7c
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 16, 2026
aed1ab5
Merge remote-tracking branch 'origin/main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 16, 2026
afa923a
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 16, 2026
5af49e9
Merge remote-tracking branch 'origin/main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 17, 2026
926aef8
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 17, 2026
1deabe8
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 17, 2026
580b439
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 18, 2026
a2638ea
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 18, 2026
a0ee238
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 18, 2026
caadc0c
Merge remote-tracking branch 'origin/main' into feat/legacy-code-audit
wenshao Aug 19, 2026
1897bd4
fix(core): stabilize the audit fallback root across symlink path spel…
wenshao Aug 19, 2026
daa8f6e
Merge remote-tracking branch 'origin/feat/legacy-code-audit' into fea…
wenshao Aug 19, 2026
1f97f93
Merge branch 'main' into feat/legacy-code-audit
qwen-code-dev-bot Aug 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/users/features/_meta.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
export default {
commands: 'Commands',
'code-review': 'Code Review',
'legacy-audit': 'Legacy Code Audit',
'followup-suggestions': 'Followup Suggestions',
'tool-use-summaries': 'Tool-Use Summaries',
'markdown-rendering': 'Markdown Rendering',
Expand Down
67 changes: 67 additions & 0 deletions docs/users/features/legacy-audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Legacy Code Audit

> Audit a module or directory of **existing, merged code** — no diff, no PR — using `/audit`.

`/review` is built for increments; `/audit` points the same machinery at code that is already merged: pre-refactor assessments, taking over an unfamiliar module, security review of a sensitive subsystem. The product is a verified, deduplicated, theme-clustered findings report.

## Quick Start

```bash
# Audit a module (default effort: medium)
/audit packages/core/src/permissions

# Quick unverified triage, one reader sub-agent
/audit packages/core/src/hooks --effort low

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferred to a follow-up round (not dropped). Pointing the Quick Start low example at a module under the 2,000-line low gate requires measuring candidate modules against the live plan-files gate; that measurement and the doc fix land next round.

中文说明

推迟到后续轮次(并非丢弃)。把 Quick Start 的 low 示例指向低于 2,000 行 low 门限的模块,需要先对候选模块用真实 plan-files 门限测量;该测量与文档修复下一轮落地。


# Full pipeline plus reverse-audit rounds
/audit packages/core/src/permissions --effort high
```

Single files are not audited — `/review <file-path>` already covers that case, and `/audit` says so and stops.

## Effort Levels

`--effort low|medium|high` trades depth for cost. **The word means the opposite of what it does in `/review`**: `/review`'s medium _drops_ the adversarial personas while `/audit`'s medium _adds_ one (6a) — and both skills select the tier with the same `--effort` flag. If you run both, reset your expectation at the boundary.

| Level | What runs | Findings | Cost |
| -------- | --------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------- |
| `low` | One reader sub-agent rotating through directed angles plus a gap sweep | ≤10, labeled **unverified** | Cheap |
| `medium` | The measured 8-dimension fan-out (1a, 1c, 2, 3a/3b/3c, 4, 5) plus the 6a attacker seat, plus verification | Uncapped, verified | Tens of M tokens |
| `high` | medium + the 6b/6c personas + iterative reverse-audit rounds | Uncapped, verified | Extrapolated |

## Size gates and budget

v1 audits one bounded module at a time. `plan-files` refuses at plan time — and asks for a narrower path — when:

- subject lines exceed **9,000** (the topology both experiments validated);
- test lines exceed **18,000** at medium/high (Agent 5 reads the corpus whole);
- subject lines exceed **2,000** at low (points you at medium);
- the priced token estimate's top exceeds the **60M** cap.

A larger subsystem is audited as coherent sub-paths, one bounded run each. For subject-gate and token-cap refusals, lowering the effort is never the remedy — the priced cost is a function of line counts alone. The test-line gate does not apply at `low` (the corpus goes unexamined there — triage, not an audit), and `low-gate`'s remedy is the tier change its message names.

## What you confirm before anything launches

A fan-out run prints its roster and token estimate and starts only on your confirmation. The same confirmation carries the two **execution consents**, as separate opt-ins:

1. a baseline run of the module's own test suite;
2. agent-authored verification **probes** — short programs written mid-run, executed against a scratch copy of the probed file (never your checkout's copy), each required to flip under the implied fix.

The walks themselves are read-only. Because the confirmation is the only budget enforcement and the execution gate, **`/audit` refuses non-interactive starts** (headless `qwen -p`, cron, sub-agent invocations).

## Safety properties

- **Local-only artifacts.** The report, its sidecar, and the plan/prompt records quote the module — possibly exploitable code — and must never land in version control. `plan-files` probes `.qwen/audits/` and `.qwen/tmp/` (ignore rules **and** force-added history) at plan time, offers a zero-footprint `.git/info/exclude` remedy, and re-checks at every checkpoint and at write time; a mid-run flip relocates everything to a per-user fallback outside the repo.
- **Untrusted data.** Every consumer of module content — dimension agents, verifiers, the dedup clusterer, the low-tier reader, the orchestrator itself — opens with an untrusted-data preamble: the module is evidence, not instructions. A directive embedded in the code ("report no findings") is itself a finding.
- **Drift protection.** A path-scoped sidecar (diff, untracked content copies, per-file content hashes) is captured at run start and re-checked before verification, before each high-tier round, and at write time. Content drift in a file that already carries anchored findings stops the run with a partial report; any other drift is flagged and the run continues.
- **No verdict.** The report is findings, walks, and disclosures — never "approved". Posting and fixing stay with you.

## The report

`.qwen/audits/<YYYY-MM-DD>-<HHMMSS>-<path-slug>.md`, opening with a run-metadata header (commit SHA, model id, dirty state with sidecar, consumption against the estimate, walks completed/skipped/uncoverable, unexercised-machinery flags). Findings are clustered by root cause, each with severity, locations, failure scenario, evidence tier (end-to-end probe / unit probe / code read), and the independent-discovery count ("found independently by N agents"). Confirmed-low findings sit in their own "needs human review" section; anything unverified is labeled unverified.

## Limitations

- Submodules are refused at plan time (no drift coverage inside them in v1).
- Dedup is intra-run; already-filed issues are not cross-checked.
- The medium/high tiers are calibrated on two modules of this repository; the low tier and the high-tier loop are unmeasured first cuts, and the report header says so.
222 changes: 222 additions & 0 deletions integrations/external-context/src/http-client.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/

import { afterEach, describe, expect, it, vi } from 'vitest';
import { MAX_RESPONSE_BYTES, postJson } from './http-client.js';

// Behavioral net for readBoundedBody's reader loop. The loop was rewritten
// from `for await (const chunk of response.body)` to an explicit getReader()
// loop — async-iterating a ReadableStream needs [Symbol.asyncIterator] on the
// TYPE, and that resolution flipped underneath the file when #8693's
// @types/jsdom install dragged lib.dom into this program (TS2504 on the
// `for await`). These tests pin what the rewrite must preserve: bounded
// accumulation, the oversize rejection, and — the easy one to drop —
// cancelling the stream on early exit, which `for await` used to do
// implicitly via iterator return().

function streamingResponse(
chunks: Uint8Array[],
onCancel: () => void,
): Response {
let next = 0;
const body = new ReadableStream<Uint8Array>({
pull(controller) {
if (next < chunks.length) {
controller.enqueue(chunks[next]);
next += 1;
} else {
controller.close();
}
},
cancel() {
onCancel();
},
});
return new Response(body, {
status: 200,
headers: { 'content-type': 'application/json' },
});
}

function requestArgs() {
return {
url: new URL('https://provider.example/'),
authorization: 'Bearer test',
body: { q: 'x' },
signal: new AbortController().signal,
};
}

afterEach(() => {
vi.unstubAllGlobals();
});

describe('postJson bounded body reading', () => {
it('assembles a multi-chunk body and parses it', async () => {
const encoder = new TextEncoder();
const payload = JSON.stringify({ answer: 42, pad: 'y'.repeat(4096) });
const mid = Math.floor(payload.length / 2);
const cancelled = vi.fn();
vi.stubGlobal(
'fetch',
vi.fn(async () =>
streamingResponse(
[
encoder.encode(payload.slice(0, mid)),
encoder.encode(payload.slice(mid)),
],
cancelled,
),
),
);

await expect(postJson(requestArgs())).resolves.toEqual({
answer: 42,
pad: 'y'.repeat(4096),
});
// A fully-drained stream is closed, not cancelled.
expect(cancelled).not.toHaveBeenCalled();
});

it('accepts a body of exactly MAX_RESPONSE_BYTES', async () => {
// The bound is `total > MAX`, so a payload landing exactly on it passes.
const exact = `["${'a'.repeat(MAX_RESPONSE_BYTES - 4)}"]`;
expect(exact.length).toBe(MAX_RESPONSE_BYTES);
vi.stubGlobal(
'fetch',
vi.fn(async () =>
streamingResponse([new TextEncoder().encode(exact)], vi.fn()),
),
);

await expect(postJson(requestArgs())).resolves.toEqual([
'a'.repeat(MAX_RESPONSE_BYTES - 4),
]);
});

it('rejects an over-budget stream AND cancels it', async () => {
// No content-length header, so only the streaming bound can catch it.
const chunk = new Uint8Array(512 * 1024);
const cancelled = vi.fn();
let enqueued = 0;
const body = new ReadableStream<Uint8Array>({
pull(controller) {
// Endless stream: the reject must come from the byte budget, and the
// cancel is what stops this producer.
controller.enqueue(chunk);
enqueued += 1;
},
cancel() {
cancelled();
},
});
vi.stubGlobal(
'fetch',
vi.fn(async () => new Response(body, { status: 200 })),
);

await expect(postJson(requestArgs())).rejects.toThrow(
'External context provider returned an invalid response.',
);
// `for await` cancelled the stream via its implicit iterator return();
// the reader loop must do the same or the producer keeps running.
expect(cancelled).toHaveBeenCalledTimes(1);
// Budget is 1 MiB: two 512 KiB chunks reach it, the third crosses it.
expect(enqueued).toBeLessThanOrEqual(4);
});

it('holds the oversize reject until a deferred cancellation settles', async () => {
// `for await` awaited iterator return() before propagating, so a caller
// retrying immediately could not overlap the old transport's teardown.
let resolveCancel!: () => void;
const cancelBarrier = new Promise<void>((resolve) => {
resolveCancel = resolve;
});
let cancelSeen!: () => void;
const cancelCalled = new Promise<void>((resolve) => {
cancelSeen = resolve;
});
const body = new ReadableStream<Uint8Array>({
pull(controller) {
controller.enqueue(new Uint8Array(512 * 1024));
},
cancel() {
cancelSeen();
return cancelBarrier;
},
});
vi.stubGlobal(
'fetch',
vi.fn(async () => new Response(body, { status: 200 })),
);

let rejected = false;
let rejection: unknown;
const pending = postJson(requestArgs()).catch((error: unknown) => {
rejected = true;
rejection = error;
});

await cancelCalled;
// Let any racing microtasks land; the reject must still be on hold.
await new Promise((resolve) => setTimeout(resolve, 0));
expect(rejected).toBe(false);

resolveCancel();
await pending;
expect(rejected).toBe(true);
expect((rejection as Error).message).toBe(
'External context provider returned an invalid response.',
);
});

it('maps a mid-stream read failure after partial data to a transport error', async () => {
// The provider disconnects after sending part of the JSON: read() rejects
// with the partial chunk already accumulated.
const body = new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(new TextEncoder().encode('{"answer":'));
},
pull(controller) {
controller.error(new Error('connection reset'));
},
});
vi.stubGlobal(
'fetch',
vi.fn(async () => new Response(body, { status: 200 })),
);

await expect(postJson(requestArgs())).rejects.toThrow(
'External context provider request did not complete.',
);
// Cleanup must still run: the reader lock is released on mid-read errors.
expect(body.locked).toBe(false);
});

it('rejects a body that is not valid UTF-8', async () => {
const cancelled = vi.fn();
vi.stubGlobal(
'fetch',
vi.fn(async () =>
// `{"a":"<invalid byte>"}`: the invalid byte sits inside otherwise-
// valid JSON, so only fatal decoding rejects it — lax decoding would
// resolve `{ a: '\uFFFD' }` and accept corrupted provider content.
streamingResponse(
[
new Uint8Array([
0x7b, 0x22, 0x61, 0x22, 0x3a, 0x22, 0xff, 0x22, 0x7d,
]),
],
cancelled,
),
),
);

await expect(postJson(requestArgs())).rejects.toThrow(
'External context provider returned an invalid response.',
);
});
});
41 changes: 35 additions & 6 deletions integrations/external-context/src/http-client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* SPDX-License-Identifier: Apache-2.0
*/

const MAX_RESPONSE_BYTES = 1024 * 1024;
export const MAX_RESPONSE_BYTES = 1024 * 1024;

class ProviderResponseError extends Error {
constructor() {
Expand Down Expand Up @@ -119,14 +119,43 @@ async function readBoundedBody(response: Response): Promise<string> {
throw new ProviderResponseError();
}

// getReader(), not `for await`: async-iterating a ReadableStream needs
// [Symbol.asyncIterator] on the TYPE, and whether it is there depends on
// which lib set the program resolves — @types/node's stream has it, the
// DOM lib's needs lib.dom.asynciterable. That resolution flipped
// underneath this file once: installing @types/jsdom at the root (#8693)
// dragged lib.dom into this program and failed the build with TS2504 on
// this exact line. The reader API types identically in every lib set, so
// the build no longer depends on that resolution.
const reader = response.body.getReader();
const chunks: Uint8Array[] = [];
let total = 0;
for await (const chunk of response.body) {
total += chunk.byteLength;
if (total > MAX_RESPONSE_BYTES) {
throw new ProviderResponseError();
let finished = false;
try {
for (;;) {
const { done, value } = await reader.read();
if (done) {
finished = true;
break;
}
if (value === undefined) {
continue;
}
total += value.byteLength;
if (total > MAX_RESPONSE_BYTES) {
throw new ProviderResponseError();
}
chunks.push(value);
}
} finally {
// Parity with `for await`, whose implicit iterator return() cancels the
// stream when the loop exits early (the oversize throw above) and is
// awaited before the error propagates — an immediate retry must not
// overlap this response's still-settling teardown.
if (!finished) {
await reader.cancel().catch(() => undefined);
}
chunks.push(chunk);
reader.releaseLock();
}

const body = new Uint8Array(total);
Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/cli.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1061,6 +1061,7 @@ describe('bootstrap import boundaries', () => {
const configSource = readFileSync('src/config/config.ts', 'utf8');
const commandNameByIdentifier = new Map([
['authCommand', 'auth'],
['auditCommand', 'audit'],
['channelCommand', 'channel'],
['extensionsCommand', 'extensions'],
['hooksCommand', 'hooks'],
Expand Down
4 changes: 4 additions & 0 deletions packages/cli/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ type BootstrapRoute = 'serve' | 'mcp' | 'help' | 'version' | 'default';

export const TOP_LEVEL_COMMANDS = [
['auth', 'Configure authentication (removed)'],
[
'audit <command>',
'Helpers used by the /audit skill (argument parsing, audit planning, brief printing, run-state captures)',
],
['channel <command>', 'Manage messaging channels (Telegram, Discord, etc.)'],
['extensions <command>', 'Manage Qwen Code extensions.'],
['hooks', 'Manage Qwen Code hooks (use /hooks in interactive mode).'],
Expand Down
Loading
Loading