diff --git a/docs/design/structured-output/structured-output.md b/docs/design/structured-output/structured-output.md new file mode 100644 index 00000000000..8938d9506c2 --- /dev/null +++ b/docs/design/structured-output/structured-output.md @@ -0,0 +1,426 @@ +# Structured Output (`--json-schema`) — Design + +This document captures the implementation decisions behind the +`--json-schema` headless feature. User-facing usage lives in +[`docs/users/features/structured-output.md`](../../users/features/structured-output.md). + +## Goal + +In headless runs (`qwen -p`, piped stdin, or positional prompt), let +the caller constrain the model's final answer to a user-supplied JSON +Schema and surface the validated payload as machine-readable output +that scripts and downstream tooling can consume directly. The model's +incidental prose during planning is allowed, but the run must +terminate with a payload that conforms to the schema, not with +free-form text. + +## Approach: synthetic tool whose parameter schema IS the user schema + +When `--json-schema` is set, `Config.createToolRegistry` registers a +synthetic `structured_output` tool +([`syntheticOutput.ts`](../../../packages/core/src/tools/syntheticOutput.ts)). +Its `parametersJsonSchema` is exactly the schema the user passed; its +`execute()` returns a stop-message `llmContent`. The tool-call +infrastructure already validates args against `parametersJsonSchema` +client-side (via Ajv in `BaseDeclarativeTool.build()`), so "the model +returned an answer conforming to the schema" reduces to "the model +successfully called `structured_output`." + +Three properties fall out of this for free: + +1. **No bespoke validator path.** Ajv-backed `validateToolParams` + already runs inside `BaseDeclarativeTool.build()` and rejects + non-conforming args before `execute()` ever fires. +2. **Standard retry behavior.** A validation failure surfaces to the + model as a tool-call error the same way any other tool's args error + does. The model sees the Ajv message and can correct in the next + turn. +3. **Provider-agnostic.** Gemini, OpenAI, and Anthropic all serialize + tool param schemas the same way (via the `DeclarativeTool` + abstraction); the synthetic tool plugs into all three. + +The tool is registered with `alwaysLoad: true` so the ToolSearch +on-demand-loading infrastructure (introduced in #3589 — keeps the +exposed tool surface small by deferring rarely-used tools behind a +search call, only mounting their full schemas when the model asks) +never hides it from the model. Without that flag, the model wouldn't +know the terminal contract exists. + +## Parse-time validation pipeline + +`resolveJsonSchemaArg(raw)` in +[`packages/cli/src/config/config.ts`](../../../packages/cli/src/config/config.ts) +runs four checks before the schema reaches `Config.createToolRegistry`: + +1. **Source resolution.** Accept either an inline JSON literal or + `@path/to/file`. The `@path` form `stat`s the resolved path first, + refuses non-regular files (FIFOs, character devices, directories), + caps size at 4 MiB, and on JSON parse failure emits a generic error + (no file-content prefix in stderr). +2. **JSON shape.** Parsed result must be a non-array object — + primitives, booleans, and arrays are rejected with a clear + message. +3. **Root accepts objects** — + [`schemaRootAcceptsObject`](../../../packages/cli/src/config/config.ts). + Function-calling APIs always pass objects as tool args; a root + schema like `{type: "array"}` would register an unusable tool. + The walk handles `type`, `const`, `enum`, `anyOf`, `oneOf`, + `allOf`, `not`, `if` / `then` / `else`, and root `$ref`. +4. **Strict Ajv compile** — + [`SchemaValidator.compileStrict`](../../../packages/core/src/utils/schemaValidator.ts). + A dedicated Ajv instance with `strictSchema: true` surfaces + typos like `propertees` that the lenient runtime validator would + silently swallow. + +### `schemaRootAcceptsObject` boundaries + +The walk is intentionally best-effort. It catches the unambiguous +"this can never accept an object" cases, and defers anything that +needs whole-schema satisfiability analysis to Ajv at runtime. + +**Decided at parse time:** + +| Pattern | Outcome | +| ------------------------------------------------------ | ----------------------------------------------------------------- | +| `type` present, doesn't include `"object"` | reject | +| `type: ["object", "null"]` etc. | accept | +| `const`: non-object value | reject | +| `enum`: no object members (incl. empty) | reject | +| `anyOf`/`oneOf`: empty array | reject | +| `anyOf`/`oneOf`: no branch admits object | reject | +| `allOf`: any branch is `false` or rejects object | reject | +| Root `$ref` (with or without sibling `type`) | reject | +| `not`: bare `{type: "object"}` (no narrowing keywords) | reject | +| `not`: `{type: "object", required: […], …}` etc. | accept (narrowing keywords leave some objects satisfiable; defer) | +| `if: true` + `then` rejects object | reject | +| `if: false` + `else` rejects object | reject | + +**Deferred to Ajv at runtime:** + +- `$ref` inside `anyOf` / `oneOf` / `allOf` branches (opaque — local + `$ref` resolution would need cycle detection, JSON Pointer escapes, + and `$defs` vs `definitions` handling; the cost outweighs the + benefit for a parse-time best-effort check). +- `if` whose value is an object schema (decidable only against a + candidate value). +- Negated `anyOf` / `oneOf` / `const` patterns more complex than + `not.type`. +- Arbitrary `pattern` ReDoS exposure (user-supplied; the threat model + is narrow because the flag is a CLI argument, not a network input). + +The `maxSessionTurns` exit path appends a `--json-schema`-specific +hint pointing users at the common stuck-run symptom (model never +called `structured_output`) and its two likely causes (tool denied +via permissions / schema unsatisfiable) so the runtime fallthrough +has user-visible diagnostics. + +## Runtime: turn dispatch + +[`packages/cli/src/nonInteractiveCli.ts`](../../../packages/cli/src/nonInteractiveCli.ts) +handles the runtime dispatch. The structured-output specifics: + +### Pre-scan + sibling suppression + +When the model emits `structured_output` alongside other tools in the +same assistant turn, the synthetic call is the terminal contract. The +pre-scan in `processToolCallBatch` filters `requestsToExecute` to +**only** `structured_output` calls, so side-effecting siblings +(`write_file`, `run_shell_command`, `edit`, …) never run. + +Example batches (when `--json-schema` is active): + +| Model emits | Behavior | +| -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `[write_file(…), structured_output(…)]` | `write_file` is skipped. `structured_output` validates, run ends. | +| `[structured_output(bad-args), structured_output(good)]` | First fails Ajv validation; second succeeds. Run ends with the second call's args. | +| `[structured_output(bad-args), write_file(…)]` | `structured_output(bad)` fails. `write_file` is also skipped (it was suppressed up front). The model sees both: Ajv's error message for the structured call, and a synthesised `"Skipped: …"` tool_result for the side-effect call. Next turn, the model may re-issue both or correct the structured call alone. | +| `[other_tool_a, other_tool_b]` (no `structured_output`) | Pre-scan is inert. Both tools run normally; the run does NOT terminate. | + +The synthesised "Skipped:" body has two variants: + +- **Success path** (a structured call captured the contract this turn): + `"Skipped: this turn's structured_output contract took precedence as +the terminal output."` — short, because the session terminates + immediately and no consumer (model or SDK) acts on it. +- **Retry path** (no structured call captured, the model gets another + turn): adds `"Re-issue this call in a separate turn if needed."` — + this is the only model-actionable case. + +### Main-turn / drain-turn parity + +`processToolCallBatch(batchRequests, setModelOverride)` is defined +inside `runNonInteractive` and called from both: + +- The main-turn loop (top of the function). +- `drainOneItem` (cron-prompt / background-task notification reply + loop). + +The drain turn matters because `structured_output` is registered for +the whole session, so a cron job or a notification reply MIGHT also +fire the tool. The helper handles both call sites identically at +invocation time; the only call-site-specific binding is which +`modelOverride` variable to write to — passed in as a setter. + +The **post-helper termination flow** differs between the two sites: +the main-turn path directly calls `return emitStructuredSuccess()`, +while the drain-turn path requires a two-hop termination +(`processToolCallBatch` captures the result into the closure-scoped +`structuredSubmission`; `drainLocalQueue` checks it to stop the drain +loop, then the holdback loop checks it to break out and call +`emitStructuredSuccess`). Both converge on the same terminal block, +but the extra indirection in the drain path is load-bearing — +without it the drain loop would continue processing queued items +after the structured result was captured. + +### Structured success terminal block + +`emitStructuredSuccess()` (also defined inside `runNonInteractive`) is +the shared "we got a valid call, shut down" path: + +1. `registry.abortAll()` aborts in-flight background agents — the + structured-output contract is single-shot and shouldn't race + `task_notification`s into the terminal emit. +2. Bounded holdback (`STRUCTURED_SHUTDOWN_HOLDBACK_MS = 500` ms) so + the natural cancel handlers of just-aborted agents have a chance + to emit their terminal `task_notification` and land it in + `localQueue`. The loop guard is + `Date.now() < deadline && registry.hasUnfinalizedTasks()`, so the + wait exits immediately when nothing is in flight (typical path) + and never blocks longer than the cap. The 500 ms ceiling is + best-effort — orphaned `task_started` events remain possible under + load if a particular agent's abort handler exceeds the budget. + The loop does **not** poll the abort signal: a SIGINT received + during holdback or during the emit path that follows will not + short-circuit the result that was already captured. Without the + holdback, stream-json consumers would routinely see `task_started` + events without matching `task_notification`. +3. `flushQueuedNotificationsToSdk(localQueue)` drains everything still + queued. +4. `finalizeOneShotMonitors()` (idempotent — safe to call twice; the + drain-turn path already invoked it). +5. `adapter.emitResult({ structuredResult: …, isError: false, … })`. + +### Failure paths + +| Cause | Exit code | Surface | +| ----------------------------------------------------------------- | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Model emits plain text only | 1 | Error with turn count + truncated `Output preview`. | +| Model never calls `structured_output` for `maxSessionTurns` turns | 53 | `Reached max session turns` + `--json-schema` hint pointing at the common stuck-run symptom and its two likely causes. | +| Validation fails repeatedly | (eventually 53 via max-turns) | Each failure surfaces to the model on the next turn with the Ajv message. | +| Abort / SIGINT | 130 | Cancellation path. A structured result is normally not emitted, but `emitStructuredSuccess()`'s holdback loop does not poll the abort signal — a SIGINT that arrives after capture but before/during the stdout emit may still flush the result. Exit code is the reliable signal. | + +## Output envelope + +The adapter pipeline in +[`BaseJsonOutputAdapter.buildResultMessage`](../../../packages/cli/src/nonInteractive/io/BaseJsonOutputAdapter.ts) +treats the presence of `structuredResult` (tracked via `'structuredResult' in options`, +not `!== undefined`, so the contract is preserved even when the model +called `structured_output` with no args under an empty schema): + +- `result` is forced to `JSON.stringify(payload)` — overriding any + free-text summary the adapter accumulated. +- A top-level `structured_result` field carries the raw object for + consumers that don't want to re-parse the stringified form. +- `undefined` payloads normalize to `null` (rendered as the literal + JSON `null` in both fields) so the field can't silently disappear. + In practice this fallback is rarely reached: upstream, `turn.ts` + applies `(fnCall.args || {})` before storing the submission, so a + zero-arg call against an empty schema lands as `{}` and renders as + `{}` on stdout, not `null`. The `?? null` step is defence-in-depth + for the strictly-undefined case. + +TEXT mode writes just the `result` field + newline to stdout (any +incidental assistant prose accumulated during the run is discarded — +not mirrored to stderr). JSON mode emits the full event log as a +JSON array; `structured_result` lives on the final `type: "result"` +element of that array, not at the document root. Stream-json mode +emits each message on its own line as JSONL; the terminating `result` +line carries `structured_result`. + +## Privacy: cross-surface redaction + +The args submitted via `structured_output` ARE the structured payload. +On the success path they're already on stdout; on validation-failure +retries they may never reach stdout at all. Either way, persisting +them on durable on-device surfaces (or exporting them off-device +through telemetry) is duplication that leaks the payload into +longer-lived storage than the user asked for. The redaction rule is +therefore "never persist any args from this synthetic tool, regardless +of outcome," not just "dedup what's already on stdout." + +Two surfaces have to redact, and both share the same placeholder +constant +[`STRUCTURED_OUTPUT_REDACTED_ARGS`](../../../packages/core/src/tools/syntheticOutput.ts): + +- `ToolCallEvent.function_args` (telemetry) — covers OTLP exports, + QwenLogger, ui-telemetry, and the chat-recording UI event mirror. +- `redactStructuredOutputArgsForRecording` (used by + `recordAssistantTurn` in `geminiChat.ts`) — covers the on-disk + chat-recording JSONL at + `~/.qwen/projects//chats/.jsonl`. + Validation-failure retries land here too — each retry's args also + get the same placeholder. + +The shared constant prevents drift between the two surfaces. Tool-call +metrics (duration, success, decision) are preserved. + +Hooks (`PreToolUse`, `PostToolUse`, `PostToolUseFailure`) are +intentionally **not** redacted — they receive the raw `tool_input` +because the hook contract is "see what the tool sees." This is +documented in the user-doc Privacy section as a "Hooks see raw args" +callout so operators can filter on `tool_name` or add hook-side +redaction before running `--json-schema` against sensitive data. + +The redaction is intentionally scoped to **on-device** persistence +surfaces (telemetry exports + chat-recording JSONL). The schema +itself still travels to the model provider on every request as the +`structured_output` function declaration's `parameters` block — no +provider-side redaction is possible, since the model needs the +schema to satisfy the tool-call contract. The user-doc Privacy +section warns users to keep `enum` / `const` / `default` / +`examples` / `description` payloads free of secrets for the same +reason. + +## Permission gating + +`structured_output` is deliberately excluded from +`PermissionManager.CORE_TOOLS` (the set of tools subject to the +`--core-tools` allowlist check) — alongside the other synthetic +tools (`agent`, `exit_plan_mode`, `ask_user_question`, `task_stop`, +`send_message`). Dynamically discovered tools (`skill`, MCP) are a +separate exclusion category that also bypasses the allowlist for +unrelated reasons. The synthetic tool only exists when `--json-schema` +is set; adding it to the allowlist machinery would mean +`--core-tools read_file --json-schema X` silently drops the terminal +contract. + +Explicit `permissions.deny` rules and `--exclude-tools` settings still +apply via `PermissionManager.evaluate` → `isToolEnabled`. Both use +the same deny mechanism and both prevent registration — the tool +declaration is stripped from the registry, so the model never sees +the tool. The typical outcome is that the model answers in plain text +(exit 1). If the model loops through other tools without producing +text, it eventually hits `maxSessionTurns` (exit 53) and the +`--json-schema` hint in `handleMaxTurnsExceededError` tells the user +where to look. + +**`--bare` interaction.** Bare mode short-circuits the settings → CLI +config bridge: `packages/cli/src/config/config.ts` builds +`mergedDeny` as `[...(bareMode ? [] : settings.permissions.deny), ...]`, +so settings-level denies (and `tools.exclude`) are dropped under +`--bare`. Argv-level `--exclude-tools` is unconditionally appended +into `mergedDeny`, so it still applies. The synthetic tool is +registered independently of all this (driven by `jsonSchema`, not by +the deny list), so a settings-only deny of `structured_output` +silently no-ops under `--bare` while the tool remains callable. + +## Subagent contexts + +`Config.createToolRegistry` accepts a `forSubAgent: true` option that +suppresses the synthetic registration. Subagent overrides reuse the +parent Config via prototype delegation (`createApprovalModeOverride` / +`buildSubagentContextOverride` → `Object.create(base)`), and +`this.jsonSchema` propagates through the prototype chain. Without the +flag, the synthetic tool would register in the subagent's registry +too, and a subagent calling it would receive the "session ends now" +llmContent — but only `runNonInteractive`'s main / drain loops detect +that as terminal, so the subagent would keep running and burn tokens +on a tool whose contract its loop can't honor. + +> **Maintainer note.** This suppression hangs on the single call path +> through `createToolRegistry(forSubAgent: true)`. Any future subagent +> spawn mechanism that bypasses this path will leak the synthetic +> tool into the subagent's registry and reintroduce the +> burn-tokens-forever failure mode. The fail-safe complement would be +> a runtime guard inside `syntheticOutput.execute()` that returns a +> `fatalError` (or no-op) when invoked from a subagent context. Land +> one if a second leak path appears. + +## MCP shadow-tool guard + +`tool-registry.ts:registerTool` checks the lazy `factories` map for +name collisions, not just the eager `tools` map. If an MCP server +discovers a tool literally named `structured_output`, the +auto-qualification path that exists for eager-tool collisions fires +for factory collisions too: the MCP tool gets renamed to +`mcp____structured_output` and the synthetic factory keeps +the bare name. Without this guard, an MCP server could silently hijack +the structured-output contract. + +## Compatibility surface + +| Combination | Status | Rationale | +| -------------------------------------------------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| `--json-schema` + `-p` (or stdin, or positional) | Supported | Primary headless path. | +| `--json-schema` + `--output-format text` (default) | Supported | `JSON.stringify(payload)` + newline. | +| `--json-schema` + `--output-format json` / `stream-json` | Supported | `structured_result` field carries the raw object. | +| `--json-schema` + `--bare` | Supported | `--bare` restricts the registry to `read_file`, `edit`, `run_shell_command`; the synthetic tool is registered alongside that minimal set. | +| `--json-schema` + `-i` | Rejected at parse time | TUI has no terminal contract for the synthetic tool. | +| `--json-schema` + `--input-format stream-json` | Rejected at parse time | Single-shot contract vs. long-lived protocol. | +| `--json-schema` + `--acp` / `--experimental-acp` | Rejected at parse time | ACP loop is independent. | +| `--json-schema` + `--prompt-interactive` | Rejected at parse time | Same as `-i`. | +| `--json-schema` + no prompt + no piped stdin | Rejected at parse time | Headless requires a prompt. | + +## Alternatives considered + +**Schema-aware response prompting (no synthetic tool).** Asking the +model to "respond with JSON matching this schema" via the system +prompt and parsing the final assistant message instead. Rejected +because the model has no syntactic guarantee — the output might be +fenced, prefixed with chatter, or hallucinate fields. Tool-call +validation is enforced by the function-calling layer before +`execute()`, which gives us a hard syntactic + semantic guard. + +**OpenAI's `response_format: {type: "json_schema", …}`.** Provider- +specific; would require parallel implementations for Gemini and +Anthropic. The synthetic-tool approach is provider-agnostic. + +**Reorder structured_output to the front of the batch instead of +filtering.** Lets side-effecting siblings run if the structured call +fails validation. Rejected because the contract for `--json-schema` is +"produce structured output" — if the model is in this mode, sibling +side-effects are probably a mistake. Suppressing them entirely is +safer; the model sees a "Skipped:" tool_result and can re-issue them +in a separate turn. + +**Local `$ref` resolution inside `schemaRootAcceptsObject`.** Would +catch schemas like `{anyOf: [{$ref: "#/$defs/String"}], $defs: {…}}` +at parse time. Rejected for now because the cost (cycle detection, +JSON Pointer syntax, `$defs` vs `definitions`, partial pointers, +remote refs) outweighs the benefit; the `maxSessionTurns` hint already +points users at "schema is unsatisfiable" as a likely cause. + +## Open work + +- Schema-aware response validation could grow a `pattern`-based + ReDoS guard if real users hit catastrophic-backtracking patterns + in `--json-schema` arguments. +- SDK protocol additions (Python / TypeScript / Java SDKs exposing a + typed `structured_result` field) — track separately; + [PR #4001](https://github.com/QwenLM/qwen-code/pull/4001) (closed + unmerged on 2026-05-11) covered that scope before the cli/core work + landed and was superseded. + +## File index + +- `packages/cli/src/config/config.ts` — `resolveJsonSchemaArg`, + `schemaRootAcceptsObject`, yargs `.check` mutex rules. +- `packages/cli/src/gemini.tsx` — TUI guard, exit-code plumbing. +- `packages/cli/src/nonInteractiveCli.ts` — + `processToolCallBatch`, `emitStructuredSuccess`, + `suppressedOutputBody`, plain-text failure path. +- `packages/cli/src/nonInteractive/io/BaseJsonOutputAdapter.ts` — + `structuredResult` → `result` + `structured_result` envelope. +- `packages/core/src/config/config.ts` — registration with + `registerStructuredOutputIfRequested`, `forSubAgent` skip. +- `packages/core/src/tools/syntheticOutput.ts` — synthetic tool + + `STRUCTURED_OUTPUT_REDACTED_ARGS` placeholder. +- `packages/core/src/tools/tool-registry.ts` — factory-collision + rename for MCP shadow tools. +- `packages/core/src/telemetry/types.ts` — `function_args` redaction. +- `packages/core/src/core/geminiChat.ts` — + `redactStructuredOutputArgsForRecording`. +- `packages/core/src/utils/schemaValidator.ts` — `compileStrict` + with strict Ajv instance. +- `packages/cli/src/utils/errors.ts` — + `handleMaxTurnsExceededError`'s `--json-schema` hint. diff --git a/docs/users/features/_meta.ts b/docs/users/features/_meta.ts index 7f76ae1526d..4231d934865 100644 --- a/docs/users/features/_meta.ts +++ b/docs/users/features/_meta.ts @@ -9,6 +9,7 @@ export default { skills: 'Skills', memory: 'Memory', headless: 'Headless Mode', + 'structured-output': 'Structured Output', 'dual-output': 'Dual Output', checkpointing: { display: 'hidden', diff --git a/docs/users/features/structured-output.md b/docs/users/features/structured-output.md new file mode 100644 index 00000000000..78c7e30eee1 --- /dev/null +++ b/docs/users/features/structured-output.md @@ -0,0 +1,309 @@ +# Structured Output (`--json-schema`) + +Constrain the model's final answer to a JSON Schema you supply. Qwen +Code registers a synthetic terminal tool the model is required to call, +parses the call's arguments against your schema, and exposes the +validated payload on stdout (or in the JSON / stream-json result +envelope). The first valid call ends the run. + +Headless only — works with `qwen -p`, a positional prompt, or a prompt +piped via stdin. + +## Quick start + +```bash +qwen --prompt "Summarize the changes in HEAD with risk_level" \ + --json-schema '{ + "type": "object", + "properties": { + "summary": { "type": "string" }, + "risk_level": { "type": "string", "enum": ["low", "medium", "high"] } + }, + "required": ["summary", "risk_level"], + "additionalProperties": false + }' +``` + +Output on stdout (default `--output-format text`): + +```json +{ "summary": "…", "risk_level": "low" } +``` + +The line is exactly the JSON-stringified payload + newline — no +envelope, no event log. Pipe it straight into `jq` or another consumer. + +In **text** mode, stdout is reserved for the JSON payload on success +and is empty on failure; error messages and log lines go to stderr. +That makes `$(qwen --json-schema …) || exit 1` capture patterns safe +under text mode — failures land in stderr, not mixed into the captured +variable. The model's incidental prose during planning is **not** +mirrored to stderr either — text mode discards it; reach for +`--output-format json` or `stream-json` if you need to see it. + +In `--output-format json` and `stream-json`, the failure result +message is emitted on **stdout** alongside the success path (as the +final element of the JSON array, or the terminating `result` line on +the JSONL stream). Not all failure modes emit a result to stdout — +max-session-turns (exit 53) and signal interrupts (exit 130) exit with +stderr output only. Check the exit code first; `is_error` on the +result object disambiguates within the subset of failures that do +produce a result event. + +> **Empty schema:** Passing `{}` produces `{}` (an empty JSON object) +> on stdout. The model calls `structured_output` with no arguments; +> the upstream argument-normalisation path turns the empty function +> call into an empty-object payload, which passes validation against +> the empty schema and is emitted verbatim. + +## Supplying the schema + +Two equivalent forms: + +```bash +# Inline JSON literal +qwen -p "…" --json-schema '{"type":"object", "properties":{…}}' + +# Read from a file +qwen -p "…" --json-schema @./schemas/summary.json +``` + +The `@path` form expands `~`, normalizes the path, and reads the file +with `utf8` encoding. + +> **Latency note:** Successful runs incur a shutdown holdback **capped +> at ~500 ms** while in-flight background agents flush their final +> notifications before the result is emitted. The holdback exits early +> if no background tasks are pending, so simple runs barely notice it; +> batch pipelines that fan out hundreds of `--json-schema` invocations +> against busy agents should account for this upper bound. + +> **Security note:** Schemas may contain user-supplied regular +> expressions in `pattern` keywords. Ajv compiles these with the +> ECMAScript regex engine, which is vulnerable to catastrophic +> backtracking. Because tool arguments are always objects, the +> `pattern` keyword only fires inside string properties — a malicious +> schema like +> `{"type":"object","properties":{"value":{"type":"string","pattern":"(a+)+b"}}}` +> can hang the CLI when the model supplies a moderately long +> matching value. Only run `--json-schema` with schemas from sources +> you trust. + +Validation at parse time: + +- The file must be a regular file (no FIFOs, character devices, or + directories). +- File size is capped at 4 MiB. Real-world JSON schemas are well under + this; multi-MiB files almost always indicate a wrong-path mistake. +- The schema must be valid JSON. For `@path` input, the parse error is + generic ("content of `` is not valid JSON") rather than echoing + the SyntaxError detail, so a wrapping process that surfaces stderr + can't read a prefix of the file's contents back from the error. +- The schema must compile under the strict Ajv configuration — + typos like `propertees` are surfaced, but spec-valid patterns + (e.g. `required` without listing every key in `properties`) are + accepted. +- The schema root must accept object-typed values. Function-calling + APIs (Gemini, OpenAI, Anthropic) all require tool arguments to be + JSON objects, so a non-object root would register an unusable tool. + +The root-acceptance check walks `type`, `const`, `enum`, `anyOf`, +`oneOf`, `allOf`, `not`, and `if`/`then`/`else` (best-effort for the +decidable cases). When in doubt it defers to Ajv at runtime. + +> **Root `$ref` is rejected** by the parse-time check. If your schema +> reuses a definition via `$ref`, wrap it in `allOf`: +> +> ```jsonc +> // Rejected: +> { "$ref": "#/$defs/MyObj", "$defs": { "MyObj": { "type": "object", "properties": { "name": { "type": "string" } } } } } +> +> // Accepted (root accepts objects via the allOf branch): +> { "allOf": [{ "$ref": "#/$defs/MyObj" }], "$defs": { "MyObj": { "type": "object", "properties": { "name": { "type": "string" } } } } } +> ``` +> +> `$ref` inside `anyOf` / `oneOf` / `allOf` is deferred to Ajv at +> runtime, so the wrapped form passes the root-acceptance check. + +## Output shape per format + +| `--output-format` | What goes to stdout | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `text` (default) | `JSON.stringify(payload) + "\n"` — one line, the validated object. | +| `json` | A single JSON **array** of message objects (the full event log). The final element is the `type: "result"` message, which carries both `result` (`JSON.stringify(payload)`) and `structured_result` (the raw object). | +| `stream-json` | Each event on its own line as JSONL. The terminating `result` line carries `result` (stringified) and `structured_result` (raw object). | + +In both JSON formats, prefer reading `structured_result` over `result` +when you want the object; `result` is the stringified form provided for +consumers that always expect a string in that field. For `--output-format +json`, read the last element of the array and pull `structured_result` +from there (e.g. `jq '.[-1].structured_result'`); for `stream-json`, +read the final `type: "result"` line on the stream. + +## Restrictions + +| Combination | Behavior | +| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--json-schema` + `-i` / `--prompt-interactive` | Rejected at parse time. The synthetic tool's "session ends now" message has no terminator in the TUI loop. | +| `--json-schema` + `--input-format stream-json` | Rejected at parse time. The single-shot terminal contract is incompatible with the long-lived stream-json input protocol. | +| `--json-schema` + `--acp` / `--experimental-acp` | Rejected at parse time. ACP runs its own turn loop that doesn't honor the synthetic-tool terminal contract. | +| `--json-schema` with no prompt and no piped stdin | Rejected at parse time. Headless mode needs a prompt — pass `-p`, a positional argument, or pipe one in. | +| `--bare` + `--json-schema` | Supported. The synthetic tool is registered alongside the bare three (`read_file`, `edit`, `run_shell_command`). | +| `--json-schema` inside a subagent | Tool is NOT registered. Only the main / drain turns of the top-level run honor the terminal contract; a subagent calling the tool would receive "session ends now" and then keep running because its loop has no terminator. | + +## Retry and failure modes + +> **Cost note.** Two things multiply token spend in a `--json-schema` +> run, both worth designing for: +> +> - **Schema embedded in every turn.** The schema ships as the +> `structured_output` function declaration's `parameters` block on +> every model request, not just the first. Large schemas (up to the +> 4 MiB parse cap) proportionally increase per-turn input tokens +> for the entire run. +> - **Each validation retry is a full model turn.** A schema the +> model misses repeatedly is multiplied per failure (request + +> inference + response). Keep schemas constrained enough to guide +> the model and simple enough to nail on the first try; raise +> `--max-session-turns` when retries are expected. + +The session ends on the first valid call. Until then: + +- **Args fail validation.** `structured_output` returns a tool-result + error with Ajv's message, the model sees it on the next turn, and + may correct the arguments and call again. +- **Model calls a side-effecting tool in the same turn as + `structured_output`.** The pre-scan suppresses the sibling — it + never runs, regardless of whether the structured call ultimately + validates. The two paths split on what the model sees next: + - **Validation succeeds:** the run ends immediately, and the model + never gets another turn — the suppressed sibling is silently + discarded. + - **Validation fails:** the model gets another turn and sees a + synthesised "Skipped:" `tool_result` for the suppressed call, + so it can re-issue that call in a **separate turn** (one that + does not include `structured_output`). +- **Model emits plain text instead of calling + `structured_output`.** Exit code `1`. The error message includes + the turn count and a truncated preview of the model's output so + you can see what it actually said. +- **Run reaches `maxSessionTurns`.** Exit code `53`. Standard + "Reached max session turns" exit, plus a `--json-schema`-specific + hint that points at the three common stuck-run causes: model never + called the tool, `structured_output` is denied by permission rules, + or the schema is unsatisfiable. +- **Run is interrupted (SIGINT / Ctrl-C).** Exit code `130`. The + structured result is normally not emitted, but the shutdown + holdback loop does not poll the abort signal, so a SIGINT that + arrives after a successful call has been captured but before the + result reaches stdout may still land on stdout. Treat the exit + code as the source of truth. + +## Privacy + +The args you submit through `structured_output` ARE the structured +payload — already emitted on stdout. To avoid persisting the same +payload a second time into on-device surfaces that may be exported off +the machine, args are redacted with the placeholder +`{ __redacted: 'structured_output payload (see stdout result)' }` on: + +- The `ToolCallEvent` telemetry path (OTLP exports, QwenLogger, + ui-telemetry stream, chat-recording UI event mirror). +- The on-disk chat-recording JSONL at + `~/.qwen/projects//chats/.jsonl` (re-fed + into model context on `--continue` / `--resume`), including every + validation-failure retry. + +Tool-call metrics (duration, success, decision) and surrounding event +metadata are preserved. + +> **Schema is sent to the model provider.** Redaction covers the +> _call arguments_ on local surfaces only. The schema itself rides +> on every model request as the `structured_output` function +> declaration's `parameters` block — so any literal values you put +> inside it (`enum`, `const`, `default`, `examples`, `description`, +> `$comment`, etc.) reach the provider in cleartext just like prompt +> text. Schemas should describe shape and constraints; treat them as +> public toward the provider and keep secrets, customer records, and +> other sensitive payloads out of the schema body. + +> **Hooks see raw args.** The redaction described above only applies +> to telemetry and chat-recording. `PreToolUse`, `PostToolUse`, and +> `PostToolUseFailure` hooks (including HTTP hooks that can forward +> payloads off-device) receive the unredacted `tool_input` for +> `structured_output`, since the hook contract is "see what the tool +> sees." If you operate audit-style catch-all hooks, either disable +> them for `structured_output` (filter on `tool_name`) or add +> hook-side redaction before running `--json-schema` against +> sensitive data. + +## Session resumption (`--continue` / `--resume`) + +`--json-schema` is a per-run flag, not a per-session property. The +synthetic tool is registered when the CLI parses its arguments, so: + +- Re-pass `--json-schema` on every `--continue` / `--resume` you want + the terminal contract to apply to. The same schema as the original + run is the safe default — a mid-session schema swap is allowed but + changes the contract the model is being held to. +- If you `--continue` without `--json-schema`, the resumed run is an + ordinary headless session: `structured_output` simply doesn't + exist as a tool, and the model will respond in free-form text. +- The `__redacted` placeholder in the resumed chat-recording does + not affect resumability in practice. A successful `structured_output` + call terminates the session immediately, so the only redacted args + a resumed run could see are from failed attempts. The model still + has each attempt's Ajv validation error in the recorded `tool_result` + and the live parameter schema (re-registered from `--json-schema`), + which is enough to retry. + +## Permission gating + +`structured_output` deliberately bypasses the `--core-tools` allowlist: +the tool only exists when `--json-schema` is set, so excluding it +would leave the run with no terminal contract. + +Explicit `permissions.deny` rules and `--exclude-tools` settings DO +take effect — both use the same deny mechanism and both prevent +`structured_output` from being registered, so the model never sees +the tool declaration. The typical result is that the model answers in +plain text (exit 1). If the model loops through other tools without +ever producing text, it will eventually hit `maxSessionTurns` +(exit 53) and the `--json-schema` hint in the error message tells you +where to look. + +> **`--bare` caveat.** Bare mode ignores most settings-derived inputs, +> including settings-level `permissions.deny` and `tools.exclude`. The +> synthetic tool stays registered, so a settings-only deny of +> `structured_output` will silently no-op under `--bare`. Argv-level +> `--exclude-tools structured_output` still applies in bare mode — use +> the flag rather than settings if you need to lock down a bare run. + +## Conflict with MCP tools + +If an MCP server registers a tool literally named `structured_output`, +the tool-registry collision check renames the MCP tool to +`mcp____structured_output` so the synthetic tool keeps +the bare name. The user-supplied schema is always the one the model +sees. + +## Example: gating a multi-step run on the structured output + +```bash +RESULT=$(qwen --prompt "Audit this diff and rate its risk." \ + --json-schema @./schemas/audit.json) || exit 1 + +risk=$(jq -r '.risk_level' <<<"$RESULT") +if [ "$risk" = "high" ]; then + echo "High-risk diff; pausing pipeline." >&2 + exit 2 +fi +``` + +## See also + +- [Headless Mode](headless.md) — the `-p`-based flow `--json-schema` + builds on. +- [Dual Output](dual-output.md) — records a JSON-event sidecar + alongside the TUI (a different approach to machine-readable output; + does not require `--json-schema`).