feat(symphony): the constrained format for a forced or required tool choice

[slin1237]

Draft: waits for its fixtures, bellwether #19 (ground rules 1 and 26). The design question is answered: the maintainer decided on 2026-10-06 that tool_choice is part of Symphony (bellwether #21, closed), with this parser as the fallback for outputs constrained by a JSON schema.

Description

Problem

When a request forces a function (tool_choice: {"type": "function", ...}) or requires a call (tool_choice: "required", or an allowed-tools list in required mode), the gateway constrains the engine with a JSON schema, unless the model's tool parser has a structural tag, in which case the output keeps the model's own markers, and the output has no markers at all: the whole output is the forced function's arguments, or a JSON array of {"name", "parameters"} objects. Today no parser handles that shape. The gateway parses the whole output once it has ended (parse_json_schema_response in utils/chat_utils.rs) and, when streaming a forced function, sends the whole output as one call's arguments without looking at it (regular/streaming.rs). Symphony needs a Parser for this shape before the driver can switch, and it is the one shape where the format is known from the request rather than from the model.

Solution

formats::Constrained, a Parser built from Choice::Function(name) or Choice::Required. It needs no scanner, since it knows the shape, and it streams the arguments as the model's own bytes, as every format does.

One function: whitespace before the value is Dropped { Whitespace }; the call starts at the value's first byte, named by the request, its source empty since no output byte names it; the argument fragments are the output's bytes for as long as they keep the arguments a valid JSON prefix (checked with the ported prefix parser, exactly as the assembler does), and from the first byte that does not they come back as Malformed { InvalidArguments }. Once the value is whole, the whitespace after it is dropped and everything from the first other byte is Malformed { Other }. The call ends when the output ends, whole or not, so a value cut short is a call with what arrived.

Required: the list's syntax is checked as JSON has it (a call or the end after the opening bracket, a comma or the end after a call, a call after a comma); brackets and commas in their places are Dropped { Wrapper }, whitespace between them Dropped { Whitespace }, and each object goes to the json::Assembler, which emits the call's events and says where the object ended; the assembler already accepts parameters as the arguments key. An output that is not a list, bytes the syntax does not allow where they stand (a missing comma, a leading or trailing one, two in a row), and bytes after the list are Malformed { Other } from the first such byte to the end, with a reason that names what was expected there. A list cut short ends with what arrived.

Every byte lands in one event, every text event carries its token count from the Ledger, ids are call_<index>, the engine's finish reason is kept with tool_calls counting the calls that started, and reasoning cannot occur. The finish-reason mapping both formats need moves to formats::finish_reason, and Qwen3 uses it.

The contract tests now run over every format with its own corpus: FORMATS lists a constructor and outputs per format, and the five properties plus the lifecycle check iterate over all of them. The two new corpora (twelve and fourteen outputs) hold the grammar's shapes and what a grammar never produces, since the parser must survive it anyway. The corpora avoid junk that begins like a JSON literal (nope, not json): where InvalidArguments begins for a literal's prefix depends on the chunking, as Alex noted on #2792, because the prefix parser is as lenient as the old crate's, so those inputs cannot be chunking-invariant for any format built on it.

Stacked on #2799 (base branch feat/symphony-token-attribution); it retargets when that merges.

Changes

• crates/symphony/src/formats/constrained.rs: Constrained, Choice, the module doc, ten tests.
• crates/symphony/src/formats/mod.rs: the module, finish_reason; crates/symphony/src/formats/qwen3.rs: uses it (the tests import FinishReason themselves now).
• crates/symphony/src/lib.rs: exports Choice and Constrained.
• crates/symphony/tests/contract.rs: Format, FORMATS, corpus(), the two corpora; replay and replay_counted take the format's constructor; every property and the lifecycle check run per format.

Nothing outside crates/symphony/ changes; Cargo.lock is unchanged.

Test Plan

Run on the head, exit codes checked:

• cargo +nightly fmt -p smg-symphony: 0, no change.
• cargo +1.98.0 clippy -p smg-symphony --all-targets -- -D warnings: 0.
• cargo +1.98.0 test -p smg-symphony: 0, 126 unit tests, two fixture tests on their skip path and 7 contract tests. The new unit tests: a forced function takes the whole output as its arguments; whitespace around the value is dropped and the call starts at the value; a value cut short is a call with what arrived; bytes that break the prefix are InvalidArguments and bytes after a whole value are Other after the dropped space; an empty or blank output makes no call; a required list streams each call and drops the list syntax (event by event, with the object cut across two deltas); an output that is not a list, bytes after the list, and a list of non-objects are malformed from the first such byte; a comma comes between calls and nowhere else ([{…}{…}], [,{…}], [{…},] and [,,], each malformed from the byte the syntax refuses); a list cut short ends with what arrived, and a nameless object makes no call; the engine's finish reason is kept. The contract: conservation, chunking invariance, well-formedness, order and token identity hold for both constrained shapes under every chunking of their corpora, and the lifecycle check passes for all three formats. One defect found by the contract and fixed before this head: after a whole value, each delta re-trimmed its own leading whitespace, so a space inside trailing junk was dropped in one chunking and malformed in another; junk is sticky now.
• RUSTDOCFLAGS="-D warnings" cargo +1.98.0 doc -p smg-symphony --no-deps: 0.
• git diff --stat feat/symphony-token-attribution...HEAD: the five files above.

The shapes were taken from parse_json_schema_response and the forced-function streaming branch in the gateway; the Symphony pieces from json::Assembler, json::PartialJson, json::is_complete and tokens::Ledger.

Checklist

• cargo +nightly fmt passes
• cargo clippy --all-targets -- -D warnings passes for smg-symphony (the workspace-wide run with --all-features needs OpenCV for llm-multimodal on this machine; CI runs it)
• Documentation updated (module docs, crate exports)
• (Optional) Please join us on Slack #sig-smg to discuss, review, and merge PRs

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[hello-alexmcc]

Review on behalf of the maintainers (rule 17 checklist, rules 0, 19 to 24 and 26), at head ff44c4b: one commit, feat(symphony): the constrained format for a forced or required tool choice, stacked on #2799's 36bc7b1.

Scope, commit, gates. formats/constrained.rs (new), formats/mod.rs (the module and the shared finish_reason), formats/qwen3.rs (its private match replaced by the shared function, its tests importing FinishReason themselves), lib.rs (one doc line), tests/contract.rs (every property and the lifecycle check over FORMATS, each with its own corpus); Cargo.lock unchanged; event.rs, input.rs and parser.rs untouched. Conventional subject, the author's Signed-off-by, Co-authored-by: Chang Su, no AI trailer. PR body per the template. Gates run here on ff44c4b in a clean worktree, exit codes captured: cargo +nightly fmt --all -- --check 0; cargo +1.98.0 clippy -p smg-symphony --all-targets -- -D warnings 0; cargo +1.98.0 test -p smg-symphony 0 (115 unit tests, six contract tests over the three formats); RUSTDOCFLAGS=-D warnings cargo +1.98.0 doc -p smg-symphony --no-deps 0; codespell 2.4.1 over crates/symphony 0.

Reading. Single accounts for the output's bytes in one place: a fragment while the prefix parser takes them whole, whole_end once the parsed prefix is a complete value (is_complete on it), InvalidArguments from the first byte the parser leaves, and after a whole value the whitespace dropped and the first byte of substance making the rest malformed for good (junk), so no chunking can move the boundary; the start comes at the value's first byte with an empty source and the end at the output's end whether or not the value is whole, and a blank output makes no call. List is a four-phase walk (before, between, object, after, plus broken) that drops the brackets and commas as wrapper and the JSON whitespace as whitespace, hands each { to an Assembler indexed by the calls that started, takes the next free index only when the assembler started a call, and turns anything out of shape into Malformed from that byte to the end; the loop always advances, since the assembler takes everything before its close. Both emit in byte order and every event goes through the ledger, so token identity holds here as it does for Qwen3. The reference behaviour (parse_json_schema_response: the whole output as the forced function's arguments; a list of {name, parameters} for required) is what the oracle below encodes.

Harness. A scratch program outside the repository: 40 outputs (23 for a forced function, 17 for required), each replayed whole, at every two-way cut, character by character and at ten seeded plans: 1539 replays. Checked on every replay: every byte in exactly one event in order with Finish last; no content or reasoning event; starts before fragments before ends, indices from zero in order with ids following, one end per start, tool_calls equal to the starts; and that the fragments, the dropped and malformed text by reason, the calls and the ends do not depend on the cuts. Oracles: for a forced function, any output that is a whole JSON value with whitespace around it gives one call named by the request whose fragments are exactly the value and nothing malformed (objects with nested escapes, Japanese and emoji, the surrogate-pair escape, an array, a string, a number, true, {}); a blank output no call and tool_calls: 0; for required, any output that is a whole list gives the calls serde_json reads from it, by name and the parameters or arguments bytes, in order, a nameless object skipped and the ids running call_0, call_1 across it. The malformed outputs behave as the doc says: a value cut short or cut inside a string ends with what arrived; junk after a whole value is Other("bytes after the arguments") with the whitespace before it dropped and the whitespace inside it kept malformed; a second value after the first is junk; 12abc is the whole 12 and junk abc; hello world is InvalidArguments from its first byte; not a list, junk where a comma should be, bytes after the list, a list cut short before or inside an object, [{"na as the assembler's UnterminatedRegion, all from the first offending byte to the end. 0 failures.

One observation, no change asked. The one output whose events depend on the cuts is truex: fed whole it is InvalidArguments from the first byte, fed true then x it is the fragment true and InvalidArguments for x, and every cut in between moves the boundary (12 of the 1539 replays). That is the prefix parser's literal leniency the #2792 review observed and the stricter acceptor listed in STATE.md would end; it is why the contract corpora avoid junk that begins like a literal, and here it is reachable only when the grammar is violated. A related asymmetry worth a sentence somewhere: the same bytes classify differently by position, 12abc as a whole value plus junk here (Other) and as InvalidArguments inside an assembler's object, each as its module documents.

Approved. Rule 25 at merge time: stacked on #2799, so it retargets as that lands and merges with every check green on the head, the squash body carrying the sign-off and the Chang Su trailer.

[slin1237]

Converted to draft. Ground rule 26 says a format PR is opened only after bellwether's fixtures for that format are merged, and none exist for outputs constrained by tool_choice; this one was opened too early. It also adds a format chosen by the request and a public type (Choice) that design.md does not describe, which rules 7 and 18 leave to the maintainer, and its hand-written impl Parser needs the escape-hatch line in design.md section 4. Fixtures are requested as bellwether #19, the design question is bellwether #21 (needs:decision). The code and Alex's review stand until both are answered.

[slin1237]

The design question behind this draft is answered. The maintainer, 2026-10-06, on bellwether #21: "makes sense, this should be part of symphony".

What that settles: generating the tool_choice constraint belongs to Symphony (the format's own structural tag, the main path), and Constrained is the parser for the fallback, the bare JSON an engine writes under a JSON-schema constraint. The driver picks it when it sent a JSON schema; it is not a row of the per-model table.

This stays a draft until its fixtures are on bellwether main (#19). They can be built from the BFCL import and need no model, since the request decides the shape.