From 9153c536dda79b091213722d34bd8cb5be670527 Mon Sep 17 00:00:00 2001 From: Simo Lin <25425177+slin1237@users.noreply.github.com> Date: Thu, 8 Oct 2026 00:54:48 -0700 Subject: [PATCH 1/4] feat(symphony): Kimi K3's structural tag, derived from the table's regions and the old builder's argument shapes The grammar derivation now reads the table as regions: a region is opened by the terminal of a row into its state and closed by the terminal of a row back to a state it is entered from, and the terminal every state has a row for is the end of the turn, which closes no region. The call, the block of calls and the thought all take their markers that way, so a table with two wrapper states, as Kimi K3's (the turn itself and the block), derives the right ones whatever the order of its rows. Kimi K3's tag comes from that: one way into the block per state it is entered from (the turn itself, the thought, the answer), each that state's own close before the block's opener, since the prompt leaves the thought or the answer open and the model must close it on its way to the calls; the opener and the closes as triggers; inside, one or more calls, each the call's opener with the tool's name and index, one argument tag per property of the schema with its type and value pinned, and the call's close. The argument shapes (string as text to a marker, string enum as its values, number, object and array as the property's schema with the tool's definitions attached, boolean and null, anything for an untyped property or a dangling pointer, any arguments for a schema without properties) are the old Kimi K3 parser's, ported to the Grammar tree and pinned to it in the tests. Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com> Co-authored-by: Chang Su <8605658+CatherineSue@users.noreply.github.com> Co-authored-by: Keyang Ru Co-authored-by: LightSeek Foundation <243258330+lightseek-bot@users.noreply.github.com> --- crates/symphony/src/format.rs | 5 + crates/symphony/src/grammar.rs | 223 ++++++++--- crates/symphony/src/grammar/xtml.rs | 555 ++++++++++++++++++++++++++++ 3 files changed, 738 insertions(+), 45 deletions(-) create mode 100644 crates/symphony/src/grammar/xtml.rs diff --git a/crates/symphony/src/format.rs b/crates/symphony/src/format.rs index f55e5c66e8..968bb79623 100644 --- a/crates/symphony/src/format.rs +++ b/crates/symphony/src/format.rs @@ -247,6 +247,11 @@ impl Format { self.states[index].emits } + /// How many states the table has; the first is the one the turn opener leaves the engine in. + pub(crate) fn states(&self) -> usize { + self.states.len() + } + /// Where terminal `on` takes the engine from state `from`, if the table says. pub(crate) fn next(&self, from: usize, on: usize) -> Option { self.transitions diff --git a/crates/symphony/src/grammar.rs b/crates/symphony/src/grammar.rs index 3f99f7a0d1..4eb629b0f8 100644 --- a/crates/symphony/src/grammar.rs +++ b/crates/symphony/src/grammar.rs @@ -8,28 +8,36 @@ //! compose; `const_string`, `any_text`, `regex` and `json_schema` as leaves. [`Grammar::to_json`] //! writes it as the engine reads it, and [`Grammar::payload`] wraps it as the request carries it. //! -//! [`Format::grammar`] derives the tree from the table. The call markers are the terminals that -//! enter and leave the arguments state, the block markers the ones around a wrapper state, the -//! thought's close the terminal that leaves the reasoning state, and the request's tools give each -//! call its name and schema. What the syntax inside the markers constrains comes from the call -//! syntax: a JSON object around each tool's schema for the JSON family; the engines' `glm_xml` -//! style of a schema for keyed arguments in GLM's spelling; a tag grammar per parameter for DSML. -//! The gateway's old parsers wrote these tags by hand, one function per parser; where one exists -//! for a format Symphony has, the derived tag is pinned to it in the tests, so the switch changes -//! nothing an engine sees. +//! [`Format::grammar`] derives the tree from the table. A region of the table (a call, a block of +//! calls, the thought, the answer) is opened by the terminal of a row into its state and closed +//! by the terminal of a row back to a state it is entered from; the terminal every state has a +//! row for is the end of the turn, which ends whatever is open and closes no region of its own. +//! So the call markers are the rows into and out of the arguments state, the block markers the +//! rows into and out of the wrapper state the calls sit in, and the thought's close the row from +//! the reasoning state back to where the thought opened; the request's tools give each call its +//! name and schema. What the syntax inside the markers constrains comes from the call syntax: a +//! JSON object around each tool's schema for the JSON family; the engines' `glm_xml` style of a +//! schema for keyed arguments in the compact spelling; a tag grammar per parameter for DSML; for +//! Kimi K3's XTML, one argument tag per property of the schema (the `xtml` module). The gateway's +//! old parsers wrote these tags by hand, one function per parser; where one exists for a format +//! Symphony has, the derived tag is pinned to it in the tests, so the switch changes nothing an +//! engine sees. //! //! With the prompt's reasoning still open, the tag is wrapped in a prefix: free text that can //! write none of the table's markers nor the call syntax's inner tags, closed by the thought's //! close, so the forced call follows the reasoning instead of replacing it, as the gateway's //! `wrap_in_reasoning_prefix` does. A table without a thought has nothing to wrap, and the tag -//! comes back as it is; a thought that closes somewhere other than content has no prefix yet, and -//! the table gives `None` rather than a tag the model could write inside its thought. +//! comes back as it is; a thought that does not close back to where it opened has no prefix yet, +//! and the table gives `None` rather than a tag the model could write inside its thought. Kimi +//! K3's tag takes no prefix: the block of calls can be entered from the turn itself, from the +//! thought or from the answer, and the tag has one way in per state, that state's own close +//! followed by the block's opener, so the model closes whatever the prompt left open on its way +//! to the calls, whichever that was. //! //! Not derived yet, so [`Format::grammar`] gives `None` for them: the tagged syntax (Qwen 3.5 and //! later, Seed-OSS), keyed arguments in another spelling than the compact one (Ling's newlines, -//! Hy4's suffixed tags), MiniMax M3's XML tree, Kimi K3's XTML, the pythonic tables, xLAM's list, -//! and a table without calls. The gateway then takes the JSON-schema path, as it does for those -//! models today. +//! Hy4's suffixed tags), MiniMax M3's XML tree, the pythonic tables, xLAM's list, and a table +//! without calls. The gateway then takes the JSON-schema path, as it does for those models today. use openai_protocol::common::Tool; use serde_json::{json, Value}; @@ -39,6 +47,8 @@ use crate::{ tagged::{dsml, keyed}, }; +mod xtml; + /// A structural-tag grammar, as the engines take it. #[derive(Clone, Debug, PartialEq)] pub enum Grammar { @@ -162,14 +172,23 @@ impl Grammar { } } -/// The markers a table's rows give a call: the terminals that enter and leave the arguments state, -/// and the ones around a wrapper state when the calls sit in a block. +/// The markers a table's rows give a call: the terminals into and out of the arguments state, and +/// the block's own when the calls sit in a block. struct CallMarkers<'a> { - block: Option<(&'a str, &'a str)>, + block: Option>, call_open: &'a str, call_close: &'a str, } +/// A block of calls: the terminals into and out of its wrapper state, and what the model writes +/// before the opener on each way in, one per state the block is entered from, in the order of the +/// states: nothing from the turn's own state, the state's own close from a thought or an answer. +struct Block<'a> { + open: &'a str, + close: &'a str, + before_open: Vec<&'a str>, +} + impl Format { /// The structural tag for a forced or required call to one of `tools`, or `None` when the /// table's call syntax has no derivation yet, the table has no calls, or no tool has a name. @@ -177,7 +196,8 @@ impl Format { /// an allowed-tools list in `required` mode). With `reasoning_open`, the prompt left the model /// inside its thought, and the tag follows a reasoning prefix the thought's close ends; a /// table without a thought has no prefix, and the tag comes back as it is; a table whose - /// thought closes somewhere other than content has no prefix yet, and gives `None`. + /// thought does not close back to where it opened has no prefix yet, and gives `None`. Kimi + /// K3's tag closes the thought on its own way in, and is the same either way. pub fn grammar( &self, tools: &[Tool], @@ -198,20 +218,20 @@ impl Format { Self::keyed_calls(&markers, &named, at_least_one) } CallSyntax::Dsml => Self::dsml_calls(&markers, &named, at_least_one)?, + CallSyntax::Xtml => return xtml::calls(&markers, &named, at_least_one), CallSyntax::Keyed(_) | CallSyntax::Tagged | CallSyntax::Pythonic | CallSyntax::JsonList - | CallSyntax::Xml - | CallSyntax::Xtml => return None, + | CallSyntax::Xml => return None, }; if !reasoning_open { return Some(calls); } - if !self.has_state(Emits::Reasoning) { + let Some(reasoning) = self.state_emitting(Emits::Reasoning) else { return Some(calls); - } - let think_close = self.marker(Emits::Reasoning, Emits::Content)?; + }; + let think_close = self.close_of(reasoning, |_| false)?; let mut excludes: Vec = self.terminal_texts().map(str::to_string).collect(); let inner: &[&str] = match self.call_syntax() { Some(CallSyntax::Keyed(tags)) => &[ @@ -287,7 +307,8 @@ impl Format { tools: &[&Tool], at_least_one: bool, ) -> Option { - let (block_open, block_close) = markers.block?; + let block = markers.block.as_ref()?; + let (block_open, block_close) = (block.open, block.close); let parameter_close = dsml::PARAMETER_CLOSE; let parameter = Grammar::Tag(Tag::new( dsml::PARAMETER_OPEN, @@ -337,38 +358,93 @@ impl Format { ])) } - /// The call markers the rows give: the terminal from the wrapper state, or without one from - /// the content state, into the arguments state, and the one back; and, when the calls sit in - /// a block, the terminals from the content state into the wrapper state and back. + /// The call markers the rows give: the opener and the close of the arguments state, and of + /// the wrapper state with a row into it when the calls sit in a block. fn call_markers(&self) -> Option> { - let call_open = self - .marker(Emits::Wrapper, Emits::Arguments) - .or_else(|| self.marker(Emits::Content, Emits::Arguments))?; - let call_close = self - .marker(Emits::Arguments, Emits::Wrapper) - .or_else(|| self.marker(Emits::Arguments, Emits::Content))?; + let arguments = self.state_emitting(Emits::Arguments)?; + let is_arguments = |state: usize| self.emits(state) == Emits::Arguments; let block = self - .marker(Emits::Content, Emits::Wrapper) - .zip(self.marker(Emits::Wrapper, Emits::Content)); + .transitions() + .find(|&(source, _, target)| { + target == arguments && self.emits(source) == Emits::Wrapper + }) + .map(|(source, _, _)| source); + let block = match block { + Some(state) => Some(Block { + open: self.opener_of(state, is_arguments)?, + close: self.close_of(state, is_arguments)?, + before_open: self + .entered_from(state, is_arguments) + .into_iter() + .map(|from| self.close_of_region(from).unwrap_or("")) + .collect(), + }), + None => None, + }; Some(CallMarkers { block, - call_open, - call_close, + call_open: self.opener_of(arguments, |_| false)?, + call_close: self.close_of(arguments, |_| false)?, }) } - /// The text of the terminal that moves the engine from a state emitting `from` to one - /// emitting `to`: the two states name the marker, not the order of the rows. - fn marker(&self, from: Emits, to: Emits) -> Option<&str> { + /// The first state emitting `emits`. + fn state_emitting(&self, emits: Emits) -> Option { + (0..self.states()).find(|&state| self.emits(state) == emits) + } + + /// The states with a row into `state`, other than itself and the states `within` it, in the + /// order of the states: the states the region is entered from. + fn entered_from(&self, state: usize, within: impl Fn(usize) -> bool) -> Vec { + let mut from: Vec = self + .transitions() + .filter(|&(source, _, target)| target == state && source != state && !within(source)) + .map(|(source, _, _)| source) + .collect(); + from.sort_unstable(); + from.dedup(); + from + } + + /// The text of the terminal of a row into `state` from a state it is entered from: the + /// region's opener. + fn opener_of(&self, state: usize, within: impl Fn(usize) -> bool) -> Option<&str> { self.transitions() - .find(|&(source, _, target)| self.emits(source) == from && self.emits(target) == to) + .find(|&(source, _, target)| target == state && source != state && !within(source)) .map(|(_, on, _)| self.terminal_text(on)) } - /// Whether a row of the table enters or leaves a state emitting `emits`. - fn has_state(&self, emits: Emits) -> bool { + /// The text of the terminal of a row from `state` back to a state it is entered from, other + /// than the end of the turn: the region's close. The two states name it, not the order of the + /// rows. + fn close_of(&self, state: usize, within: impl Fn(usize) -> bool + Copy) -> Option<&str> { + let from = self.entered_from(state, within); + let turn_close = self.turn_close(); self.transitions() - .any(|(source, _, target)| self.emits(source) == emits || self.emits(target) == emits) + .find(|&(source, on, target)| { + source == state && from.contains(&target) && Some(on) != turn_close + }) + .map(|(_, on, _)| self.terminal_text(on)) + } + + /// The close of the region `state` is, or `None` for the turn's own state, where the opener + /// leaves the engine and which no row opens as a region. + fn close_of_region(&self, state: usize) -> Option<&str> { + (state != 0) + .then(|| self.close_of(state, |_| false)) + .flatten() + } + + /// The terminal with a row from every state: the end of the turn, which ends whatever is + /// open, and so closes no region of its own. + fn turn_close(&self) -> Option { + let terminals = self.terminal_texts().count(); + (0..terminals).find(|&on| { + (0..self.states()).all(|state| { + self.transitions() + .any(|(source, t, _)| source == state && t == on) + }) + }) } } @@ -710,7 +786,6 @@ mod tests { formats::ling(), formats::hy4(), formats::minimax_m3(), - formats::kimi_k3(), formats::olmo3(), formats::lfm2_5(), formats::xlam(), @@ -724,6 +799,64 @@ mod tests { } } + #[test] + fn kimi_k3_derives_the_tag_the_gateway_hand_writes_today() { + // The old crate's `KimiK3Parser::build_structural_tag`: one tag per way into the block + // of calls (from the turn itself, after the thought's close, after the answer's close), + // each the same `plus` of an `or` of the tools' calls, closed by the block's close, with + // the opener and the two closes as triggers. + let tools = weather_tools(); + let payload = formats::kimi_k3() + .grammar(&tools, true, false) + .expect("a grammar") + .payload(); + let format = &payload["format"]; + assert_eq!(format["type"], "triggered_tags"); + assert_eq!( + format["triggers"], + value!([ + "<|open|>tools<|sep|>", + "<|close|>think<|sep|>", + "<|close|>response<|sep|>" + ]) + ); + assert_eq!(format["at_least_one"], true); + let tags = format["tags"].as_array().expect("three ways in"); + assert_eq!( + tags.iter().map(|tag| &tag["begin"]).collect::>(), + [ + "<|open|>tools<|sep|>", + "<|close|>think<|sep|><|open|>tools<|sep|>", + "<|close|>response<|sep|><|open|>tools<|sep|>", + ] + ); + for tag in tags { + assert_eq!(tag["end"], "<|close|>tools<|sep|>"); + assert_eq!(tag["content"], tags[0]["content"]); + } + assert_eq!(tags[0]["content"]["type"], "plus"); + let calls = tags[0]["content"]["content"]["elements"] + .as_array() + .expect("one call per named tool"); + assert_eq!(calls.len(), 3, "a nameless tool has no call"); + assert_eq!( + calls[0]["elements"][0]["value"], + "<|open|>call tool=\"get_weather\" index=\"" + ); + // The prompt's open thought changes nothing: the tag closes it on its own way in. + assert_eq!( + formats::kimi_k3().grammar(&tools, false, true), + formats::kimi_k3().grammar(&tools, false, false) + ); + assert_eq!( + formats::kimi_k3() + .grammar(&tools, false, false) + .expect("a grammar") + .payload()["format"]["at_least_one"], + false + ); + } + #[test] fn every_grammar_node_writes_the_engines_keys() { let leaf = Grammar::Sequence(vec![ diff --git a/crates/symphony/src/grammar/xtml.rs b/crates/symphony/src/grammar/xtml.rs new file mode 100644 index 0000000000..7123af24a6 --- /dev/null +++ b/crates/symphony/src/grammar/xtml.rs @@ -0,0 +1,555 @@ +//! Kimi K3's structural tag: the block of calls as the gateway's Kimi K3 parser writes it by hand, +//! here from the table's markers and the request's tools. +//! +//! The table gives the block its opener and close and the ways into it: Kimi K3's prompt leaves +//! the thought open in thinking mode and the answer otherwise, and a block the model could not +//! reach from there would force the calls inside a region the model can never close. So the tag +//! has one way in per state the block is entered from, that state's own close followed by the +//! opener, each closed by the block's close, and the opener and the closes trigger it. Inside, one +//! or more calls, each `<|open|>call tool="NAME" index="N"<|sep|>`, its arguments and the call's +//! close, with the name as the request gives it (`&` and `"` escaped as the template does). +//! +//! The arguments follow the tool's schema: one argument tag per property, in the schema's order, +//! each optional unless the schema requires it, its `type` attribute and value pinned by the +//! property's type. A string is written as it is, so its value is any text up to a marker; a +//! string `enum` is one of its values; a number, an object or an array is JSON the property's +//! schema accepts, with the tool's definitions attached so a `$ref` inside it still resolves; a +//! boolean is JSON `true` or `false`; `null` is the word. A property the schema leaves untyped, or +//! one whose `$ref` points at nothing, keeps its key but takes any type and any value; a schema +//! without properties takes any number of such arguments. The engine then accepts exactly what +//! the XTML assembler reads ([`tagged::xtml`](crate::tagged::xtml)). + +use openai_protocol::common::Tool; +use serde_json::{json, Value}; + +use super::{CallMarkers, Grammar, Tag}; + +const OPEN: &str = "<|open|>"; +const CLOSE: &str = "<|close|>"; +const SEP: &str = "<|sep|>"; +/// The call's count of the turn's calls, from one. +const INDEX: &str = "[1-9][0-9]*"; + +/// The block of calls for `tools`, or `None` for a table whose calls sit in no block. +pub(super) fn calls( + markers: &CallMarkers<'_>, + tools: &[&Tool], + at_least_one: bool, +) -> Option { + let block = markers.block.as_ref()?; + let one_call = Grammar::Or( + tools + .iter() + .map(|tool| call(markers, &tool.function.name, &tool.function.parameters)) + .collect(), + ); + let content = Grammar::Plus(Box::new(one_call)); + let tags = block + .before_open + .iter() + .map(|before| { + Tag::new( + format!("{before}{}", block.open), + content.clone(), + block.close, + ) + }) + .collect(); + let mut triggers = vec![block.open.to_string()]; + triggers.extend( + block + .before_open + .iter() + .filter(|before| !before.is_empty()) + .map(|before| before.to_string()), + ); + Some(Grammar::TriggeredTags { + triggers, + tags, + at_least_one, + }) +} + +/// One call: the call's opener and the tool's name, its index, the arguments and the call's close. +fn call(markers: &CallMarkers<'_>, name: &str, parameters: &Value) -> Grammar { + Grammar::Sequence(vec![ + Grammar::ConstString(format!("{}{}\" index=\"", markers.call_open, escape(name))), + Grammar::Regex(INDEX.to_string()), + Grammar::ConstString(format!("\"{SEP}")), + arguments(parameters), + Grammar::ConstString(markers.call_close.to_string()), + ]) +} + +/// The arguments the tool's schema asks for: one tag per property, in the schema's order, each +/// optional unless required; any arguments at all for a schema without properties. +fn arguments(parameters: &Value) -> Grammar { + let properties = parameters + .get("properties") + .and_then(Value::as_object) + .filter(|properties| !properties.is_empty()); + let Some(properties) = properties else { + return Grammar::Star(Box::new(any_argument())); + }; + let required: Vec<&str> = parameters + .get("required") + .and_then(Value::as_array) + .map(|names| names.iter().filter_map(Value::as_str).collect()) + .unwrap_or_default(); + let definitions = Definitions::of(parameters); + Grammar::Sequence( + properties + .iter() + .map(|(key, schema)| { + let tag = argument(key, schema, definitions); + if required.contains(&key.as_str()) { + tag + } else { + Grammar::Optional(Box::new(tag)) + } + }) + .collect(), + ) +} + +/// One argument tag: the key, the type the property's schema pins and a value of that type, or +/// the key with any type and any value when the schema pins none. +fn argument(key: &str, schema: &Value, definitions: Definitions<'_>) -> Grammar { + let key = escape(key); + match shape(schema, definitions) { + Some((type_name, value)) => Grammar::Sequence(vec![ + Grammar::ConstString(format!( + "{OPEN}argument key=\"{key}\" type=\"{type_name}\"{SEP}" + )), + value, + Grammar::ConstString(format!("{CLOSE}argument{SEP}")), + ]), + None => Grammar::Sequence(vec![ + Grammar::ConstString(format!("{OPEN}argument key=\"{key}\" type=\"")), + attribute_text(), + Grammar::ConstString(format!("\"{SEP}")), + any_value(), + Grammar::ConstString(format!("{CLOSE}argument{SEP}")), + ]), + } +} + +/// An argument tag with any key, any type and any value. +fn any_argument() -> Grammar { + Grammar::Sequence(vec![ + Grammar::ConstString(format!("{OPEN}argument key=\"")), + attribute_text(), + Grammar::ConstString("\" type=\"".to_string()), + attribute_text(), + Grammar::ConstString(format!("\"{SEP}")), + any_value(), + Grammar::ConstString(format!("{CLOSE}argument{SEP}")), + ]) +} + +/// The text of a quoted attribute: no quote, and not the start of a marker. +fn attribute_text() -> Grammar { + Grammar::AnyText { + excludes: vec!["\"".to_string(), "<|".to_string()], + } +} + +/// A value written as it is: any text up to a marker, a lone `<` or `|` included. +fn any_value() -> Grammar { + Grammar::AnyText { + excludes: vec![CLOSE.to_string(), OPEN.to_string(), SEP.to_string()], + } +} + +/// The `type` attribute and the value grammar a property's schema pins, or `None` when it pins +/// no type the syntax has, or a `$ref` in it points at nothing (a grammar with such a pointer +/// would not compile, and the whole call with it). A `$ref` lends its target's keywords, and the +/// property's own win, as in JSON Schema 2020-12. The template writes `integer` as `number`. +fn shape(schema: &Value, definitions: Definitions<'_>) -> Option<(&'static str, Grammar)> { + if !definitions.pointers_resolve(schema) { + return None; + } + let target = definitions.target_of(schema); + let keyword = |name: &str| { + schema + .get(name) + .or_else(|| target.and_then(|target| target.get(name))) + }; + if let Some(values) = keyword("enum").and_then(Value::as_array) { + if !values.is_empty() && values.iter().all(Value::is_string) { + let options = values + .iter() + .filter_map(Value::as_str) + .map(|value| Grammar::ConstString(value.to_string())) + .collect(); + return Some(("string", Grammar::Or(options))); + } + } + let json = |schema: Value| Grammar::JsonSchema { + schema, + style: None, + }; + match keyword("type").and_then(Value::as_str)? { + "string" => Some(("string", any_value())), + "integer" | "number" => Some(("number", json(definitions.attached_to(schema)))), + "boolean" => Some(("boolean", json(json!({"type": "boolean"})))), + "object" => Some(("object", json(definitions.attached_to(schema)))), + "array" => Some(("array", json(definitions.attached_to(schema)))), + "null" => Some(("null", Grammar::ConstString("null".to_string()))), + _ => None, + } +} + +/// What the template escapes in an attribute: `&` first, then `"`. +fn escape(value: &str) -> String { + value.replace('&', "&").replace('"', """) +} + +/// The definitions at the root of a tool's schema, `$defs` and `definitions`, and the root itself. +/// Each property's schema goes to the engine as a document of its own, and a `$ref` such as +/// `#/$defs/Node` points from the root of the document it stands in, so the definitions travel +/// with the property's schema. +#[derive(Clone, Copy)] +struct Definitions<'a> { + root: &'a Value, + defs: Option<&'a Value>, + definitions: Option<&'a Value>, +} + +impl<'a> Definitions<'a> { + fn of(root: &'a Value) -> Self { + Self { + root, + defs: root.get("$defs"), + definitions: root.get("definitions"), + } + } + + /// The schema a local pointer names: `#` the root, `#/$defs/Name` or `#/definitions/Name` an + /// entry, with the pointer's escapes (`~1` for `/`, `~0` for `~`) undone. + fn resolve(self, pointer: &str) -> Option<&'a Value> { + if pointer == "#" || pointer == "#/" { + return Some(self.root); + } + let mut segments = pointer.strip_prefix("#/")?.split('/'); + let mut node = match segments.next()? { + "$defs" => self.defs?, + "definitions" => self.definitions?, + _ => return None, + }; + for segment in segments { + node = node.get(segment.replace("~1", "/").replace("~0", "~"))?; + } + Some(node) + } + + /// The schema the property's own `$ref` names, one hop: its keywords stand beside the + /// property's, and a cycle of definitions is the engine's to expand, not this function's. + fn target_of(self, schema: &'a Value) -> Option<&'a Value> { + self.resolve(schema.get("$ref")?.as_str()?) + } + + /// Whether every local pointer in `schema` names something. + fn pointers_resolve(self, schema: &Value) -> bool { + match schema { + Value::Object(map) => map.iter().all(|(key, value)| { + let points_at_nothing = is_reference(key) + && value.as_str().is_some_and(|pointer| { + pointer.starts_with('#') && self.resolve(pointer).is_none() + }); + !points_at_nothing && self.pointers_resolve(value) + }), + Value::Array(items) => items.iter().all(|item| self.pointers_resolve(item)), + _ => true, + } + } + + /// The property's schema with the root's definitions attached, so a local pointer in it still + /// resolves when the engine compiles it on its own. A definition the schema declares itself + /// stays; the root's fill in the names it lacks. A schema with no local pointer goes as it is. + fn attached_to(self, schema: &Value) -> Value { + let mut attached = schema.clone(); + if self.defs.is_none() && self.definitions.is_none() || !has_local_reference(schema) { + return attached; + } + let Some(object) = attached.as_object_mut() else { + return attached; + }; + for (name, block) in [("$defs", self.defs), ("definitions", self.definitions)] { + let Some(Value::Object(entries)) = block else { + continue; + }; + match object.get_mut(name) { + Some(Value::Object(own)) => { + for (key, value) in entries { + own.entry(key.clone()).or_insert_with(|| value.clone()); + } + } + Some(_) => {} + None => { + object.insert(name.to_string(), Value::Object(entries.clone())); + } + } + } + attached + } +} + +/// Whether `key` is one of the keywords that point at another schema. +fn is_reference(key: &str) -> bool { + matches!(key, "$ref" | "$dynamicRef" | "$recursiveRef") +} + +/// Whether `schema` points anywhere inside its own document. +fn has_local_reference(schema: &Value) -> bool { + match schema { + Value::Object(map) => map.iter().any(|(key, value)| { + is_reference(key) + && value + .as_str() + .is_some_and(|pointer| pointer.starts_with('#')) + || has_local_reference(value) + }), + Value::Array(items) => items.iter().any(has_local_reference), + _ => false, + } +} + +#[cfg(test)] +mod tests { + use openai_protocol::common::Function; + use serde_json::json as value; + + use super::*; + use crate::formats; + + fn tool(name: &str, parameters: Value) -> Tool { + Tool { + tool_type: "function".to_string(), + function: Function { + name: name.to_string(), + description: None, + parameters, + strict: None, + }, + } + } + + /// An argument tag's opener for `key`, typed `type_name`. + fn argument_open(key: &str, type_name: &str) -> String { + format!("<|open|>argument key=\"{key}\" type=\"{type_name}\"<|sep|>") + } + + const ARGUMENT_CLOSE: &str = "<|close|>argument<|sep|>"; + + /// The first call of the tag's first way in, as JSON. + fn first_call(tools: &[Tool]) -> Value { + let payload = formats::kimi_k3() + .grammar(tools, true, false) + .expect("a grammar") + .payload(); + payload["format"]["tags"][0]["content"]["content"]["elements"][0].clone() + } + + #[test] + fn a_call_pins_its_framing_and_the_tools_name() { + // The old crate's `build_call_format`: the opener with the name, the index, `"<|sep|>`, + // the arguments, the call's close. + let call = first_call(&[tool( + "get_weather", + value!({"type": "object", "properties": {"city": {"type": "string"}}}), + )]); + assert_eq!(call["type"], "sequence"); + let elements = call["elements"].as_array().expect("five elements"); + assert_eq!(elements.len(), 5); + assert_eq!( + elements[0]["value"], + "<|open|>call tool=\"get_weather\" index=\"" + ); + assert_eq!( + elements[1], + value!({"type": "regex", "pattern": "[1-9][0-9]*"}) + ); + assert_eq!(elements[2]["value"], "\"<|sep|>"); + assert_eq!(elements[3]["type"], "sequence"); + assert_eq!(elements[4]["value"], "<|close|>call<|sep|>"); + } + + #[test] + fn the_arguments_follow_the_schema_one_tag_per_property_in_its_order() { + // The old crate's `build_args_format` and `classify_arg`: a required string as any text up + // to a marker, a required integer as `number` with the property's schema, an optional + // string enum as one of its values inside `optional`. + let call = first_call(&[tool( + "get_weather", + value!({ + "type": "object", + "properties": { + "city": {"type": "string"}, + "days": {"type": "integer", "minimum": 1}, + "units": {"type": "string", "enum": ["c", "f"]}, + }, + "required": ["city", "days"], + }), + )]); + let markers = value!(["<|close|>", "<|open|>", "<|sep|>"]); + assert_eq!( + call["elements"][3], + value!({"type": "sequence", "elements": [ + {"type": "sequence", "elements": [ + {"type": "const_string", "value": argument_open("city", "string")}, + {"type": "any_text", "excludes": markers}, + {"type": "const_string", "value": ARGUMENT_CLOSE}, + ]}, + {"type": "sequence", "elements": [ + {"type": "const_string", "value": argument_open("days", "number")}, + {"type": "json_schema", "json_schema": {"type": "integer", "minimum": 1}}, + {"type": "const_string", "value": ARGUMENT_CLOSE}, + ]}, + {"type": "optional", "content": {"type": "sequence", "elements": [ + {"type": "const_string", "value": argument_open("units", "string")}, + {"type": "or", "elements": [ + {"type": "const_string", "value": "c"}, + {"type": "const_string", "value": "f"}, + ]}, + {"type": "const_string", "value": ARGUMENT_CLOSE}, + ]}}, + ]}) + ); + } + + #[test] + fn every_type_the_syntax_has_pins_its_attribute_and_value() { + let call = first_call(&[tool( + "t", + value!({ + "type": "object", + "properties": { + "flag": {"type": "boolean", "description": "ignored"}, + "tags": {"type": "array", "items": {"type": "string"}}, + "where": {"type": "object", "properties": {"x": {"type": "number"}}}, + "nothing": {"type": "null"}, + "ratio": {"type": "number"}, + }, + "required": ["flag", "tags", "where", "nothing", "ratio"], + }), + )]); + let slots = call["elements"][3]["elements"] + .as_array() + .expect("five slots"); + let opener = |slot: &Value| slot["elements"][0]["value"].clone(); + let pinned = |slot: &Value| slot["elements"][1].clone(); + assert_eq!( + opener(&slots[0]), + "<|open|>argument key=\"flag\" type=\"boolean\"<|sep|>" + ); + assert_eq!( + pinned(&slots[0]), + value!({"type": "json_schema", "json_schema": {"type": "boolean"}}) + ); + assert_eq!( + opener(&slots[1]), + "<|open|>argument key=\"tags\" type=\"array\"<|sep|>" + ); + assert_eq!( + pinned(&slots[1])["json_schema"], + value!({"type": "array", "items": {"type": "string"}}) + ); + assert_eq!( + opener(&slots[2]), + "<|open|>argument key=\"where\" type=\"object\"<|sep|>" + ); + assert_eq!( + opener(&slots[3]), + "<|open|>argument key=\"nothing\" type=\"null\"<|sep|>" + ); + assert_eq!( + pinned(&slots[3]), + value!({"type": "const_string", "value": "null"}) + ); + assert_eq!( + opener(&slots[4]), + "<|open|>argument key=\"ratio\" type=\"number\"<|sep|>" + ); + } + + #[test] + fn a_reference_brings_the_tools_definitions_along_and_a_dangling_one_pins_nothing() { + let call = first_call(&[tool( + "t", + value!({ + "type": "object", + "$defs": {"Count": {"type": "integer"}, "Unused": {"type": "string"}}, + "properties": { + "count": {"$ref": "#/$defs/Count"}, + "lost": {"$ref": "#/$defs/Missing"}, + "untyped": {"description": "anything"}, + }, + "required": ["count", "lost", "untyped"], + }), + )]); + let slots = call["elements"][3]["elements"] + .as_array() + .expect("three slots"); + // The `$ref` lends its target's type, and the definitions travel with the schema. + assert_eq!( + slots[0]["elements"][0]["value"], + "<|open|>argument key=\"count\" type=\"number\"<|sep|>" + ); + assert_eq!( + slots[0]["elements"][1]["json_schema"], + value!({ + "$ref": "#/$defs/Count", + "$defs": {"Count": {"type": "integer"}, "Unused": {"type": "string"}}, + }) + ); + // A pointer at nothing, or no type: the key stays, the type and the value are the model's. + for (slot, key) in [(&slots[1], "lost"), (&slots[2], "untyped")] { + let begin = format!("<|open|>argument key=\"{key}\" type=\""); + assert_eq!( + *slot, + value!({"type": "sequence", "elements": [ + {"type": "const_string", "value": begin}, + {"type": "any_text", "excludes": ["\"", "<|"]}, + {"type": "const_string", "value": "\"<|sep|>"}, + {"type": "any_text", "excludes": ["<|close|>", "<|open|>", "<|sep|>"]}, + {"type": "const_string", "value": ARGUMENT_CLOSE}, + ]}) + ); + } + } + + #[test] + fn attribute_values_are_escaped_as_the_template_writes_them() { + let parameters = value!({ + "type": "object", + "properties": {"k\"q": {"type": "string"}}, + "required": ["k\"q"], + }); + let call = first_call(&[tool("a&b\"c", parameters)]); + assert_eq!( + call["elements"][0]["value"], + "<|open|>call tool=\"a&b"c\" index=\"" + ); + assert_eq!( + call["elements"][3]["elements"][0]["elements"][0]["value"], + "<|open|>argument key=\"k"q\" type=\"string\"<|sep|>" + ); + } + + #[test] + fn a_schema_without_properties_takes_any_arguments() { + for parameters in [ + value!({"type": "object"}), + value!({}), + value!({"properties": {}}), + ] { + let call = first_call(&[tool("ping", parameters)]); + assert_eq!(call["elements"][3]["type"], "star"); + assert_eq!( + call["elements"][3]["content"]["elements"][0]["value"], + "<|open|>argument key=\"" + ); + } + } +} From a265b2512cd2452b56199ea58fe58a823900bc12 Mon Sep 17 00:00:00 2001 From: Simo Lin <25425177+slin1237@users.noreply.github.com> Date: Thu, 8 Oct 2026 01:13:45 -0700 Subject: [PATCH 2/4] fix(symphony): a region's opener pairs with its close, and the turn's end returns to the turn's state The review of the Kimi K3 tag found the region rules still leaning on the order of the rows in two places. A region's opener was the first row into its state; it is now the row from the state the region's close goes back to, and each way into a block of calls takes the opener of its own row, after its own state's close. The end of the turn was the terminal every state has a row for; Qwen 2.5's call opener met that through its self-row, so the rows must now land in the turn's own state as well. A way in from a state with no close of its own gives no grammar instead of a bare opener. A test rebuilds Kimi K3's table from its rows in reverse and gets the same grammar; the block entered by two terminals pins its opener as DSML. Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com> Co-authored-by: Chang Su <8605658+CatherineSue@users.noreply.github.com> Co-authored-by: Keyang Ru Co-authored-by: key4ng Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com> Co-authored-by: LightSeek Foundation <243258330+lightseek-bot@users.noreply.github.com> --- crates/symphony/src/grammar.rs | 223 ++++++++++++++++++++++------ crates/symphony/src/grammar/xtml.rs | 29 ++-- 2 files changed, 186 insertions(+), 66 deletions(-) diff --git a/crates/symphony/src/grammar.rs b/crates/symphony/src/grammar.rs index 4eb629b0f8..46c3030d4e 100644 --- a/crates/symphony/src/grammar.rs +++ b/crates/symphony/src/grammar.rs @@ -180,13 +180,39 @@ struct CallMarkers<'a> { call_close: &'a str, } -/// A block of calls: the terminals into and out of its wrapper state, and what the model writes -/// before the opener on each way in, one per state the block is entered from, in the order of the -/// states: nothing from the turn's own state, the state's own close from a thought or an answer. +/// A block of calls: the terminals into and out of its wrapper state, and its ways in, one per +/// state the block is entered from, in the order of the states. struct Block<'a> { open: &'a str, close: &'a str, - before_open: Vec<&'a str>, + ways_in: Vec>, +} + +/// One way into a block of calls: from the turn's own state, the opener of that state's row into +/// the block; from a thought or an answer, that region's close first, then the opener of its own +/// row, so the model closes what the prompt left open on its way to the calls. A region with no +/// close of its own has no way in, and the block no grammar. +struct WayIn<'a> { + close: Option<&'a str>, + open: &'a str, +} + +impl WayIn<'_> { + /// What the model writes to enter the block this way. + fn begin(&self) -> String { + format!("{}{}", self.close.unwrap_or(""), self.open) + } + + /// The text that triggers this way in: the close when there is one, else the opener. + fn trigger(&self) -> &str { + self.close.unwrap_or(self.open) + } +} + +/// A region's two markers: the terminal of the row into its state, and of the row back out. +struct Region<'a> { + open: &'a str, + close: &'a str, } impl Format { @@ -231,7 +257,7 @@ impl Format { let Some(reasoning) = self.state_emitting(Emits::Reasoning) else { return Some(calls); }; - let think_close = self.close_of(reasoning, |_| false)?; + let think_close = self.region(reasoning, |_| false)?.close; let mut excludes: Vec = self.terminal_texts().map(str::to_string).collect(); let inner: &[&str] = match self.call_syntax() { Some(CallSyntax::Keyed(tags)) => &[ @@ -358,11 +384,12 @@ impl Format { ])) } - /// The call markers the rows give: the opener and the close of the arguments state, and of - /// the wrapper state with a row into it when the calls sit in a block. + /// The call markers the rows give: the region of the arguments state, and of the wrapper state + /// with a row into it when the calls sit in a block. fn call_markers(&self) -> Option> { let arguments = self.state_emitting(Emits::Arguments)?; let is_arguments = |state: usize| self.emits(state) == Emits::Arguments; + let call = self.region(arguments, |_| false)?; let block = self .transitions() .find(|&(source, _, target)| { @@ -370,21 +397,20 @@ impl Format { }) .map(|(source, _, _)| source); let block = match block { - Some(state) => Some(Block { - open: self.opener_of(state, is_arguments)?, - close: self.close_of(state, is_arguments)?, - before_open: self - .entered_from(state, is_arguments) - .into_iter() - .map(|from| self.close_of_region(from).unwrap_or("")) - .collect(), - }), + Some(state) => { + let Region { open, close } = self.region(state, is_arguments)?; + Some(Block { + open, + close, + ways_in: self.ways_into(state, is_arguments)?, + }) + } None => None, }; Some(CallMarkers { block, - call_open: self.opener_of(arguments, |_| false)?, - call_close: self.close_of(arguments, |_| false)?, + call_open: call.open, + call_close: call.close, }) } @@ -406,43 +432,59 @@ impl Format { from } - /// The text of the terminal of a row into `state` from a state it is entered from: the - /// region's opener. - fn opener_of(&self, state: usize, within: impl Fn(usize) -> bool) -> Option<&str> { - self.transitions() - .find(|&(source, _, target)| target == state && source != state && !within(source)) - .map(|(_, on, _)| self.terminal_text(on)) - } - - /// The text of the terminal of a row from `state` back to a state it is entered from, other - /// than the end of the turn: the region's close. The two states name it, not the order of the + /// The markers of the region `state` is: its close is the terminal of the row from `state` + /// back to a state it is entered from, other than the end of the turn, and its opener the + /// terminal of that state's row into `state`. The states name the two, not the order of the /// rows. - fn close_of(&self, state: usize, within: impl Fn(usize) -> bool + Copy) -> Option<&str> { + fn region(&self, state: usize, within: impl Fn(usize) -> bool + Copy) -> Option> { let from = self.entered_from(state, within); let turn_close = self.turn_close(); - self.transitions() - .find(|&(source, on, target)| { - source == state && from.contains(&target) && Some(on) != turn_close - }) - .map(|(_, on, _)| self.terminal_text(on)) + let (_, close, home) = self.transitions().find(|&(source, on, target)| { + source == state && from.contains(&target) && Some(on) != turn_close + })?; + let (_, open, _) = self + .transitions() + .find(|&(source, _, target)| source == home && target == state)?; + Some(Region { + open: self.terminal_text(open), + close: self.terminal_text(close), + }) } - /// The close of the region `state` is, or `None` for the turn's own state, where the opener - /// leaves the engine and which no row opens as a region. - fn close_of_region(&self, state: usize) -> Option<&str> { - (state != 0) - .then(|| self.close_of(state, |_| false)) - .flatten() + /// The ways into the block `state` is, one per state it is entered from, in the order of the + /// states: the terminal of that state's row into the block, after the close of the region the + /// state is, except for the turn's own state. `None` when a state on the way has no close. + fn ways_into( + &self, + state: usize, + within: impl Fn(usize) -> bool + Copy, + ) -> Option>> { + self.entered_from(state, within) + .into_iter() + .map(|from| { + let (_, open, _) = self + .transitions() + .find(|&(source, _, target)| source == from && target == state)?; + let close = match from { + 0 => None, + region => Some(self.region(region, |_| false)?.close), + }; + Some(WayIn { + close, + open: self.terminal_text(open), + }) + }) + .collect() } - /// The terminal with a row from every state: the end of the turn, which ends whatever is - /// open, and so closes no region of its own. + /// The terminal that every state has a row for, each back to the turn's own state: the end of + /// the turn, which ends whatever is open and closes no region of its own. fn turn_close(&self) -> Option { let terminals = self.terminal_texts().count(); (0..terminals).find(|&on| { (0..self.states()).all(|state| { self.transitions() - .any(|(source, t, _)| source == state && t == on) + .any(|(source, t, target)| source == state && t == on && target == 0) }) }) } @@ -625,10 +667,11 @@ mod tests { } #[test] - fn a_thought_that_closes_somewhere_other_than_content_has_no_prefix_yet() { - // A thought that closes into the calls block, as Kimi K3's closes into its message: the - // tag derives without the prefix, but a prefix ending at the block's opener would force - // the call inside the thought, so with the reasoning open the table gives `None`. + fn a_thought_without_a_close_of_its_own_gives_no_grammar() { + // A thought that closes into the calls block and nowhere else: the block is entered from a + // region with no close of its own, so there is no way in that closes what the prompt left + // open, and the table gives `None` rather than a tag the model could write inside its + // thought, with the reasoning open or not. let into_the_block = Format::new("into_the_block") .terminal("think_open", "") .terminal("think_close", "") @@ -648,10 +691,94 @@ mod tests { .transition("calls", "calls_close", "content") .calls(CallSyntax::Json); let tools = weather_tools(); - assert!(into_the_block.grammar(&tools, true, false).is_some()); + assert!(into_the_block.grammar(&tools, true, false).is_none()); assert!(into_the_block.grammar(&tools, true, true).is_none()); } + #[test] + fn a_blocks_opener_is_the_row_from_the_state_its_close_returns_to() { + // The block is entered by two terminals, `` from the thought (the + // first row in) and `` from content; its close returns to content, so `` is + // its opener, whichever row comes first, and DSML writes it as the block's opening. + let two_doors = Format::new("two_doors") + .terminal("think_open", "") + .terminal("think_close", "") + .terminal("from_thought", "") + .terminal("calls_open", "") + .terminal("calls_close", "") + .terminal("call_open", "") + .terminal("call_close", "") + .state("content", Emits::Content) + .state("reasoning", Emits::Reasoning) + .state("calls", Emits::Wrapper) + .state("call", Emits::Arguments) + .transition("content", "think_open", "reasoning") + .transition("reasoning", "think_close", "content") + .transition("reasoning", "from_thought", "calls") + .transition("content", "calls_open", "calls") + .transition("calls", "call_open", "call") + .transition("call", "call_close", "calls") + .transition("calls", "calls_close", "content") + .calls(CallSyntax::Dsml); + let tools = weather_tools(); + let payload = two_doors + .grammar(&tools, true, false) + .expect("a grammar") + .payload(); + assert_eq!(payload["format"]["elements"][0]["value"], "\n\n\n"); + } + + #[test] + fn kimi_k3_derives_the_same_tag_whatever_the_order_of_its_rows() { + // Kimi K3's rows in reverse: the end of the turn is still the terminal every state's row + // sends back to the turn's state, the block's close is still the tools tag and not that + // end, and the three ways in keep the order of the states. + let rows = [ + ("message", "think_open", "reasoning"), + ("reasoning", "think_close", "message"), + ("reasoning", "response_open", "content"), + ("reasoning", "tools_open", "tools"), + ("reasoning", "message_close", "message"), + ("message", "response_open", "content"), + ("content", "response_close", "message"), + ("content", "message_close", "message"), + ("content", "tools_open", "tools"), + ("message", "tools_open", "tools"), + ("tools", "call_open", "call"), + ("call", "call_close", "tools"), + ("call", "call_open", "call"), + ("call", "tools_close", "message"), + ("call", "message_close", "message"), + ("tools", "tools_close", "message"), + ("tools", "message_close", "message"), + ("message", "message_close", "message"), + ]; + let mut reversed = Format::new("kimi_k3_reversed") + .terminal("think_open", "<|open|>think<|sep|>") + .terminal("think_close", "<|close|>think<|sep|>") + .terminal("response_open", "<|open|>response<|sep|>") + .terminal("response_close", "<|close|>response<|sep|>") + .terminal("tools_open", "<|open|>tools<|sep|>") + .terminal("tools_close", "<|close|>tools<|sep|>") + .terminal("call_open", "<|open|>call tool=\"") + .terminal("call_close", "<|close|>call<|sep|>") + .terminal("message_close", "<|close|>message<|sep|>") + .state("message", Emits::Wrapper) + .state("reasoning", Emits::Reasoning) + .state("content", Emits::Content) + .state("tools", Emits::Wrapper) + .state("call", Emits::Arguments); + for (from, on, to) in rows.iter().rev() { + reversed = reversed.transition(from, on, to); + } + let reversed = reversed.calls(CallSyntax::Xtml); + let tools = weather_tools(); + assert_eq!( + reversed.grammar(&tools, true, true), + formats::kimi_k3().grammar(&tools, true, true) + ); + } + #[test] fn the_markers_come_from_the_states_not_from_the_order_of_the_rows() { // DeepSeek V4.1's rows with the reasoning state's two exits swapped and the invoke's two diff --git a/crates/symphony/src/grammar/xtml.rs b/crates/symphony/src/grammar/xtml.rs index 7123af24a6..70f606b75d 100644 --- a/crates/symphony/src/grammar/xtml.rs +++ b/crates/symphony/src/grammar/xtml.rs @@ -5,7 +5,8 @@ //! the thought open in thinking mode and the answer otherwise, and a block the model could not //! reach from there would force the calls inside a region the model can never close. So the tag //! has one way in per state the block is entered from, that state's own close followed by the -//! opener, each closed by the block's close, and the opener and the closes trigger it. Inside, one +//! opener, each closed by the block's close, and each way's first text (the opener from the turn +//! itself, the close from the thought or the answer) triggers it. Inside, one //! or more calls, each `<|open|>call tool="NAME" index="N"<|sep|>`, its arguments and the call's //! close, with the name as the request gives it (`&` and `"` escaped as the template does). //! @@ -22,7 +23,7 @@ use openai_protocol::common::Tool; use serde_json::{json, Value}; -use super::{CallMarkers, Grammar, Tag}; +use super::{CallMarkers, Grammar, Tag, WayIn}; const OPEN: &str = "<|open|>"; const CLOSE: &str = "<|close|>"; @@ -45,24 +46,16 @@ pub(super) fn calls( ); let content = Grammar::Plus(Box::new(one_call)); let tags = block - .before_open + .ways_in .iter() - .map(|before| { - Tag::new( - format!("{before}{}", block.open), - content.clone(), - block.close, - ) - }) + .map(|way| Tag::new(way.begin(), content.clone(), block.close)) .collect(); - let mut triggers = vec![block.open.to_string()]; - triggers.extend( - block - .before_open - .iter() - .filter(|before| !before.is_empty()) - .map(|before| before.to_string()), - ); + let mut triggers: Vec = Vec::new(); + for trigger in block.ways_in.iter().map(WayIn::trigger) { + if !triggers.iter().any(|known| known == trigger) { + triggers.push(trigger.to_string()); + } + } Some(Grammar::TriggeredTags { triggers, tags, From 6c656c1888e75b99c4526ec72068efb61cd3fd62 Mon Sep 17 00:00:00 2001 From: Simo Lin <25425177+slin1237@users.noreply.github.com> Date: Thu, 8 Oct 2026 01:24:13 -0700 Subject: [PATCH 3/4] fix(symphony): the ways into a block are the table's own paths, and only the XTML tag reads them Three findings on the second round. A way into the block from a thought or an answer paired that region's close with the region's own row into the block; after the close the engine stands in the state the close returns to, so the way in now goes on through that state's row (the region's own row when it has none). A state with no close of its own gave no grammar at all; every way in is a path the table has, so such a state, like MiniMax M3's text after the thought, now enters by its own row, and the turn's own state keeps doing so. And the ways in were built for every syntax though only Kimi K3's tag reads them; they are built in its branch now. Two more rules keep the derivation honest: a thought's or an answer's region is read with the calls' states excluded, so its own row into the block is not taken for its close; and the same way in is kept once. Tests: a block entered by two terminals (its ways in are `` and ``), a thought that closes into the block (`` and ``, and no prefix), and MiniMax M3's table under XTML (one way in, ``). Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com> Co-authored-by: Chang Su <8605658+CatherineSue@users.noreply.github.com> Co-authored-by: Keyang Ru Co-authored-by: key4ng Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com> Co-authored-by: LightSeek Foundation <243258330+lightseek-bot@users.noreply.github.com> --- crates/symphony/src/grammar.rs | 267 ++++++++++++++++++---------- crates/symphony/src/grammar/xtml.rs | 9 +- 2 files changed, 180 insertions(+), 96 deletions(-) diff --git a/crates/symphony/src/grammar.rs b/crates/symphony/src/grammar.rs index 46c3030d4e..89ddec3b60 100644 --- a/crates/symphony/src/grammar.rs +++ b/crates/symphony/src/grammar.rs @@ -30,9 +30,10 @@ //! comes back as it is; a thought that does not close back to where it opened has no prefix yet, //! and the table gives `None` rather than a tag the model could write inside its thought. Kimi //! K3's tag takes no prefix: the block of calls can be entered from the turn itself, from the -//! thought or from the answer, and the tag has one way in per state, that state's own close -//! followed by the block's opener, so the model closes whatever the prompt left open on its way -//! to the calls, whichever that was. +//! thought or from the answer, and the tag has one way in per state: from a region with a close +//! of its own, that close and then the row into the block from where the close returns; from any +//! other state, its own row into the block. So the model closes whatever the prompt left open on +//! its way to the calls, whichever that was. //! //! Not derived yet, so [`Format::grammar`] gives `None` for them: the tagged syntax (Qwen 3.5 and //! later, Seed-OSS), keyed arguments in another spelling than the compact one (Ling's newlines, @@ -180,18 +181,18 @@ struct CallMarkers<'a> { call_close: &'a str, } -/// A block of calls: the terminals into and out of its wrapper state, and its ways in, one per -/// state the block is entered from, in the order of the states. +/// A block of calls: its wrapper state, and the terminals into and out of it. struct Block<'a> { + state: usize, open: &'a str, close: &'a str, - ways_in: Vec>, } -/// One way into a block of calls: from the turn's own state, the opener of that state's row into -/// the block; from a thought or an answer, that region's close first, then the opener of its own -/// row, so the model closes what the prompt left open on its way to the calls. A region with no -/// close of its own has no way in, and the block no grammar. +/// One way into a block of calls, from a state with a row into it: from a region with a close of +/// its own (a thought, an answer), that close first, then the opener of the row into the block +/// from the state the close returns to, so the model closes what the prompt left open and goes on +/// from where that leaves it; from the turn's own state, or a state with no close, the opener of +/// its own row. Every way in is a path the table has. struct WayIn<'a> { close: Option<&'a str>, open: &'a str, @@ -209,10 +210,12 @@ impl WayIn<'_> { } } -/// A region's two markers: the terminal of the row into its state, and of the row back out. +/// A region's two markers, the terminal of the row into its state and of the row back out, and +/// the state the row back out returns to. struct Region<'a> { open: &'a str, close: &'a str, + home: usize, } impl Format { @@ -223,7 +226,8 @@ impl Format { /// inside its thought, and the tag follows a reasoning prefix the thought's close ends; a /// table without a thought has no prefix, and the tag comes back as it is; a table whose /// thought does not close back to where it opened has no prefix yet, and gives `None`. Kimi - /// K3's tag closes the thought on its own way in, and is the same either way. + /// K3's tag has a way in per state its block is entered from, the state's close first when it + /// has one, so it is the same either way. pub fn grammar( &self, tools: &[Tool], @@ -244,7 +248,10 @@ impl Format { Self::keyed_calls(&markers, &named, at_least_one) } CallSyntax::Dsml => Self::dsml_calls(&markers, &named, at_least_one)?, - CallSyntax::Xtml => return xtml::calls(&markers, &named, at_least_one), + CallSyntax::Xtml => { + let ways_in = self.ways_into(markers.block.as_ref()?.state); + return xtml::calls(&markers, &ways_in, &named, at_least_one); + } CallSyntax::Keyed(_) | CallSyntax::Tagged | CallSyntax::Pythonic @@ -257,7 +264,9 @@ impl Format { let Some(reasoning) = self.state_emitting(Emits::Reasoning) else { return Some(calls); }; - let think_close = self.region(reasoning, |_| false)?.close; + let think_close = self + .region(reasoning, |state| self.is_call_state(state))? + .close; let mut excludes: Vec = self.terminal_texts().map(str::to_string).collect(); let inner: &[&str] = match self.call_syntax() { Some(CallSyntax::Keyed(tags)) => &[ @@ -398,12 +407,8 @@ impl Format { .map(|(source, _, _)| source); let block = match block { Some(state) => { - let Region { open, close } = self.region(state, is_arguments)?; - Some(Block { - open, - close, - ways_in: self.ways_into(state, is_arguments)?, - }) + let Region { open, close, .. } = self.region(state, is_arguments)?; + Some(Block { state, open, close }) } None => None, }; @@ -434,9 +439,13 @@ impl Format { /// The markers of the region `state` is: its close is the terminal of the row from `state` /// back to a state it is entered from, other than the end of the turn, and its opener the - /// terminal of that state's row into `state`. The states name the two, not the order of the - /// rows. + /// terminal of that state's row into `state`; that state is its home. The states name the + /// markers, not the order of the rows. The turn's own state, where the opener leaves the + /// engine, is no region: the rows back into it close the regions it holds. fn region(&self, state: usize, within: impl Fn(usize) -> bool + Copy) -> Option> { + if state == 0 { + return None; + } let from = self.entered_from(state, within); let turn_close = self.turn_close(); let (_, close, home) = self.transitions().find(|&(source, on, target)| { @@ -448,33 +457,51 @@ impl Format { Some(Region { open: self.terminal_text(open), close: self.terminal_text(close), + home, }) } /// The ways into the block `state` is, one per state it is entered from, in the order of the - /// states: the terminal of that state's row into the block, after the close of the region the - /// state is, except for the turn's own state. `None` when a state on the way has no close. - fn ways_into( - &self, - state: usize, - within: impl Fn(usize) -> bool + Copy, - ) -> Option>> { - self.entered_from(state, within) - .into_iter() - .map(|from| { - let (_, open, _) = self + /// states, the same way in once: from a region with a close, the close and then the row into + /// the block from where the close returns (the state's own row when there is none from there); + /// from any other state, its own row. + fn ways_into(&self, state: usize) -> Vec> { + let door = |from: usize| { + self.transitions() + .find(|&(source, _, target)| source == from && target == state) + .map(|(_, on, _)| self.terminal_text(on)) + }; + let mut ways: Vec> = Vec::new(); + for from in self.entered_from(state, |state| self.is_arguments(state)) { + let (close, open) = match self.region(from, |state| self.is_call_state(state)) { + Some(Region { close, home, .. }) => { + (Some(close), door(home).or_else(|| door(from))) + } + None => (None, door(from)), + }; + let Some(open) = open else { continue }; + let way = WayIn { close, open }; + if !ways.iter().any(|known| known.begin() == way.begin()) { + ways.push(way); + } + } + ways + } + + /// Whether `state` emits a call's arguments. + fn is_arguments(&self, state: usize) -> bool { + self.emits(state) == Emits::Arguments + } + + /// Whether `state` is one of the calls': a call's arguments, or the block the calls sit in, + /// the wrapper state with a row into an arguments state. A thought or an answer is a region + /// outside them, so a row into the calls is no close of its own. + fn is_call_state(&self, state: usize) -> bool { + self.is_arguments(state) + || self.emits(state) == Emits::Wrapper + && self .transitions() - .find(|&(source, _, target)| source == from && target == state)?; - let close = match from { - 0 => None, - region => Some(self.region(region, |_| false)?.close), - }; - Some(WayIn { - close, - open: self.terminal_text(open), - }) - }) - .collect() + .any(|(source, _, target)| source == state && self.is_arguments(target)) } /// The terminal that every state has a row for, each back to the turn's own state: the end of @@ -667,65 +694,121 @@ mod tests { } #[test] - fn a_thought_without_a_close_of_its_own_gives_no_grammar() { - // A thought that closes into the calls block and nowhere else: the block is entered from a - // region with no close of its own, so there is no way in that closes what the prompt left - // open, and the table gives `None` rather than a tag the model could write inside its - // thought, with the reasoning open or not. - let into_the_block = Format::new("into_the_block") - .terminal("think_open", "") - .terminal("think_close", "") - .terminal("calls_open", "") - .terminal("calls_close", "") - .terminal("call_open", "") - .terminal("call_close", "") - .state("content", Emits::Content) - .state("reasoning", Emits::Reasoning) - .state("calls", Emits::Wrapper) - .state("call", Emits::Arguments) - .transition("content", "think_open", "reasoning") - .transition("reasoning", "think_close", "calls") - .transition("content", "calls_open", "calls") - .transition("calls", "call_open", "call") - .transition("call", "call_close", "calls") - .transition("calls", "calls_close", "content") - .calls(CallSyntax::Json); + fn a_thought_that_closes_into_the_block_has_no_prefix_but_its_own_door() { + // A thought that closes into the calls block and nowhere else has no close of its own, so + // with the reasoning open the prefix path gives `None` rather than a tag the model could + // write inside its thought; without it the tag derives, and under XTML the thought's way + // in is its own row into the block, ``, since that is a path the table has. + let into_the_block = |syntax: CallSyntax| { + Format::new("into_the_block") + .terminal("think_open", "") + .terminal("think_close", "") + .terminal("calls_open", "") + .terminal("calls_close", "") + .terminal("call_open", "") + .terminal("call_close", "") + .state("content", Emits::Content) + .state("reasoning", Emits::Reasoning) + .state("calls", Emits::Wrapper) + .state("call", Emits::Arguments) + .transition("content", "think_open", "reasoning") + .transition("reasoning", "think_close", "calls") + .transition("content", "calls_open", "calls") + .transition("calls", "call_open", "call") + .transition("call", "call_close", "calls") + .transition("calls", "calls_close", "content") + .calls(syntax) + }; + let tools = weather_tools(); + assert!(into_the_block(CallSyntax::Json) + .grammar(&tools, true, false) + .is_some()); + assert!(into_the_block(CallSyntax::Json) + .grammar(&tools, true, true) + .is_none()); + let as_xtml = into_the_block(CallSyntax::Xtml) + .grammar(&tools, true, true) + .expect("a grammar") + .payload(); + assert_eq!( + as_xtml["format"]["triggers"], + value!(["", ""]) + ); + assert_eq!(begins(&as_xtml), ["", ""]); + } + + #[test] + fn a_state_the_calls_return_to_enters_the_block_by_its_own_row() { + // MiniMax M3's table under XTML: the block is entered from the turn's start, from the + // thought and from the text after it, which the block's close returns to. None of the + // three is a region with a close of its own (the text's row into the block is no close, + // the block being one of the calls' states), so each enters by its own row, and the one + // row they share makes one way in. let tools = weather_tools(); - assert!(into_the_block.grammar(&tools, true, false).is_none()); - assert!(into_the_block.grammar(&tools, true, true).is_none()); + let as_xtml = formats::minimax_m3() + .calls(CallSyntax::Xtml) + .grammar(&tools, true, true) + .expect("a grammar") + .payload(); + assert_eq!(as_xtml["format"]["triggers"], value!([""])); + assert_eq!(begins(&as_xtml), [""]); + assert_eq!(as_xtml["format"]["tags"][0]["end"], ""); + } + + /// The `begin` of each way into a block, from a `triggered_tags` payload. + fn begins(payload: &Value) -> Vec { + payload["format"]["tags"] + .as_array() + .expect("tags") + .iter() + .map(|tag| tag["begin"].as_str().expect("a begin").to_string()) + .collect() } #[test] fn a_blocks_opener_is_the_row_from_the_state_its_close_returns_to() { // The block is entered by two terminals, `` from the thought (the // first row in) and `` from content; its close returns to content, so `` is - // its opener, whichever row comes first, and DSML writes it as the block's opening. - let two_doors = Format::new("two_doors") - .terminal("think_open", "") - .terminal("think_close", "") - .terminal("from_thought", "") - .terminal("calls_open", "") - .terminal("calls_close", "") - .terminal("call_open", "") - .terminal("call_close", "") - .state("content", Emits::Content) - .state("reasoning", Emits::Reasoning) - .state("calls", Emits::Wrapper) - .state("call", Emits::Arguments) - .transition("content", "think_open", "reasoning") - .transition("reasoning", "think_close", "content") - .transition("reasoning", "from_thought", "calls") - .transition("content", "calls_open", "calls") - .transition("calls", "call_open", "call") - .transition("call", "call_close", "calls") - .transition("calls", "calls_close", "content") - .calls(CallSyntax::Dsml); + // its opener, whichever row comes first, and DSML writes it as the block's opening. The + // way in from the thought closes it, which leaves the engine in content, so it goes on + // through content's door, ``, not the thought's own row. + let two_doors = |syntax: CallSyntax| { + Format::new("two_doors") + .terminal("think_open", "") + .terminal("think_close", "") + .terminal("from_thought", "") + .terminal("calls_open", "") + .terminal("calls_close", "") + .terminal("call_open", "") + .terminal("call_close", "") + .state("content", Emits::Content) + .state("reasoning", Emits::Reasoning) + .state("calls", Emits::Wrapper) + .state("call", Emits::Arguments) + .transition("content", "think_open", "reasoning") + .transition("reasoning", "think_close", "content") + .transition("reasoning", "from_thought", "calls") + .transition("content", "calls_open", "calls") + .transition("calls", "call_open", "call") + .transition("call", "call_close", "calls") + .transition("calls", "calls_close", "content") + .calls(syntax) + }; let tools = weather_tools(); - let payload = two_doors + let as_dsml = two_doors(CallSyntax::Dsml) .grammar(&tools, true, false) .expect("a grammar") .payload(); - assert_eq!(payload["format"]["elements"][0]["value"], "\n\n\n"); + assert_eq!(as_dsml["format"]["elements"][0]["value"], "\n\n\n"); + let as_xtml = two_doors(CallSyntax::Xtml) + .grammar(&tools, true, false) + .expect("a grammar") + .payload(); + assert_eq!( + as_xtml["format"]["triggers"], + value!(["", ""]) + ); + assert_eq!(begins(&as_xtml), ["", ""]); } #[test] diff --git a/crates/symphony/src/grammar/xtml.rs b/crates/symphony/src/grammar/xtml.rs index 70f606b75d..17361cf7b2 100644 --- a/crates/symphony/src/grammar/xtml.rs +++ b/crates/symphony/src/grammar/xtml.rs @@ -31,9 +31,11 @@ const SEP: &str = "<|sep|>"; /// The call's count of the turn's calls, from one. const INDEX: &str = "[1-9][0-9]*"; -/// The block of calls for `tools`, or `None` for a table whose calls sit in no block. +/// The block of calls for `tools`, entered by `ways_in`, or `None` for a table whose calls sit +/// in no block. pub(super) fn calls( markers: &CallMarkers<'_>, + ways_in: &[WayIn<'_>], tools: &[&Tool], at_least_one: bool, ) -> Option { @@ -45,13 +47,12 @@ pub(super) fn calls( .collect(), ); let content = Grammar::Plus(Box::new(one_call)); - let tags = block - .ways_in + let tags = ways_in .iter() .map(|way| Tag::new(way.begin(), content.clone(), block.close)) .collect(); let mut triggers: Vec = Vec::new(); - for trigger in block.ways_in.iter().map(WayIn::trigger) { + for trigger in ways_in.iter().map(WayIn::trigger) { if !triggers.iter().any(|known| known == trigger) { triggers.push(trigger.to_string()); } From 307cbdf48e59a5dc21192339d623ade552fa6cb1 Mon Sep 17 00:00:00 2001 From: Simo Lin <25425177+slin1237@users.noreply.github.com> Date: Thu, 8 Oct 2026 05:22:50 -0700 Subject: [PATCH 4/4] fix(symphony): a way into the block is always a path the table has A way in from a thought or an answer paired the region's close with the home state's row into the block, and fell back to the region's own row when the home had none: `` for a table whose block is reachable from inside the thought alone, a path no row of the table walks. The close is taken only when the home has a row into the block; otherwise the state enters by its own row, the thought left open, which is the path the table has. The XTML module's doc states the final rule and points at `Format::ways_into`. Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com> Co-authored-by: Chang Su <8605658+CatherineSue@users.noreply.github.com> Co-authored-by: Keyang Ru Co-authored-by: key4ng Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com> Co-authored-by: LightSeek Foundation <243258330+lightseek-bot@users.noreply.github.com> --- crates/symphony/src/grammar.rs | 55 +++++++++++++++++++++++------ crates/symphony/src/grammar/xtml.rs | 17 ++++----- 2 files changed, 53 insertions(+), 19 deletions(-) diff --git a/crates/symphony/src/grammar.rs b/crates/symphony/src/grammar.rs index 89ddec3b60..b80f1f3d11 100644 --- a/crates/symphony/src/grammar.rs +++ b/crates/symphony/src/grammar.rs @@ -189,10 +189,10 @@ struct Block<'a> { } /// One way into a block of calls, from a state with a row into it: from a region with a close of -/// its own (a thought, an answer), that close first, then the opener of the row into the block -/// from the state the close returns to, so the model closes what the prompt left open and goes on -/// from where that leaves it; from the turn's own state, or a state with no close, the opener of -/// its own row. Every way in is a path the table has. +/// its own (a thought, an answer) whose home has a row into the block, that close first, then the +/// home's row, so the model closes what the prompt left open and goes on from where that leaves +/// it; from any other state (the turn's own, one with no close, one whose home has no row into +/// the block), the opener of its own row. Every way in is a path the table has. struct WayIn<'a> { close: Option<&'a str>, open: &'a str, @@ -462,9 +462,8 @@ impl Format { } /// The ways into the block `state` is, one per state it is entered from, in the order of the - /// states, the same way in once: from a region with a close, the close and then the row into - /// the block from where the close returns (the state's own row when there is none from there); - /// from any other state, its own row. + /// states, the same way in once: from a region with a close whose home has a row into the + /// block, the close and then that row; from any other state, its own row. fn ways_into(&self, state: usize) -> Vec> { let door = |from: usize| { self.transitions() @@ -473,10 +472,11 @@ impl Format { }; let mut ways: Vec> = Vec::new(); for from in self.entered_from(state, |state| self.is_arguments(state)) { - let (close, open) = match self.region(from, |state| self.is_call_state(state)) { - Some(Region { close, home, .. }) => { - (Some(close), door(home).or_else(|| door(from))) - } + let through_home = self + .region(from, |state| self.is_call_state(state)) + .and_then(|Region { close, home, .. }| Some((Some(close), door(home)?))); + let (close, open) = match through_home { + Some((close, open)) => (close, Some(open)), None => (None, door(from)), }; let Some(open) = open else { continue }; @@ -737,6 +737,39 @@ mod tests { assert_eq!(begins(&as_xtml), ["", ""]); } + #[test] + fn a_thought_whose_home_has_no_row_into_the_block_enters_by_its_own_row() { + // The block is reachable from inside the thought alone and closes back into it, the shape + // of a model that calls tools mid-thought. The thought has a close, but content, where the + // close returns, has no row into the block, so `` would be no path the + // table has: the way in is the thought's own row, ``, with the thought left open. + let mid_thought = Format::new("mid_thought") + .terminal("think_open", "") + .terminal("think_close", "") + .terminal("calls_open", "") + .terminal("calls_close", "") + .terminal("call_open", "") + .terminal("call_close", "") + .state("content", Emits::Content) + .state("reasoning", Emits::Reasoning) + .state("calls", Emits::Wrapper) + .state("call", Emits::Arguments) + .transition("content", "think_open", "reasoning") + .transition("reasoning", "think_close", "content") + .transition("reasoning", "calls_open", "calls") + .transition("calls", "call_open", "call") + .transition("call", "call_close", "calls") + .transition("calls", "calls_close", "reasoning") + .calls(CallSyntax::Xtml); + let payload = mid_thought + .grammar(&weather_tools(), true, true) + .expect("a grammar") + .payload(); + assert_eq!(payload["format"]["triggers"], value!([""])); + assert_eq!(begins(&payload), [""]); + assert_eq!(payload["format"]["tags"][0]["end"], ""); + } + #[test] fn a_state_the_calls_return_to_enters_the_block_by_its_own_row() { // MiniMax M3's table under XTML: the block is entered from the turn's start, from the diff --git a/crates/symphony/src/grammar/xtml.rs b/crates/symphony/src/grammar/xtml.rs index 17361cf7b2..47312911d5 100644 --- a/crates/symphony/src/grammar/xtml.rs +++ b/crates/symphony/src/grammar/xtml.rs @@ -1,14 +1,15 @@ //! Kimi K3's structural tag: the block of calls as the gateway's Kimi K3 parser writes it by hand, //! here from the table's markers and the request's tools. //! -//! The table gives the block its opener and close and the ways into it: Kimi K3's prompt leaves -//! the thought open in thinking mode and the answer otherwise, and a block the model could not -//! reach from there would force the calls inside a region the model can never close. So the tag -//! has one way in per state the block is entered from, that state's own close followed by the -//! opener, each closed by the block's close, and each way's first text (the opener from the turn -//! itself, the close from the thought or the answer) triggers it. Inside, one -//! or more calls, each `<|open|>call tool="NAME" index="N"<|sep|>`, its arguments and the call's -//! close, with the name as the request gives it (`&` and `"` escaped as the template does). +//! The table gives the block its close and its ways in (`Format::ways_into`): Kimi K3's prompt +//! leaves the thought open in thinking mode and the answer otherwise, and a block the model could +//! not reach from there would force the calls inside a region the model can never close. So the +//! tag has one way in per state the block is entered from, each a path the table has: from the +//! thought or the answer, its close and then the row into the block from where the close returns; +//! from the turn itself, its own row. Each way is closed by the block's close, and its first text +//! triggers the tag. Inside, one or more calls, each `<|open|>call tool="NAME" index="N"<|sep|>`, +//! its arguments and the call's close, with the name as the request gives it (`&` and `"` escaped +//! as the template does). //! //! The arguments follow the tool's schema: one argument tag per property, in the schema's order, //! each optional unless the schema requires it, its `type` attribute and value pinned by the