Skip to content

Reuse identical converter rules - #735

Closed
Ubospica wants to merge 5 commits into
mlc-ai:mainfrom
Ubospica:perf/converter-rule-cache
Closed

Ubospica wants to merge 5 commits into
mlc-ai:mainfrom
Ubospica:perf/converter-rule-cache

Conversation

@Ubospica

@Ubospica Ubospica commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

依赖

本拉取请求基于 #726 的最新提交 823f682a。在 #726 合并前,本拉取请求包含其提交;缓存与结构化标签转换改动位于后续提交中。

改动

  • 补全 JSON Schema(用于描述 JSON 数据结构的规格)规则缓存:普通子规格生成后立即登记规则,后续相同规格直接复用;引用目标也会先查询缓存,并在递归生成前登记规则。
  • 缓存键纳入会影响语法的上下文。包含对象或数组的规格按缩进层级区分;XML(可扩展标记语言)工具调用格式按外层、第一层和内部 JSON 三种层级区分,避免错误复用。
  • 结构化标签转换器原本已经按完整格式的序列化结果复用规则编号,本次保留该机制。其 JSON Schema 子语法现在直接加入语法树,省去打印为扩展巴科斯范式(Extended Backus–Naur Form,EBNF)文本后再次解析的过程。
  • 新增重复规格、重复引用目标、跨缩进层级、XML 嵌套上下文和结构化标签格式复用测试。

性能

使用发布构建,先预热一次,再独立运行五次并取中位数。基线为 #726be0afe61,缓存变更为 9da247e6;最终提交 71fd0b52 将同一缓存逻辑适配到 #726 最新的直接 GrammarBuilder 实现。GrammarBuilder 是代码内部的语法树构造器。JSONSchemaBench 是包含约一万个真实 JSON Schema 的基准数据集;CloudFormationKubernetesService Schema 是其中三个大型真实规格集合的名称。规则数指规范化后的语法规则数量。

工作负载 输入大小 #726 当前 加速比 规则数
人工构造:5,000 个属性复用同一复杂对象规格 3.07 MB 1573.965 ms 404.128 ms 3.89x 225,020 → 5,064
人工构造:3,000 个前缀数组项复用同一复杂对象规格 1.80 MB 903.244 ms 147.690 ms 6.12x 132,020 → 64
结构化标签:包含上述 5,000 属性规格 3.07 MB 5012.722 ms 518.935 ms 9.66x 225,021 → 5,065
JSONSchemaBench:CloudFormation 6.79 MB 599.212 ms 515.074 ms 1.16x 66,671 → 43,958
JSONSchemaBench:Kubernetes 1.28 MB 41.085 ms 39.032 ms 1.05x 6,648 → 4,494
JSONSchemaBench:Service Schema 1.01 MB 80.940 ms 75.230 ms 1.08x 7,873 → 4,829

高重复人工规格的转换时间降低 74%–90%,规则数降低到原来的约 0.05%–2.25%。三个大型真实规格的提升为 1.05–1.16 倍,同时规则数降低 32%–39%。

验证

  • 738 项 JSON Schema、函数调用格式与语法解析 Python 测试
  • 1,025 项结构化标签 Python 测试
  • 所有改动文件通过格式和静态检查

Ubospica added 5 commits July 23, 2026 01:45
Avoid rebuilding equivalent JSON schema rules and keep structural JSON subgrammars on the direct AST path.
Adapt rule caching to the direct GrammarBuilder implementation from the updated dependency branch.
@Ubospica
Ubospica marked this pull request as ready for review July 30, 2026 16:01
Copilot AI review requested due to automatic review settings July 30, 2026 16:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants