Skip to content

Avoid structural tag cache key serialization - #785

Closed
Ubospica wants to merge 1 commit into
mlc-ai:mainfrom
Ubospica:codex/perf-structural-cache-key
Closed

Ubospica wants to merge 1 commit into
mlc-ai:mainfrom
Ubospica:codex/perf-structural-cache-key

Conversation

@Ubospica

@Ubospica Ubospica commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

改动

XGrammar 是本仓库的语法约束生成库。缓存键是查找可复用编译结果时使用的标识。本改动直接使用已经规范化的 JSON 结构描述作为缓存键,避免再次解析、排序和序列化同一结构。

改动只影响结构化标签内部缓存,不改变公开接口。

性能与限制

固定 400 函数的阶段计时中,结构化标签转换从 96.199 毫秒降至 76.790 毫秒,减少 20.2%。后续完整数据集对照中,大型和小型结构化标签编译只减少 1.0% 和 3.0%;大型 JSON 结构描述增加 5.3%,但该入口不直接使用目标结构化标签缓存。

目标阶段曾有明确收益,但最新端到端数据没有证明广泛收益。评审重点是确认实际缓存命中场景是否足以采用这项改动。

正确性

  • 四类正式输入的优化前后接受结果差异为 0。
  • 对应非可编辑安装运行结构化标签测试:7 项通过,12 项因需要额外模型词表而未执行。

@Ubospica Ubospica changed the title [codex] Avoid structural tag cache key serialization Avoid structural tag cache key serialization Jul 30, 2026
@Ubospica
Ubospica marked this pull request as ready for review July 30, 2026 15:38
Copilot AI review requested due to automatic review settings July 30, 2026 15:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants