Skip to content

docs: correct reverse KL definition in OPD guide - #2254

Merged
zhuzilin merged 2 commits into
THUDM:mainfrom
XiXiphus:docs/fix-opd-reverse-kl
Aug 12, 2026
Merged

zhuzilin merged 2 commits into
THUDM:mainfrom
XiXiphus:docs/fix-opd-reverse-kl

Conversation

@XiXiphus

@XiXiphus XiXiphus commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • correct the reverse-KL direction to D_KL(student || teacher) and make the student-sampled expectation explicit, following the Thinking Machines Lab definition
  • distinguish the full token-distribution KL from slime’s per-sampled-token Monte Carlo contribution and show how it modifies the base advantage
  • clarify pure-distillation behavior and the tokenizer/vocabulary compatibility requirement for an external SGLang teacher
  • keep the English and Chinese guides aligned

Validation

  • bash docs/build.sh en (succeeded; only pre-existing warnings elsewhere in the docs)
  • bash docs/build.sh zh (succeeded; only pre-existing warnings elsewhere in the docs)
  • uvx pre-commit run --files docs/en/advanced/on-policy-distillation.md docs/zh/advanced/on-policy-distillation.md

@zhuzilin
zhuzilin merged commit 351e559 into THUDM:main Aug 12, 2026
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants