Repository navigation

[trainer, fsdp, worker, algo, cfg, recipe] feat: add reverse-KL on-policy distillation with FSDP teacher - #6920

Open

YangxSong wants to merge 17 commits into

verl-project:mainfrom

YangxSong:feature/reverse-kl-opd

Open

[trainer, fsdp, worker, algo, cfg, recipe] feat: add reverse-KL on-policy distillation with FSDP teacher#6920
YangxSong wants to merge 17 commits into
verl-project:mainfrom
YangxSong:feature/reverse-kl-opd

Commits on Jul 1, 2026
- [distillation] feat: add fsdp teacher backend hooks to DistillationConfig (verl#6676)
  
  YangxSong
  and
  claude
  committed
  1c0dc0c
- [distillation] feat: add student-top-K reverse-KL loss (verl#6676)
  
  YangxSong
  and
  claude
  committed
  9387fda
- [distillation] feat: add forward-only FSDP teacher worker (verl#6676)
  
  YangxSong
  and
  claude
  committed
  7a4ae8f
- [distillation] feat: wire FSDP teacher into trainer main loop (verl#6676)
  
  YangxSong
  and
  claude
  committed
  f69157a
- [examples] feat: add reverse-KL FSDP-teacher distillation script (verl#6676)
  
  YangxSong
  and
  claude
  committed
  be035df
- [distillation] fix: FSDP teacher pipeline glue and config knobs (verl#6676)
  
  YangxSong
  and
  claude
  committed
  0c3a90f
- [distillation] fix: align FSDP teacher topk fields through left_right padding (verl#6676)
  YangxSong
  committed
  3859df0
- [distillation] feat: add student/teacher top-K overlap diagnostics (verl#6676)
  
  YangxSong
  and
  claude
  committed
  a70ad11
- [distillation] fix: renormalize reverse-KL onto student top-K support (verl#6676)
  YangxSong
  committed
  6a9ecb6
- [distillation] fix: query FSDP teacher at the student rollout temperature (verl#6676)
  YangxSong
  committed
  2cbe9f4
- Merge branch 'main' into feature/reverse-kl-opd
  YangxSong
  committed
  c93492a
Commits on Jul 2, 2026
Commits on Jul 6, 2026
- Merge remote-tracking branch 'origin/main' into feature/reverse-kl-opd
  YangxSong
  committed
  ed3e062
- fix: add use_vllm_teacher attribute to agent_loop test mock
  YangxSong
  committed
  87cd055

Navigation Menu

Repository navigation

[trainer, fsdp, worker, algo, cfg, recipe] feat: add reverse-KL on-policy distillation with FSDP teacher - #6920

[trainer, fsdp, worker, algo, cfg, recipe] feat: add reverse-KL on-policy distillation with FSDP teacher#6920
YangxSong wants to merge 17 commits into
verl-project:mainfrom
YangxSong:feature/reverse-kl-opd

Commits on Jul 1, 2026

[distillation] feat: add fsdp teacher backend hooks to DistillationConfig (verl#6676)

[distillation] feat: add student-top-K reverse-KL loss (verl#6676)

[distillation] feat: add forward-only FSDP teacher worker (verl#6676)

[distillation] feat: wire FSDP teacher into trainer main loop (verl#6676)

[examples] feat: add reverse-KL FSDP-teacher distillation script (verl#6676)

[distillation] fix: FSDP teacher pipeline glue and config knobs (verl#6676)

[distillation] fix: align FSDP teacher topk fields through left_right padding (verl#6676)

[distillation] feat: add student/teacher top-K overlap diagnostics (verl#6676)

[distillation] fix: renormalize reverse-KL onto student top-K support (verl#6676)

[distillation] fix: query FSDP teacher at the student rollout temperature (verl#6676)

Merge branch 'main' into feature/reverse-kl-opd

Commits on Jul 2, 2026

fix: relax shape assert to check only first dimension in top-K path

feat: add reverse-KL FSDP example for DSR1 1.5B reproducing Rethinking OPD

style: apply pre-commit formatting fixes

Merge remote-tracking branch 'origin/main' into feature/reverse-kl-opd

Commits on Jul 6, 2026

Merge remote-tracking branch 'origin/main' into feature/reverse-kl-opd

fix: add use_vllm_teacher attribute to agent_loop test mock