feat: opt-in renderer-based tokenization for SFT

[ercbot]

Closes #2470

Adds use_renderer flag to SFTConfig, mirroring the RL path added in #2278. When enabled, SFTDataset tokenizes via renderers.base.build_training_sample (single render() + message_indices mask) instead of the incremental Jinja template path.

Using renderers for tokenization fixes multiturn sample drops and the Qwen3.5 system-only TemplateError crash for chat templates that render position-dependently.

---

Note

Medium Risk
Changes the SFT data/tokenization path and adds new config validation around renderer usage; misconfiguration could cause training to fail early or change which tokens are masked for loss.

Overview
Adds use_renderer + renderer configuration to SFT, including validation to forbid unsupported combos (e.g. VLMs) and to error when renderer-only knobs are set without enabling the feature.

When enabled, SFT now creates a renderers-based tokenizer (create_renderer) and routes dataset tokenization through build_training_sample, passing the renderer through setup_dataset for both train and validation; it also warns once when per-example chat_template_kwargs would be ignored and rejects falling back to DefaultRenderer.

^{Reviewed by Cursor Bugbot for commit 9bb1180. Bugbot is set up for automated code reviews on this repo. Configure here.}

[ercbot]

Adding renderer arg validation across orch and sft configs causes minor code duplication code. But there doesn't seem to be an existing pattern to make that common easily.

[ercbot]

Ran existing unit tests:

Also ran checked the two multiturn sft training issues I mentioned in #2470 can now be avoided by with --use_renderer

uv run sft @ examples/wordle/sft.toml --model.name Qwen/Qwen3-0.6B --tokenizer.name Qwen/Qwen3-0.6B --max_steps 1 --use_renderer  --renderer.name qwen3 --clean-output-dir

uv run sft @ examples/wordle/sft.toml --model.name Qwen/Qwen3.5-0.8B --tokenizer.name Qwen/Qwen3.5-0.8B --max_steps 1  --use_renderer --renderer.name qwen3.5 --clean-output-dir

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 8104505. Configure here.}

[hallerite]

thank you @ercbot !

[hallerite]

Approved — renderer-based SFT tokenization fixes the multi-turn / Qwen3.5 TemplateError bug as advertised, validators correctly gate VLM and renderer-args misconfig, DefaultRenderer fallback is refused with a helpful error, and renderer.pool_size is rejected for SFT (synchronous tokenization, one renderer per DP rank — no in-process pool needed).

Verified end-to-end on Qwen/Qwen3-0.6B + PrimeIntellect/Reverse-Text-SFT for 50 steps: training loop completes cleanly, loss descends monotonically, grad norms track the baseline path.

One follow-up tracked separately: the renderer path currently trains on <|im_start|>assistant\n scaffolding because Qwen3Renderer._render_assistant attributes the role opener to the assistant message via message_indices. RL avoids this via its structural prompt/completion split; SFT (single-render + build_training_sample) doesn't. PrimeIntellect-ai/renderers#33 just merged adding RenderedTokens.sampled_mask to fix this at the right layer; tracking the prime-rl-side bump as a follow-up issue. Doesn't block this PR — it lands the bug-fix that's actually requested.

[hallerite]

@ercbot quick heads-up: CI on #2471 fails because GitHub doesn't expose org secrets to PRs from forks, so wandb crashes the trainer process. I pushed the same head SHA to a different branch and opened #2493 as a mirror so CI can run properly.

I'll make sure to merge that one with --author="Eric Botti <ericjbotti@gmail.com>" so you stay the primary author on the resulting commit :)