Change default vllm_mode to "colocate" and add v0→v1 migration guide

[qgallouedec]

Summary

• Default vllm_mode changed from "server" to "colocate" across all configs (GRPOConfig, RLOOConfig, GOLDConfig, OnlineDPOConfig). Colocate is a better default — it works out of the box without spinning up a separate vLLM server, which is the most common friction point for new users. Users who need server mode can still set vllm_mode="server" explicitly.
• Updated docs and tests that implicitly assumed the old "server" default to explicitly set vllm_mode="server" where needed (vllm_integration.md, grpo_trainer.md, rloo_trainer.md, speeding_up_training.md, test_online_dpo_trainer.py).
• Added a minimal MIGRATION.md at the repo root for v0 → v1. It should stay small — we have already done most of the breaking work in v0.29 (trainers moved to experimental, model classes removed, etc.), so the guide just highlights what is genuinely new and keeps the v0.29 changes in collapsible sections for anyone migrating from an older version.

---

Note

Medium Risk
Changes the default vLLM execution mode from external server to in-process across multiple configs, which can alter runtime behavior and GPU memory requirements for existing training setups.

Overview
Switches the default vLLM integration mode to "colocate" (from "server") across GRPOConfig, RLOOConfig, OnlineDPOConfig, GOLDConfig, and the VLLMGeneration backend.

Docs and examples are updated to reflect the new default and to explicitly set vllm_mode="server" where server-based workflows are described (grpo_trainer.md, rloo_trainer.md, speeding_up_training.md, vllm_integration.md). Tests are adjusted to pin server-mode behavior and to assert the new default (test_online_dpo_trainer.py).

Adds a new top-level MIGRATION.md highlighting v0→v1 breaking changes, including the vllm_mode default change and an SFTConfig.packing rename.

^{Written by Cursor Bugbot for commit d8a0872. This will update automatically on new commits. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 424fc21035

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[albertvillanova]

Thanks. A couple of questions below.

[albertvillanova]

Thanks for your explanation. Just a comment below.