fully deprecate old data generation system

.coderabbit.yaml

@@ -34,7 +34,7 @@ reviews:
     - path: "examples/**/*.py"
       instructions: "Review for clarity, correctness, and educational value. Ensure examples are end-to-end runnable, configurations match current API, and comments explain speculative decoding-specific concepts for users new to the algorithm."
     - path: "scripts/**/*.py"
-      instructions: "Check that scripts handle argument parsing robustly, log progress clearly, and are safe to run in multi-GPU environments. Verify that gen_and_train.py pipeline orchestration correctly sequences data generation before training."
+      instructions: "Check that scripts handle argument parsing robustly, log progress clearly, and are safe to run in multi-GPU environments."
     - path: "docs/**/*.md"
       instructions: "Check for clarity, accuracy, and completeness. Ensure code examples match the current API and that speculative decoding concepts are explained correctly."
     - path: "**/README.md"

README.md

@@ -167,14 +167,6 @@ The following table summarizes the models that have been trained end-to-end by o
 
 ✅ = Supported, ⏳ = In Progress, ❌ = Not Yet Supported
 
-## Examples
-
-End-To-End Training Examples:
-
-- [Train Llama3 Draft Model](https://github.com/vllm-project/speculators/blob/main/examples/data_generation_and_training/llama3_8b_sharegpt_5k.py)
-- [Train Qwen3 (Non-MoE) Draft Model](https://github.com/vllm-project/speculators/blob/main/examples/data_generation_and_training/qwen3_8b_sharegpt_ultrachat.py)
-- [Train GPT-OSS Draft Model](https://github.com/vllm-project/speculators/blob/main/examples/data_generation_and_training/gpt_oss_20b_ultrachat_5k.py)
-
 ## vLLM Inference
 
 Models trained through Speculators can run seamlessly in vLLM using a simple `vllm serve <speculator_model>` command. This will run the model in vLLM using default arguments, defined in the `speculator_config` of the model's config.json.

docs/examples/index.md

@@ -8,14 +8,6 @@ Welcome to the Examples section of Speculators! This area provides end-to-end ex
 
 <div class="grid cards" markdown>
 
-- :octicons-ai-model-16:{ .lg .middle } Train
-
-  ______________________________________________________________________
-
-  End-to-end example of using Speculators to train a speculative decoding model.
-
-  [:octicons-arrow-right-24: Train](data_generation_and_training.md)
-
 - :octicons-arrow-switch-16:{ .lg .middle } Convert
 
   ______________________________________________________________________

docs/index.md

@@ -39,5 +39,5 @@ Behind the scenes, this is reading the model from Hugging Face, parsing the `spe
 
 To create a speculative decoding model for a different verifier model there are two approaches you can choose:
 
-1. Train a new speculative decoding model ([instructions](train.md))([examples](examples/data_generation_and_training.md)).
+1. Train a new speculative decoding model ([instructions](train.md)).
 2. Convert an existing model from a third-party library to the Speculators format for easy deployment with vLLM ([instructions](convert.md)) ([examples](examples/convert.md)).

docs/scripts/gen_files.py

@@ -139,12 +139,6 @@ def migrate_developer_docs():
             weight=-12,
         ),
         # Examples
-        ProcessFile(
-            root_path=Path("examples/data_generation_and_training/README.md"),
-            docs_path=Path("examples/data_generation_and_training.md"),
-            title="Train",
-            weight=1,
-        ),
         ProcessFile(
             root_path=Path("examples/convert/README.md"),
             docs_path=Path("examples/convert.md"),
@@ -158,7 +152,7 @@ def migrate_developer_docs():
             weight=4,
         ),
         ProcessFile(
-            root_path=Path("scripts/README.md"),
+            root_path=Path("examples/ONLINE_TRAINING.md"),
             docs_path=Path("train.md"),
             title="Train",
             weight=-8,

pyproject.toml

@@ -111,8 +111,6 @@ dev = [
     "mkdocs-linkcheck~=1.0.6",
 ]
 
-datagen = ["vllm>=0.12.0,<=0.16.0"]
-
 [project.entry-points.console_scripts]
 speculators = "speculators.__main__:app"
 
@@ -258,9 +256,6 @@ select = [
     "INP001", # allow implicit namespace packages in examples
 ]
 
-"scripts/gen_and_train.py" = [
-    "T201", # allow print in scripts
-]
 
 [tool.ruff.lint.isort]
 known-first-party = ["speculators", "tests"]