feat(orchestrator): add curriculum examples

[hallerite]

Stacked on #3261. This PR adds only two small policies on the user-space curriculum surface:

• DifficultyPools: for finite tasksets, sample every task once, track each task's latest group-mean reward by stable Task.key, choose a nonempty named pool by its configured weight, then sample uniformly within the pool. RNG and pool state round-trip through checkpoints.
• AdvantageRangeGate: reject a group when all trainable-token advantages fall inside [reject_min, reject_max]. The default [0, 0] implements zero-advantage rejection; algorithms without an advantage stream pass through.

The existing dynamic-sampling wiki-search and Intellect configs now select AdvantageRangeGate per source. No weighting framework, posterior estimator, built-in component registry, rejection sampler, exploration floor, or custom mini-protocol is introduced—the examples are ordinary Curriculum subclasses intended to be copied or extended in user code.

Validation

• uv run ruff format src/prime_rl/orchestrator/curricula.py tests/unit/orchestrator/test_curriculum.py
• uv run ruff check src/prime_rl/orchestrator/curricula.py tests/unit/orchestrator/test_curriculum.py
• uv run pytest tests/unit/orchestrator --ignore=tests/unit/orchestrator/test_qwen3_vl_e2e.py -q — 79 passed
• updated TOML configs parsed with tomllib
• git diff --check

---

Note

Medium Risk
AdvantageRangeGate changes which groups train and increases resampling under oversampling; mis-tuned reject ranges could starve batches. DifficultyPools only affects task selection on finite tasksets.

Overview
Adds built-in curriculum policies in prime_rl.orchestrator.curricula and wires the advantage gate into example training configs.

DifficultyPools changes finite-task sampling after each task has been seen once: it keeps the latest mean group reward per task, buckets tasks into named pools via thresholds, then picks a pool by weights and a task uniformly inside it. Unseen tasks are still drawn first. Pool occupancy and checkpoint state (task_rewards) are exposed through metrics and state_dict.

AdvantageRangeGate changes which rollout groups enter the training batch: on_result rejects a group when every trainable-token advantage lies in [reject_min, reject_max] (default [0, 0] drops all-zero-advantage groups). Groups with no advantage stream are still admitted.

docs/algorithms.md documents both with TOML examples. Wiki-search and intellect-3.1 example sources (plus CI nightly wiki-search) now set curriculum = { import_path = "prime_rl.orchestrator.curricula.AdvantageRangeGate" }. Unit tests cover pool metrics/checkpoint resume and gate behavior (tolerance band, trainable mask).

^{Reviewed by Cursor Bugbot for commit 825f622. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 825f622. Configure here.}

[cursor]

Advantage gate hangs group_size=1

High Severity

Enabling default AdvantageRangeGate on intellect-3.1 with unset group_size (defaults to 1) makes every GRPO group have zero advantages, so every group is rejected. Rejected groups never enter pending_batch, so the train sink never fills a batch and training hangs without hitting the empty-batch abort.

Additional Locations (2)

• examples/advanced/intellect-3.1/rl.toml#L67-L68
• examples/advanced/intellect-3.1/rl.toml#L84-L85

 

^{Reviewed by Cursor Bugbot for commit 825f622. Configure here.}

[cursor]

Failed tasks starve pool sampling

Medium Severity

DifficultyPools only records a task after a successful reward, so permanently failing tasks stay in the unseen set. Once every healthy task has been seen, sample only returns those failing tasks. Their groups never enter pending_batch, so training can hang.

 

^{Reviewed by Cursor Bugbot for commit 825f622. Configure here.}

[hallerite]

Superseded by #3261, which now contains the example implementations and the sampler-plus-admission-gates composition model.