feat(orchestrator): task sampler skeleton - #3262
Merged
hallerite merged 2 commits intoAug 13, 2026
Merged
Conversation
This was referenced Aug 13, 2026
TrainSource becomes TaskSampler: same pick contract to the dispatcher
(weighted-RR env, epoch-shuffled cursor — behavior identical), plus a memory.
Every finalized train group now flows back through TrainSink's new
on_group_finalized hook into TaskStats: discounted success/failure
pseudo-counts (a Beta posterior with forgetting) keyed by
(env, content-hash task key, agent role), with reward EMAs alongside.
Always-on observability, nothing enforces: new sampler/{env}/* metrics report
pool occupancy (learnable/saturated/hopeless/unseen from the posterior),
coverage, realized signal rate, and the wasted-token fraction — the evidence
base for the upcoming weighted-sampling stage. Stats ride inside the existing
"train_source" checkpoint key; older checkpoints load with empty stats and
warm back up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hallerite
force-pushed
the
feat/task-sampler-skeleton
branch
from
August 13, 2026 13:32
b1107dc to
04b5cfe
Compare
The filter subsystem served three detectors; each now has a proper home and
the pipeline itself (pre/post batch slots, double apply pass, RolloutFilter)
is gone:
- Zero-advantage splits into its three roles. Evidence: every finalized group
already feeds the task sampler. Admission: [orchestrator.sampler]
drop_degenerate_groups drops zero-signal groups at group finalization so
the batch backfills (the old pre-batch enforce + oversampling recipe).
Ship rule: Algorithm.trains_on_zero_advantage — a class declaration, not a
run-level knob. MaxRL's all-zero sentinel stops depending on a filter being
configured; echo's exception (observation CE trains through collapsed
advantages) becomes automatic and per-env instead of a global enforce=false.
- Gibberish/repetition demote to arrival-time detections ([[orchestrator.detections]]):
each reads only the rollout's own branches, so one pass at tokenization
replaces both slots. One drop semantic replaces pre-vs-post: an enforced
detection ships no samples and never occupies a batch slot (backfill), while
the reward still counts toward the group baseline.
Rollout.filter_results/is_filtered become detections/is_excluded;
pre_filters/* metrics become detections/*; degenerate-group drops surface as
sampler/{env}/dropped_degenerate_groups. Default behavior is preserved: the
old post-batch enforcing zero_advantage maps 1:1 onto the ship rule, and
gibberish/repetition still default to monitoring. Tracked configs migrated in
place, no shim. One deliberate metrics change: zero-advantage rollouts now
stay in the effective subset — an all-correct group is a real outcome, and
excluding it clipped reward stats.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The TaskSampler foundation in one PR, three commits: free the name, build the memory, retire the filter pipeline. Picks are byte-for-byte identical to main by default — this is the observability half of the curriculum design (
design/task-sampler.md): estimates get warm and visible before anything is allowed to steer sampling.1.
chore(orchestrator): rename Sampler to RolloutSource— pure rename. The class is one env's rollout source (which model generates its train rollouts); its docstring already said so. Frees "sampler" for the task-sampling layer.2.
feat(orchestrator): TaskSampler skeleton with per-task outcome stats—TrainSourcebecomesTaskSampler: same pick contract to the dispatcher, plus a memory.task_stats.py: discounted success/failure counts (Beta posterior with forgetting) keyed by(env, content-hash task key, agent role), plus reward mean/std EMAs. Keys recompute from the trace echo, so stats survive dataset reordering and resumes. Nothing is evicted — estimates decay back toward the prior.TrainSink.on_group_finalized: one callback per finalized train group (all-errored ones included). Sink stays sampler-blind; sampler stays token-blind.sampler/{env}/*metrics: pool occupancy (learnable/saturated/hopeless/unseen),coverage,realized_signal_rate,wasted_token_frac(inference tokens in zero-signal groups — the number later enforcement has to beat)."train_source"key; older checkpoints load with empty stats and warm back up.3.
feat(orchestrator): retire the filter pipeline— the filter subsystem served three detectors; each gets a proper home and the pipeline (both config slots, doubleapply_filterspass,RolloutFilterprotocol) is deleted. Group finalization becomes the pipeline's single decision point.[orchestrator.sampler] drop_degenerate_groups = truedrops zero-signal groups so the batch backfills — the old pre-batch enforce + oversampling recipe), and the ship rule (Algorithm.trains_on_zero_advantage, defaultFalse, echoTrue— MaxRL's all-zero sentinel stops depending on a filter being configured;advantages=Nonealways ships).[[orchestrator.detections]]), evaluated at group finalization. An enforced detection ships no samples and never occupies a batch slot; its reward still counts toward the group baseline.Rollout.filter_results/is_filtered→detections/is_excluded;pre_filters/*metrics →detections/*; five tracked configs migrated, no shim. Defaults map 1:1. One deliberate metrics change: zero-advantage rollouts stay in theeffectivesubset (excluding all-correct groups clipped reward stats).Overlaps in intent with #3254 (independently converged on
trains_on_zero_advantage, killing the filter slots, and drop-with-backfill); diverges on configurability, drop granularity, and the evidence wire — reconciliation discussed there.Verified: full non-GPU unit suite passes at each commit (143 orchestrator+config tests at the tip, incl. 5 new
task_statstests and the rewrittentest_detections.py); known-environmental failures (qwen3_vl_e2e,test_load_configs,reverse_text) unaffected.🤖 Generated with Claude Code