Skip to content

metrics(async): report selected policy provenance - #3334

Merged
Shi-Dong merged 3 commits into
radixark:mainfrom
modal-projects:feat/async-policy-provenance
Sep 22, 2026
Merged

Shi-Dong merged 3 commits into
radixark:mainfrom
modal-projects:feat/async-policy-provenance

Conversation

@nanjiangwill

@nanjiangwill nanjiangwill commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Contract

Fully asynchronous training needs policy-provenance metrics for the data that actually reaches training. Missing version metadata must remain visible rather than becoming implicit zero-lag data.

Changes

  • make avg_staleness and max_staleness cover only groups selected for training
  • decompose selected-group staleness into post-generation staleness and the version span accumulated during generation
  • report token-weighted staleness and selected-sample version coverage
  • document that staleness is measured in published rollout weight versions, not necessarily optimizer steps

The staleness filter is unchanged: it still compares the current rollout weight version with the oldest numeric version in the group. stale_groups_filtered remains the rejected-group signal.

Cost

This reuses the WeightVersionSpan metadata already present on every sample. The existing group-version scan now also accumulates the newest version, token-weighted version, and sample coverage. It adds no RPC, payload field, per-token loop, CUDA synchronization, collective, or model forward.

Testing

53 passed in tests/fast/rollout/test_filters.py and tests/fast/rollout/test_fully_async_rollout.py
black: passed
isort: passed
ruff: passed

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@Shi-Dong

Copy link
Copy Markdown
Collaborator

@claude review always

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread miles/rollout/fully_async_data_buffer.py Outdated

@Shi-Dong Shi-Dong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed with local execution at c51b92c: 54/54 tests across both suites; the rescoped staleness metrics verified with an exact fail-before (base counts rejected groups, head does not); the generation-span/current_version=None finding was confirmed by probe on the previous head and the same probe now passes after the fix commit. Filter semantics are unchanged (delegation verified equal to the base implementation across nested/non-numeric/empty shapes).

@Shi-Dong
Shi-Dong merged commit 8cb0242 into radixark:main Sep 22, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants