You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Resolves AISIM-1893 and AISIM-1894 (Dynamo side). AISIM-1894's shared-G2 qualification was folded in from #17879, since it only adds tests and docs for the behavior enabled here. AISimulate counterpart: ai-dynamo/aisimulate#688.
validate_agentic_host_offload() limited offline AgentX G2 to one aggregated worker or 1P1D. This PR reuses the existing multi-worker G2 and HostPinned event routing, with no AgentX-specific residency path:
Guard: any number of aggregated, prefill, or decode workers is now accepted. vLLM, attention DP=1, static pools, no speculation, and no G3 remain required.
Event validity: the offline KV router mirrors lower-tier residency per rank. It rejects a G2 store for a block that rank already holds, and a removal for a block it never stored. Offline replay delivers native events exactly once and in order, and kv_event_lag_ms delays batches FIFO, so these cases mean a duplicated, lost, or reordered event. Replay now fails instead of letting the router view drift from the engine. All 311 existing mocker tests pass with this check enabled.
Ordering: G2 events emitted at a timestamp (pass completion or offload tick) are applied before arrivals at that timestamp are placed. Lagged events become visible at exactly t + lag.
Tests (deterministic, CanonicalV1)
Router unit tests:
A HostPinned store moves placement only once it is visible. Arrivals at 5 and 14.999 ms place without it; arrivals at 15 and 50 ms place on its owner.
Duplicated, lost, or reordered G2 events are rejected. Cleared events and worker removal reset the mirror.
End-to-end, two replicas:
Without G2, or with every event hidden by lag, resume follows load and recomputes.
With G2 events visible, resume moves to the private G2 owner and restores 8 tokens.
A cluster_shared pool is visible to every replica, and load picks among them.
Router/engine reconciliation at each admission: the reported overlap equals realized G1 + G2 reuse, with host_cache_hit_weight=1.0.
Recycled snapshot plays never share G2 identity.
Reruns are identical.
Shared-G2 qualification (AISIM-1894)
Shared hits stay within the play that stored them. Across two snapshot lanes, each sibling conversation restores 8 tokens of its own play's prefix. Each play has exactly one cache_id, and two lanes occupy exactly 2× the single-lane pool residency.
Concurrent consumers. Two replicas restore the same shared prefix at the same time. Each H2D wait is exactly 2× the solo wait (the pool's 1 GB/s cap split evenly), g2_domains matches the solo run, and reruns are identical.
Stale entry. Placement scores 8 G2 tokens visible at arrival; the 2-block pool evicts them while the request is queued, and admission recomputes the prefix without failing the event-stream check.
Disabling sharing.cluster_shared config with scope=dp_rank_local reproduces the private baseline for 1 and 2 workers.
The reference doc also describes the shared-pool contract: ownership, discovery, visibility, eviction, contention, stale-entry fallback, membership, and disable semantics. Worker failure and ownership handoff are outside the boundary because AgentX G2 requires static pools.
Not in this PR
Dynamo's Python replay API goes through AISimulate's sweeper gate, which still enforces the old limit until a nightly with ai-dynamo/aisimulate#688 is published. A follow-up will bump the nightly and add a Python multi-replica AgentX test.
Validation
cargo test -p dynamo-mocker --lib passes (311 before the shared-G2 tests; agentic_multi_replica: 9 passed). cargo clippy -p dynamo-mocker --all-targets -D warnings is clean, and pre-commit passes.
No actionable comments were generated in the recent review.
Walkthrough
The replay validator now permits multi-worker AgentX G2 host-offload configurations while preserving role restrictions. The Router validates lower-tier KV residency and uses visible events for placement. Tests cover multi-replica placement, event timing, cache identity, and deterministic reports. The CLI reference documents these behaviors.
Validation no longer restricts G2 role worker counts. Existing checks for unsupported roles, scaling policies, and deployment configurations remain. The reference documents the expanded topology support.
Validate lower-tier residency and visibility lib/mocker/src/replay/offline/extensions/kv_router/mod.rs, docs/fern/pages/reference/components/dynosim-replay-cli-reference.mdx
The Router tracks lower-tier residency by storage tier and worker. It rejects duplicate stores and removals for nonresident blocks before forwarding events. Tests cover residency changes and delayed event visibility.
Replay tests cover local and shared G2 placement, disaggregated placement, recycled-play cache identities, and deterministic reports. The reference describes play-local token namespaces and per-request routing history.
No actionable issue remains before merge based on the available evidence.
Pre-merge checks: 4 · 1
Failed checks: 1 warning
Check name
Status
Explanation
Resolution
Docstring Coverage
Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…
Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed checks: 4 passed
Check name
Status
Explanation
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Title check
The title clearly identifies the main change: routing multi-replica AgentX G2 from native KV events.
Description check
The description is detailed and covers the overview, key changes, validation, tests, and ongoing work. It uses different headings from the template and omits the checklist, but the required informatio…
Full details: Description check
Explanation
The description is detailed and covers the overview, key changes, validation, tests, and ongoing work. It uses different headings from the template and omits the checklist, but the required information is mostly present.
Full details: Docstring Coverage
Explanation
Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 15 functions across 2 files. (1 skipped: 1 unsupported.)
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
CodeRabbit Verify
Run CodeRabbit Verify to test the changes in this pull request.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
…SIM-1893, AISIM-1894]
Lift the one-worker/1P1D limit on offline AgentX G2. The KV router already
places requests from each worker's native HostPinned events; AgentX now uses
that path on any number of aggregated, prefill, or decode workers. vLLM,
attention DP=1, static pools, no speculation, and no G3 remain required.
The offline router mirrors lower-tier residency per rank and rejects a G2
store for an already-resident block or a removal of a block it never stored,
so a duplicated, lost, or reordered event fails replay instead of silently
diverging the router view from the engine.
Tests cover visibility-gated placement (including the same-timestamp and
lag boundary), private and cluster-shared pools, 2P2D prefill routing,
router/engine reuse reconciliation at each admission, recycled snapshot play
isolation, and deterministic reruns. They also pin the cluster-shared G2
contract: shared hits stay within the play that stored them; concurrent
consumers on different replicas restore the same entries with residency
counted once and the pool's H2D cap split evenly and deterministically;
residency evicted while a request is queued falls back to recompute; and
switching to dp_rank_local reproduces the private-G2 result exactly. Docs
describe ownership, discovery, visibility, eviction, contention, and
fallback semantics.
Signed-off-by: Harry Lee <harrli@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
github-actions
Bot
added
feat
documentation
Summary
Resolves AISIM-1893 and AISIM-1894 (Dynamo side). AISIM-1894's shared-G2 qualification was folded in from #17879, since it only adds tests and docs for the behavior enabled here. AISimulate counterpart: ai-dynamo/aisimulate#688.
validate_agentic_host_offload()limited offline AgentX G2 to one aggregated worker or 1P1D. This PR reuses the existing multi-worker G2 andHostPinnedevent routing, with no AgentX-specific residency path:kv_event_lag_msdelays batches FIFO, so these cases mean a duplicated, lost, or reordered event. Replay now fails instead of letting the router view drift from the engine. All 311 existing mocker tests pass with this check enabled.t + lag.Tests (deterministic,
CanonicalV1)HostPinnedstore moves placement only once it is visible. Arrivals at 5 and 14.999 ms place without it; arrivals at 15 and 50 ms place on its owner.resumefollows load and recomputes.resumemoves to the private G2 owner and restores 8 tokens.cluster_sharedpool is visible to every replica, and load picks among them.host_cache_hit_weight=1.0.Shared-G2 qualification (AISIM-1894)
cache_id, and two lanes occupy exactly 2× the single-lane pool residency.g2_domainsmatches the solo run, and reruns are identical.cluster_sharedconfig withscope=dp_rank_localreproduces the private baseline for 1 and 2 workers.The reference doc also describes the shared-pool contract: ownership, discovery, visibility, eviction, contention, stale-entry fallback, membership, and disable semantics. Worker failure and ownership handoff are outside the boundary because AgentX G2 requires static pools.
Not in this PR
Dynamo's Python replay API goes through AISimulate's sweeper gate, which still enforces the old limit until a nightly with ai-dynamo/aisimulate#688 is published. A follow-up will bump the nightly and add a Python multi-replica AgentX test.
Validation
cargo test -p dynamo-mocker --libpasses (311 before the shared-G2 tests;agentic_multi_replica: 9 passed).cargo clippy -p dynamo-mocker --all-targets -D warningsis clean, and pre-commit passes.