fix(simulation): upgrade AISimulate to 0.12.0.dev1

[tedzhouhk]

Summary

Upgrade the published AISimulate Python dependency from 0.1.0.dev2 to 0.12.0.dev1 and the Rust dependency from 0.1.0-dev.2 to 0.12.0-dev.1, with the compatibility changes needed by Planner, Profiler, and Replay. Older hard-coded backend versions fail against the new queryable database slots; the KV geometry and replay API changes also require updates to Dynamo's adapter.

• Update Python, benchmark, and container requirements, both Rust manifests, and all three Cargo lockfiles to the matching published packages. Container builds continue downloading published wheels, selecting the runtime's manylinux_2_34 platform, Python version, and ABI explicitly so the manylinux_2_28 builder can stage them.
• Resolve omitted AIC backend versions through the release database's current slot, including replay capacity estimation and profiler-generated mocker arguments; preserve explicit versions.
• Keep transfer bytes independent from cache bytes, accept the legacy transfer argument, and adapt replay descriptors and placement telemetry.
• Consume SweepResult.candidates and update native AIC/Planner/Profiler regression coverage. DGDR continues using the supported legacy sweep API pending its migration; pytest ignores only the two sweep_agg/sweep_disagg deprecation messages, while unrelated warnings remain errors.

Published package verification

The release-publication dependency blocker is resolved:

• Python aisimulate==0.12.0.dev1 and Rust aisimulate-core=0.12.0-dev.1 are both published. The wheel's Python/core versions and embedded Rust SBOM agree with this version pair.
• The crate records source commit 82d4b96674. This is release/0.12.0 tip 40daf5ea55 plus version-stamping/build-test updates, with no additional functional source changes.
• Verified the downloaded package SHA256 hashes against their package indexes. All 257 Python source files in the Linux x86_64 wheel and all 156 Rust files under the crate's src/ match that source commit.

Validation

Review follow-up at 93853600d7:

• Executed the rendered AISimulate staging command in a real manylinux_2_28 container: downloads succeeded for both amd64 and arm64 target wheels. Rendered all four consuming targets (Planner, vLLM, SGLang, TensorRT-LLM).
• Both benchmark entrypoints passed actual session/replay smoke probes; frontend AIC smoke and CLI help verification passed.
• Focused AIC/replay/mocker/dependency regression: 168 passed, 8 existing skips. Aggregated router AIC E2E passed. Disaggregated AIC reaches successful native AIC initialization but times out waiting for readiness locally; the non-AIC test_router_decisions_disagg[no_bootstrap] control reproduces the same timeout. This is not counted as a pass; CI must recheck the disaggregated path.
• Full pre-commit, Rust formatting, and diff checks passed. The edited docs page passes docs lint; fern check has zero errors. The full-site broken-link check reports an unchanged link in deepseek-v4-pro-0813.mdx:254, outside these changes.

Earlier validation of the dependency upgrade:

Fresh validation uses the published Python wheel and a newly built Dynamo runtime linked against the published Rust crate, from this PR's source:

• Previously reported 20 compatibility failures: 20 passed, including both backend parity matrices, all four CLI cases, spawned Sweeper workers, native AIC, and both Rapid entry points. The DGDR Rapid case also passed independently in a fresh process.
• Planner suite, replay lowering, AIC capacity, profiler DGD generation, and dependency consistency: 1435 passed, 2 existing skips.
• cargo test -p dynamo-mocker --locked: 213 passed.
• Dynamo Python/runtime wheel builds, the container's published-wheel download command, and native AIC wheel/data smoke passed.
• Full pre-commit, cargo fmt --all -- --check, and git diff --check passed.

Full backend-image/GPU CI was not run locally; these counts describe the targeted local regression runs, not full repository CI.

Summary by CodeRabbit

• New Features

  • Backend versions for vLLM, SGLang, and TensorRT-LLM can now resolve automatically to the current release.
  • Replay configuration automatically resolves backend versions and GPU capacity when details are omitted.
  • Added separate KV-cache capacity configuration and improved KV overlap information for worker placement.
  • Legacy KV transfer configuration names remain accepted for compatibility.

• Updates

  • Upgraded AISimulate to version 0.12.0.dev1.
  • Updated supported AIC integration paths to use vLLM 0.24.0.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4b3101e7-c59c-49d6-ac8c-f322bff29c28

📥 Commits

Reviewing files that changed from the base of the PR and between 7d2d5da and 08e5466.

⛔ Files ignored due to path filters (3)

• Cargo.lock is excluded by !**/*.lock
• lib/bindings/kvbm/Cargo.lock is excluded by !**/*.lock
• lib/bindings/python/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (25)

• Cargo.toml
• benchmarks/pyproject.toml
• components/src/dynamo/planner/tests/integration/test_aic_native_estimator.py
• components/src/dynamo/profiler/tests/integration/test_aic_task_v2_contract.py
• components/src/dynamo/profiler/tests/unit/test_dgd_generation_aic.py
• components/src/dynamo/profiler/tests/unit/test_replay_aic_parity.py
• components/src/dynamo/profiler/utils/dgd_generation.py
• components/src/dynamo/profiler/utils/replay_optimize/constants.py
• components/src/dynamo/replay/config.py
• components/src/dynamo/replay/simulation.py
• components/src/dynamo/replay/tests/test_simulation.py
• components/src/dynamo/replay/tests/test_simulation_integration.py
• container/deps/requirements.aisimulate.txt
• lib/bindings/python/Cargo.toml
• lib/bindings/python/src/dynamo/_internal/aic.py
• lib/bindings/python/tests/replay/test_replay_aic_capacity.py
• lib/bindings/python/tests/replay/test_replay_aic_integration.py
• lib/bindings/python/tests/test_aic_capacity.py
• lib/mocker/src/common/protocols.rs
• lib/mocker/src/engine_adapter.rs
• lib/mocker/src/replay/offline/extensions/kv_router/mod.rs
• pyproject.toml
• tests/dependencies/test_aiconfigurator_consistency.py
• tests/frontend/router_aic_image_smoke.py
• tests/wheels/smoke_install.py

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

Changes

AIC replay and mocker changes

Layer / File(s)	Summary
Backend version defaults and dependency updates Cargo.toml, pyproject.toml, benchmarks/pyproject.toml, container/deps/requirements.aisimulate.txt, lib/bindings/python/..., components/src/dynamo/profiler/..., tests/...	AISimulate dependencies use version 0.12.0.dev1. Default AIC backend versions use the current release-database slot. Related tests use vLLM 0.24.0 or current.
Replay capacity resolution and sweep results components/src/dynamo/replay/..., components/src/dynamo/replay/tests/..., pyproject.toml	Replay configuration resolves AIC backend versions and GPU blocks through shared helpers. Sweep tests inspect returned candidates and require feasible results.
Mock engine KV and replay configuration lib/mocker/src/common/protocols.rs, lib/mocker/src/engine_adapter.rs	Mock engine arguments separate transfer and cache KV bytes per token. Replay configurations set num_gpu_blocks_is_explicit to None. Boundary tests validate the separate fields.
KV-router overlap telemetry lib/mocker/src/replay/offline/extensions/kv_router/mod.rs	Worker admissions and outcomes now carry the best eligible-worker KV overlap. Placement telemetry and admission fixtures include the new value, including removed-worker coverage.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 08e54

The dependency and replay configuration updates have no identified merge-blocking risk.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 35.14% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 37 functions across 20 files. (5 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	The description gives detailed upgrade scope, compatibility changes, package verification, and validation results. However, it omits the required Related Issues section and does not identify where rev…	Add the required Related Issues section with either an issue reference or confirmation that no related issue exists. Add a Where should the reviewer start? section with the key files or areas for review.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the primary change: upgrading AISimulate to version 0.12.0.dev1.

Full details: Docstring Coverage

Explanation

Docstring coverage is 35.14% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 37 functions across 20 files. (5 skipped: 5 unsupported.)

Full details: Description check

Explanation

The description gives detailed upgrade scope, compatibility changes, package verification, and validation results. However, it omits the required Related Issues section and does not identify where reviewers should start.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

⚔️ Resolve merge conflicts 💡

• Resolve merge conflict in branch hzhou-codex/aisimulate-release-compat

---

_{Comment @coderabbitai help to get the list of available commands.}

[MatejKosec]

No actionable frontend findings identified in this diff. Reviewed the affected AIC backend-version resolution and mocker adapter paths by source inspection; tests were not run.

[MatejKosec]

Approved from the frontend review scope. No actionable findings in the affected AIC backend-version resolution and mocker adapter paths. This was source inspection; tests were not run.

[dmitry-tokarev-nv]

Review: fix(simulation): upgrade AISimulate to 0.12.0.dev1

What this is for: bump the AISimulate Python wheel 0.1.0.dev2 -> 0.12.0.dev1 and the Rust crate 0.1.0-dev.2 -> 0.12.0-dev.1, and carry the Planner/Profiler/Replay/mocker changes the new release's APIs require.

Verified by execution

Installed aisimulate==0.12.0.dev1 from PyPI (macOS arm64 wheel, CPython 3.12) and ran lib/bindings/python/src/dynamo/_internal/aic.py from this PR's tree against it. Separately downloaded aisimulate-core 0.12.0-dev.1 from crates.io; its SHA256 matches all three lockfiles (d4db6a08...b652). cargo test -p dynamo-mocker --no-default-features on macOS arm64: 209 passed, 0 failed (the author reports 213 on Linux with default features).

The premise holds. The PR says older hard-coded versions fail against the new queryable slots. Measured:

estimate_num_gpu_blocks(backend_version=None -> "current")   vllm 3948 | sglang 3702 | trtllm 4365
estimate_num_gpu_blocks(backend_version="0.19.0"/"0.5.10"/"1.3.0rc10")   all -> ValueError

aiconfigurator_core.sdk.perf_database.resolve_query_version accepts only current/previous/next or a literal already in the slots; aisimulate.aic.DEFAULT_BACKEND_VERSIONS inside the new wheel is still {vllm: 0.19.0, sglang: 0.5.10, trtllm: 1.3.0rc10} and falls back to it when backend_version is None — so the guard added in components/src/dynamo/replay/config.py is load-bearing, not defensive. Good catch.

Other things I checked and found correct, so they are not findings:

• "current" resolves on every shipped system that has data for the backend (h100/h200/b200/b300/gb200/gb300/l40s/rtx_pro_6000/a100_sxm/b60-vllm). It raises on a100_pcie, a30, h100_pcie, l4 and b60+sglang/trtllm — but those have no perf data for the backend at all, so the pre-PR pins failed there too. No regression.
• build_aic_engine (lib/bindings/python/rust/llm/aic_callback.rs:185) does not route None through resolve_backend_version, which looked like a hole. It is not: compile_engine(..., backend_version=None) and ... "current" return byte-identical specs (2446 bytes each).
• ReplayEngineConfig.num_gpu_blocks_is_explicit: None matches the upstream struct default; the execute_json fallback (role_capacity_is_explicit) is the intended path and Dynamo's direct-API callers never read it.
• best_available_overlap_blocks: the eligibility filter matches what any_eligible_worker_rank enumerates for this router (replay_worker_config gives every worker data_parallel_start_rank=0, data_parallel_size=dp_size == self.dp_size; pinned_worker/affinity_target are hard-coded None in scheduling_request). Units line up too — in the offline path effective_overlap_blocks is built 1:1 from overlaps.scores, so overlap_regret_blocks cannot go negative-then-saturate.
• kv_cache_bytes_per_token is plumbed but never populated by Dynamo. Upstream only reads it under native_host_offload, which Dynamo never sets — inert, not a defect.
• The new filterwarnings entry matches the real message (aiconfigurator/deprecation.py:53) for sweep_agg and sweep_disagg, and does not match sweep_afd or warn_legacy_cli. That is exactly what the description claims.
• SweepResult.candidates is list[CandidateRecord] and CandidateRecord.status exists, so the test_simulation_integration.py change is right.

Findings

Three, all non-blocking, all the same root cause — the hard-coded-version sweep is incomplete. Details inline.

1. P2 — benchmarks/mocker/bench_aic_concurrency.py:42 and bench_aic_rust_callback.py:68 still pin 0.19.0; both reach a real DB lookup and now raise. No CI covers them.
2. P2 — --aic-backend-version help text (components/src/dynamo/mocker/args.py:347), mocker-cli-reference.mdx:170 and protocols.rs:596 name only versions that are now rejected.
3. P3 — literal-vs-alias asymmetry in the updated pins, plus a stale dead duplicate of AIC_PARITY_VERSIONS in lib/bindings/python/tests/replay/replay_utils.py:41.

CI triage — not findings, you already see these

Recording the determination so it does not get re-litigated:

• All four image builds (planner, vllm-runtime, sglang-runtime, trtllm-runtime) fail identically at the requirements.aisimulate.txt download: Could not find a version that satisfies the requirement aisimulate==0.12.0.dev1 (from versions: 0.1.0.dev1, 0.1.0.dev2). The build resolves through the internal index, which has not mirrored the new wheel; public PyPI has it. One qualification on the description: "the container's published-wheel download command ... passed" is true against public PyPI but not against the index CI uses, so "the release-publication dependency blocker is resolved" is not yet true for the build that gates this PR. Mirroring, not a code change.
• dynamo-runtime / test / sequential is red on both arches for one root cause — tests/router/test_router_e2e_with_mockers.py:100 still pins "aic_backend_version": "0.14.0", which the new slot gate rejects; test_router_decisions_disagg_router_aic is the same failure one hop later. Same class as finding 1, and the reason I am reading the sweep as incomplete rather than as two isolated nits.
• backend-status-check / dynamo-status-check are aggregators of the above.

Not approving yet

Nothing I found is P0/P1, and three P2/P3 would normally clear my bar. I am holding approval only because finding 1, the router-test pin, and the index mirroring are one unfinished piece of work rather than independent nits — I would rather see the sweep land complete. Happy to re-review and approve as soon as the remaining constants move; ping me and I will re-run the probes above against the new head.

Reviewed at cfb95d4f3fb636d687ec19e4a14e38af38e50fa4. Where verification stopped: I ran nothing on GPU and did not build any container image, so the runtime-image and GPU-stage behaviour of this bump is unverified by me.

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-01a0a1e7-fced-73ce-ba2d-eecc3ead295c.docs.buildwithfern.com/dynamo/dev

[dmitry-tokarev-nv]

Approving — round 2

All three findings from my first pass are fixed and re-verified by execution, and 764685344c went further than I asked: it also moved tests/router/test_router_e2e_with_mockers.py off 0.14.0 and collapsed the literal/alias mix to current across every live call site.

Re-verified against aisimulate==0.12.0.dev1 (macOS arm64 / CPython 3.12), for every (system, backend) pair the changed sites use:

h200_sxm vllm   current -> 0.24.0      estimate_num_gpu_blocks -> 3948
h200_sxm sglang current -> 0.5.14                              -> 3702
gb200    trtllm current -> 1.3.0rc20                           -> 4365

Also re-grepped the whole tree: the only remaining literal AIC versions are tests/wheels/smoke_install.py (0.24.0, correctly kept — it globs that literal data directory), and three pure-plumbing fixtures that never reach a database lookup (mocker/tests/unit/test_config.py, replay/tests/test_simulation.py:379, router/tests/test_router_sweep_config_provider.py:284). Those are opaque strings; leaving them is fine. The replay_utils.py deletion is clean — no dangling references, and test_replay_smoke.py's import list touches no removed name.

I got the image-build triage wrong

My summary said the internal index had not mirrored the wheel. That was wrong, and @tedzhouhk's diagnosis is right: 0.1.0.dev2 ships manylinux_2_28 wheels and 0.12.0.dev1 ships manylinux_2_34, so on the 2.28 builder the new wheels were filtered out on platform tag. pip's from versions: ... lists platform-compatible candidates, not index contents — I read it as the latter. It would have reproduced against public PyPI too. Corrected in the thread above.

One new P2, non-blocking

container/templates/wheel_builder.Dockerfile:655 — ${PYTHON_VERSION} is not in scope in runtime_wheel_builder (the ARG is declared in wheel_builder_base and does not cross the FROM; this is why the stage re-declares ARG TARGETARCH at 576). It expands to empty, and pip treats an empty --python-version as "not provided" by design. The staging still works — I ran both forms against PyPI and they fetch the same wheel — so the flag is inert rather than wrong. One line to make the stated guarantee real. Details inline.

Approving with that outstanding: it does not change what gets staged, and I would rather not hold the bump for it.

Where verification stopped

• No GPU, no container image built. The runtime-image and GPU-stage behaviour of this bump is unverified by me, including whether vllm/vllm-openai-xpu:v0.27.1 clears the glibc 2.34 floor (every other base in container/context.yaml does — ubuntu 24.04 / 22.04).
• I could not re-run cargo test -p dynamo-mocker on this head: my local toolchain broke mid-review on an unrelated Xcode licence prompt (ld: library 'c++' not found). It passed 209/209 on cfb95d4f3f, and the only lib/mocker change since is the doc comment in protocols.rs, which cannot change behaviour.
• CI status is not part of my bar and the remaining red is yours to land; I have not re-triaged it since 93853600d7.