Skip to content

fix(aic): adopt published AIC 0.11 dev artifacts - #11986

Merged
tedzhouhk merged 35 commits into
mainfrom
agent/update-aic-nextn-accepted
Jul 30, 2026
Merged

fix(aic): adopt published AIC 0.11 dev artifacts#11986
tedzhouhk merged 35 commits into
mainfrom
agent/update-aic-nextn-accepted

Conversation

@jasonqinzhou

@jasonqinzhou jasonqinzhou commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • adopt the paired published dev releases aiconfigurator==0.11.0.dev20260728 and aiconfigurator-core==0.11.0.dev20260728
  • pin both Rust consumers to the published crates.io release aiconfigurator-core==0.11.0-dev20260728, with registry checksums recorded in both lockfiles
  • use aiconfigurator_core for Runtime, Mocker, Router, and Planner modeling while retaining upper aiconfigurator for Profiler and AI Simulate orchestration/generator surfaces
  • migrate the Python bindings to the typed AicEngineBuilder API and remove pre-0.11 compatibility/fallback paths
  • keep acceptance-rate scheduling in Dynamo while forwarding nonzero nextn to AIC core for MTP verification cost
  • align Rapid with the published Task-v2 _execute_tasks six-value return contract, including per-task outcomes
  • extend dependency, clean-wheel, native-engine, replay, Planner, frontend-image, and AIC feature test/Clippy coverage

Packaging

The Python dev releases are index-resolvable from PyPI. Upper aiconfigurator pins the exact matching aiconfigurator-core release; AI Simulate declares both because it directly imports both package layers. The core package uses its published wheel-stub flow to install the platform wheel from the NVIDIA package index.

The Rust crate is published and non-yanked on crates.io. Dynamo no longer embeds an AIC Git revision or builds AIC from source in its package metadata or container flow. The Planner image installs the standalone aisimulate wheel together with the paired AIC packages.

Reviewer guide

  1. Dependency contract: Cargo.toml, both Cargo lockfiles, root/benchmark/AI Simulate Python metadata, and container/deps/requirements.planner.txt
  2. Runtime boundary: lib/bindings/python/rust/llm/aic_callback.rs, dynamo._internal.aic, Mocker perf-model configuration, replay, and scheduler adapters
  3. Upper-package consumers: Rapid/Profiler Task-v2 integration and AI Simulate generator/core imports
  4. Packaging and gates: Planner/frontend Dockerfiles, dependency consistency, clean-wheel smoke, integration tests, and AIC-triggered CI paths

Validation

  • clean Python 3.10 installation of both exact published Python dev packages
  • dependency-consistency checks across Python metadata, Cargo manifests, and both lockfiles
  • published Task-v2 contract integration coverage
  • cargo check --locked -p dynamo-mocker --features aic-forward-pass
  • cargo check --locked --manifest-path lib/bindings/python/Cargo.toml --features aic-forward-pass
  • cargo fmt --manifest-path lib/bindings/python/Cargo.toml -- --check
  • rendered multi-arch Planner, Runtime, and Frontend Dockerfiles; verified the Planner builds and installs the standalone AI Simulate wheel
  • affected pre-commit hooks and git diff --check

Full PR CI is running on the current signed head. Merge remains gated on the required current-head reviews.

Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 21, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
@PeaBrane
PeaBrane marked this pull request as ready for review July 23, 2026 00:14
@PeaBrane
PeaBrane requested review from a team as code owners July 23, 2026 00:14

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment thread lib/bindings/python/src/dynamo/_internal/aic.py Outdated
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

The AIC speculative-decoding configuration now uses scalar nextn_accepted values derived from legacy accept-rate inputs. Python, Rust, mock-engine, planner, cache-key, dependency, and test paths were updated accordingly.

Speculative acceptance configuration

Layer / File(s) Summary
Legacy acceptance conversion
lib/bindings/python/src/dynamo/_internal/aic.py, lib/bindings/python/rust/llm/aic_callback.rs, lib/bindings/python/tests/test_aic_capacity.py
Legacy conditional accept rates are converted into scalar nextn_accepted values, forwarded to AIC model configuration, and covered by helper and session tests.
Mock-engine speculative configuration
lib/mocker/src/common/engine_perf.rs, Cargo.toml, lib/bindings/python/Cargo.toml
Mock AIC configuration parses and validates nextn_accepted, stores it in SpeculativeConfig, updates raw-iteration and native configuration paths, and pins updated AIC dependency revisions.
Planner adapter integration
components/src/dynamo/planner/core/perf_model/rust_adapter.py, components/src/dynamo/planner/tests/unit/test_rust_perf_adapter.py
Planner configuration and model cache keys conditionally use nextn and nextn_accepted, with corresponding expectations updated in unit tests.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 64.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The PR description covers the work, but it does not follow the required template or include the required Related Issues section. Reformat it to the repository template and add the required Related Issues section with a linked issue or a confirmed no-issue checkbox.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and describes the main change: adopting published AIC dev artifacts.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/bindings/python/rust/llm/aic_callback.rs`:
- Around line 116-128: Update the nextn conversion in the callback logic to use
u32::try_from instead of narrowing with as u32, and return a PyValueError when
the usize value exceeds u32::MAX. Perform this validation before the nextn > 0
check and before calling _nextn_accepted_from_accept_rates, while preserving the
existing zero and valid-value behavior.

In `@lib/bindings/python/tests/test_aic_capacity.py`:
- Around line 255-258: Update the pytest.raises match in
test_nextn_accepted_rejects_unsupported_mocker_depth to use a raw regex with
both literal dots escaped, ensuring the assertion validates the exact “nextn
must be 1..=5” message.

In `@lib/mocker/src/common/engine_perf.rs`:
- Around line 1136-1144: Update the speculative configuration construction in
the args-to-config flow to ignore aic_nextn values of zero before creating
SpeculativeConfig. Preserve the existing conversion and error propagation for
positive values, ensuring aic_nextn=0 produces no speculative config and matches
into_aic_config’s dense-mode behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b82d3afb-55e0-4487-a823-3e4278e26fb1

📥 Commits

Reviewing files that changed from the base of the PR and between cfdc8d5 and fab8967.

⛔ Files ignored due to path filters (2)
  • Cargo.lock is excluded by !**/*.lock
  • lib/bindings/python/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • Cargo.toml
  • components/src/dynamo/planner/core/perf_model/rust_adapter.py
  • components/src/dynamo/planner/tests/unit/test_rust_perf_adapter.py
  • lib/bindings/python/Cargo.toml
  • lib/bindings/python/rust/llm/aic_callback.rs
  • lib/bindings/python/src/dynamo/_internal/aic.py
  • lib/bindings/python/tests/test_aic_capacity.py
  • lib/mocker/src/common/engine_perf.rs

Comment thread lib/bindings/python/rust/llm/aic_callback.rs Outdated
Comment thread lib/bindings/python/tests/test_aic_capacity.py Outdated
Comment thread lib/mocker/src/common/engine_perf.rs
PeaBrane

This comment was marked as outdated.

@PeaBrane
PeaBrane dismissed their stale review July 23, 2026 00:49

Replacing this request-changes review with a neutral review containing both notes inline.

@PeaBrane PeaBrane left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two inline notes: the Python/Rust dependency alignment is the actionable contract issue; the FPM note only requests a TODO documenting a dormant integration boundary.

Comment thread Cargo.toml Outdated
Comment thread lib/mocker/src/common/engine_perf.rs Outdated
…n-accepted

Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
@jasonqinzhou
jasonqinzhou requested review from a team as code owners July 24, 2026 02:43
@github-actions github-actions Bot added documentation Improvements or additions to documentation container labels Jul 24, 2026
@jasonqinzhou jasonqinzhou changed the title fix(aic): adapt MTP acceptance contract fix(aic): upgrade Dynamo to AIC 0.11.0 Jul 24, 2026
Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
Signed-off-by: Jason Zhou (Engrg-Hardware 1) <jasonzho@nvidia.com>
Comment thread benchmarks/pyproject.toml Outdated
@jasonqinzhou jasonqinzhou changed the title fix(aic): upgrade Dynamo to pinned AIC 0.11 source fix(aic): upgrade Dynamo to AIC 0.11 dev release Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test d19f515

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>

Copy link
Copy Markdown
Contributor Author

Rechecked the exact current head 70e3176fe98e537bbec211478f1ca83e44704a5d.

The two blocking regressions found against the prior d19f5156 head are already addressed by 70e3176f:

  • the pre-0.11 compatibility probe for aiconfigurator.sdk.engine was removed, so the supported core-only ai-dynamo[mocker] path now enters AicEngineBuilder directly;
  • the planner image once again installs /opt/dynamo/wheelhouse/aisimulate*.whl, matching the wheel-builder output and preventing the Spica suite from being absent in the shipped planner image.

Focused validation on this head:

  • cargo check --locked --manifest-path lib/bindings/python/Cargo.toml --features aic-forward-pass
  • cargo fmt --manifest-path lib/bindings/python/Cargo.toml -- --check
  • rendered the multi-arch planner Dockerfile and verified both the AI Simulate wheel build and install lines
  • git diff --check

I also replied to and resolved the outdated benchmark dependency thread: AIC core is pinned to 0.11.0.dev20260728; aiperf==0.10.0 is a separate package/version. All inline review threads are resolved.

@tedzhouhk

Copy link
Copy Markdown
Contributor

/ok to test 70e3176

@jasonqinzhou
jasonqinzhou force-pushed the agent/update-aic-nextn-accepted branch from 70e3176 to 3d0dbf5 Compare July 29, 2026 00:27
@jasonqinzhou jasonqinzhou changed the title fix(aic): upgrade Dynamo to AIC 0.11 dev release fix(aic): adopt published AIC 0.11 dev artifacts Jul 29, 2026

@jthomson04 jthomson04 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving the one frontend change.

Comment thread .github/workflows/pr.yaml
Comment thread .github/workflows/pr.yaml Outdated
Comment thread .github/workflows/pr.yaml Outdated
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
…-mypy

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>

# Conflicts:
#	container/templates/planner.Dockerfile

@saturley-hall saturley-hall left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirmed that the planner container installed the dev version of aiconfigurator-core from the compliance CSV

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
@tedzhouhk
tedzhouhk enabled auto-merge (squash) July 29, 2026 20:36
tedzhouhk and others added 3 commits July 29, 2026 14:13
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
@jasonqinzhou
jasonqinzhou force-pushed the agent/update-aic-nextn-accepted branch from 0ab569d to 635e023 Compare July 29, 2026 23:52
@tedzhouhk
tedzhouhk merged commit 7645809 into main Jul 30, 2026
118 checks passed
@tedzhouhk
tedzhouhk deleted the agent/update-aic-nextn-accepted branch July 30, 2026 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants