Skip to content

build(deps): [release/1.6.0] pin published AISimulate 0.13.0 - #16949

Merged
nv-nmailhot merged 2 commits into
release/1.6.0from
cherry-pick/aisimulate-0.13.0-ga-release-1.6.0
Oct 21, 2026
Merged

nv-nmailhot merged 2 commits into
release/1.6.0from
cherry-pick/aisimulate-0.13.0-ga-release-1.6.0

Conversation

@saturley-hall

@saturley-hall saturley-hall commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Summary

Switch release/1.6.0 from the temporary AISimulate RC source build to the published 0.13.0 GA wheel and crate. GA publication is verified on PyPI, NVIDIA's wheel index, and crates.io.

  • Pin Python AISimulate to ==0.13.0 in the package, benchmark, and container requirements, preserving the supported Python markers and excluding nightlies.
  • Remove all three aisimulate-core Cargo source overrides and regenerate the root, Python-binding, and KVBM-binding lockfiles against crates.io. All three now record aisimulate-core 0.13.0 with checksum 1d2deb095416b9ff555cbf1053f1eb7832c610b9cae06be37944cd939a6ae2b5. No other package versions or checksums change; Cargo also normalizes existing windows-sys dependency references.
  • Remove container/deps/build_aisimulate.py and stage the published abi3 wheel through the existing binary-only download path, retaining explicit installation across image targets.
  • Preserve the Mocker PCP/schema compatibility changes from build(deps): [release/1.6.0] update AISimulate sources to RC5 #16715 and replace the source-build tests with published-artifact and exact-version checks.

Release-only change with no Dynamo main counterpart. Refs OPS-8866. Requested in Ben's UAT thread.

Validation

Validated the GA handoff on 2026-10-21, including lockfile follow-up commit 352403888aad5a043332d51fcef0883129f542c5.

  • Verified PyPI 0.13.0, both published Linux abi3 wheels (x86_64 and aarch64; HTTP 200), and the non-yanked crates.io 0.13.0 crate.
  • Installed the published macOS arm64 GA wheel into an isolated Python 3.12 environment. Dependency consistency and namespace checks: 8 passed, including canonical Rust FPM binding import and installed-wheel version checks.
  • cargo check --locked -p dynamo-mocker --no-default-features passed with the repository's Rust 1.96.1 toolchain and compiler explicitly selected.
  • Locked Cargo metadata succeeds for all three manifests. All three generated core entries match the published registry checksum.
  • Rendered all 14 CUDA container configurations (frontend, planner, Dynamo runtime, vLLM, SGLang, TensorRT-LLM, Triton; amd64 and arm64); each stages the published binary wheel with no source-wheel build helper.
  • Applicable scoped pre-commit hooks and git diff --check passed.

Full container builds and integration/GPU tests remain hosted-CI validation. The previous draft's Rust and sampled planner-build failures reported the then-unpublished GA crate; those old runs do not qualify the updated head. The old lychee run separately reported a 404 for fleet-ai/miles-fleet in an unchanged documentation page. Fresh CI and required review remain necessary before merge.

Replace the release-only source wheel build and three Cargo source overrides with published GA requirements. Keep the RC5 Mocker compatibility changes. Refs OPS-8866.

Draft preparation: aisimulate-core 0.13.0 is not yet on crates.io. Regenerating all three Cargo lockfiles is blocked on publication; the existing Git-sourced locks remain unchanged and the consistency test correctly rejects them.

Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

❌ Dynamo PR CI failed — run 39501659607 (attempt 1) on 76df58657e

Gate checks: ❌ backend-status-check · ✅ deploy-status-check · ❌ dynamo-status-check

Framework Build
vLLM ❌ 1
SGLang ❌ 1
TRT-LLM ❌ 1
Other Jobs
dynamo-runtime ❌ 1
dynamo-sidecar ❌ 1
frontend ❌ 1
planner ❌ 1
triton-runtime ❌ 1

Failure details

8 build jobs failed: 7 fail because aisimulate-core = "=0.13.0" (pinned by this PR) does not resolve on crates.io, where only 0.13.0-dev.* prereleases exist; the vLLM build hit an infrastructure HTTP 503 cloning libvpx from chromium.googlesource.com before reaching that step. No tests ran.

❌ planner / Build multi-arch cpu: aisimulate-core = "=0.13.0" not found on crates.io

Job: planner / Build multi-arch cpu · Failed step: Build and Push Image · Logs: gh run view --job 118900576088 -R ai-dynamo/dynamo --log-failed

error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
  Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?

The runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.

❌ sglang-runtime / Build multi-arch cuda13.0: aisimulate-core = "=0.13.0" not found on crates.io

Job: sglang-runtime / Build multi-arch cuda13.0 · Failed step: Build and Push Image · Logs: gh run view --job 118900576135 -R ai-dynamo/dynamo --log-failed

error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
  Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?

Same failure as planner: the runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.

❌ trtllm-runtime / Build multi-arch cuda13.1: aisimulate-core = "=0.13.0" not found on crates.io

Job: trtllm-runtime / Build multi-arch cuda13.1 · Failed step: Build and Push Image · Logs: gh run view --job 118900576211 -R ai-dynamo/dynamo --log-failed

error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
  Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?

Same failure as planner: the runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.

❌ triton-runtime / Build multi-arch cpu: aisimulate-core = "=0.13.0" not found on crates.io

Job: triton-runtime / Build multi-arch cpu · Failed step: Build and Push Image · Logs: gh run view --job 118900576160 -R ai-dynamo/dynamo --log-failed

error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
  Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?

Same failure as planner: the runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.

❌ dynamo-runtime / image / Build multi-arch cuda13.0: aisimulate-core = "=0.13.0" not found on crates.io

Job: dynamo-runtime / image / Build multi-arch cuda13.0 · Failed step: Build and Push Image · Logs: gh run view --job 118900576494 -R ai-dynamo/dynamo --log-failed

error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
  Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?

Same failure as planner: the runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.

❌ dynamo-sidecar / Build multi-arch cpu: aisimulate-core = "=0.13.0" not found on crates.io

Job: dynamo-sidecar / Build multi-arch cpu · Failed step: Build and Push Image · Logs: gh run view --job 118900576255 -R ai-dynamo/dynamo --log-failed

#32 1.147 error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
#33 1.160 error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
#32 ERROR: process "/bin/sh -c set -e;     RUSTARCH=$(case ... (sidecar cargo build step)
ERROR: failed to build: failed to solve: process "/bin/sh -c set -e; RUSTARCH=... did not complete successfully

The sidecar's Rust build step fails on both arches with the same unresolvable aisimulate-core = "=0.13.0" requirement (only 0.13.0-dev.* exists on crates.io).

❌ frontend / Build multi-arch cpu: aisimulate-core = "=0.13.0" not found on crates.io

Job: frontend / Build multi-arch cpu · Failed step: Build EPP Image (For Frontend Only) · Logs: gh run view --job 118900576299 -R ai-dynamo/dynamo --log-failed

#30 0.752 error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
#30 0.752 candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, ...
#30 0.752 required by package `dynamo-mocker v1.6.0 (/dynamo/lib/mocker)`
#32 ERROR: failed to calculate checksum of ref ...: "/epp": not found
> [linux/amd64 stage-1 3/7] COPY --from=builder /epp /epp
ERROR: failed to build: failed to solve: failed to compute cache key: ... "/epp": not found
make: *** [Makefile:125: image-multiarch] Error 1

The EPP builder's cargo build fails resolving aisimulate-core = "=0.13.0" (required by dynamo-mocker), so no /epp binary is produced and the final COPY --from=builder /epp /epp fails with not found. The /epp not found error is a downstream symptom of the same pin.

❌ vllm-runtime / Build multi-arch cuda13.0: libvpx git clone HTTP 503 (chromium.googlesource.com)

Job: vllm-runtime / Build multi-arch cuda13.0 · Failed step: Build and Push Image · Logs: gh run view --job 118900576022 -R ai-dynamo/dynamo --log-failed

#81 26.82 Cloning into 'libvpx'...
#81 29.11 error: RPC failed; HTTP 503 curl 22 The requested URL returned error: 503
#81 29.11 fatal: expected 'packfile'
> [linux/amd64 wheel_builder_base 17/20] RUN ... git clone --depth 1 --branch v1.14.1 https://chromium.googlesource.com/webm/libvpx.git && ...
ERROR: failed to build: failed to solve: process "/bin/sh -c ... git clone ... libvpx.git ..." did not complete successfully: exit code: 128

Infrastructure: the wheel_builder_base step's git clone of libvpx from chromium.googlesource.com got HTTP 503, so this build died before reaching the Rust wheel step. On its own a rerun would likely clear it, but the build uses the same Rust wheel step that fails on the aisimulate-core pin in the jobs above.

For agents
{"pr": 16949, "run_id": 39501659607, "run_attempt": 1, "head_sha": "76df58657eba37e28003ea78566e735edba3c8ba", "failures": [{"job": "planner / Build multi-arch cpu", "job_id": 118900576088, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576088 -R ai-dynamo/dynamo --log-failed"}, {"job": "sglang-runtime / Build multi-arch cuda13.0", "job_id": 118900576135, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576135 -R ai-dynamo/dynamo --log-failed"}, {"job": "trtllm-runtime / Build multi-arch cuda13.1", "job_id": 118900576211, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576211 -R ai-dynamo/dynamo --log-failed"}, {"job": "triton-runtime / Build multi-arch cpu", "job_id": 118900576160, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576160 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-runtime / image / Build multi-arch cuda13.0", "job_id": 118900576494, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576494 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-sidecar / Build multi-arch cpu", "job_id": 118900576255, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576255 -R ai-dynamo/dynamo --log-failed"}, {"job": "frontend / Build multi-arch cpu", "job_id": 118900576299, "failed_step": "Build EPP Image (For Frontend Only)", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576299 -R ai-dynamo/dynamo --log-failed"}, {"job": "vllm-runtime / Build multi-arch cuda13.0", "job_id": 118900576022, "failed_step": "Build and Push Image", "signature": "libvpx git clone HTTP 503 (chromium.googlesource.com)", "tests": [], "log_cmd": "gh run view --job 118900576022 -R ai-dynamo/dynamo --log-failed"}]}

Posted automatically by Devin for run 39501659607. Updated on every full-CI run of this PR.

Regenerate all three Cargo lockfiles from the published GA crate and record its registry checksum. No other package versions change. Refs OPS-8866.

Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
@saturley-hall
saturley-hall marked this pull request as ready for review October 21, 2026 20:41
@saturley-hall
saturley-hall requested review from a team as code owners October 21, 2026 20:41
@saturley-hall saturley-hall added the cherry-pick:requested Fix requested for the release branch in code freeze label Oct 21, 2026
pyproject, cargo = _root_configs()
assert cargo_version == Version("0.13.0")
assert python_versions == SpecifierSet("==0.13.0")
assert python_versions.contains(cargo_version)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These membership checks cannot distinguish any regression beyond the retained exact assertions that Cargo is 0.13.0 and the Python specifier is ==0.13.0: those assertions already reject a Cargo/Python mismatch, every listed prerelease, 0.13.1, and 0.14.0. Remove these duplicate cases; the exact pins plus the existing lockfile/source checks still protect the published-version contract.

🤖 AI Fix

Delete the contains(cargo_version) assertion and the four-version exclusion loop, retaining the two exact-version assertions.

cargo["workspace"]["dependencies"]["aisimulate-core"]["version"].removeprefix(
"="
)
assert "requirements.aisimulate.txt" in wheel_builder

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This filename-only substring assertion cannot fail while the immediately following assertion for the complete --requirement /opt/dynamo/container/deps/requirements.aisimulate.txt argument passes. The full argument check already protects both use of the requirements file and its expected location.

🤖 AI Fix

Remove the standalone requirements.aisimulate.txt substring assertion and retain the full --requirement argument assertion.

@nv-nmailhot
nv-nmailhot merged commit 2c2ea2c into release/1.6.0 Oct 21, 2026
114 checks passed
@nv-nmailhot
nv-nmailhot deleted the cherry-pick/aisimulate-0.13.0-ga-release-1.6.0 branch October 21, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build cherry-pick:requested Fix requested for the release branch in code freeze container size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants