Repository navigation
build(deps): [release/1.6.0] pin published AISimulate 0.13.0 - #16949
Conversation
Replace the release-only source wheel build and three Cargo source overrides with published GA requirements. Keep the RC5 Mocker compatibility changes. Refs OPS-8866. Draft preparation: aisimulate-core 0.13.0 is not yet on crates.io. Regenerating all three Cargo lockfiles is blocked on publication; the existing Git-sourced locks remain unchanged and the consistency test correctly rejects them. Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
❌ Dynamo PR CI failed — run 39501659607 (attempt 1) on
|
| Framework | Build |
|---|---|
| vLLM | ❌ 1 |
| SGLang | ❌ 1 |
| TRT-LLM | ❌ 1 |
| Other | Jobs |
|---|---|
| dynamo-runtime | ❌ 1 |
| dynamo-sidecar | ❌ 1 |
| frontend | ❌ 1 |
| planner | ❌ 1 |
| triton-runtime | ❌ 1 |
Failure details
8 build jobs failed: 7 fail because aisimulate-core = "=0.13.0" (pinned by this PR) does not resolve on crates.io, where only 0.13.0-dev.* prereleases exist; the vLLM build hit an infrastructure HTTP 503 cloning libvpx from chromium.googlesource.com before reaching that step. No tests ran.
❌ planner / Build multi-arch cpu: aisimulate-core = "=0.13.0" not found on crates.io
Job: planner / Build multi-arch cpu · Failed step: Build and Push Image · Logs: gh run view --job 118900576088 -R ai-dynamo/dynamo --log-failed
error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
The runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.
❌ sglang-runtime / Build multi-arch cuda13.0: aisimulate-core = "=0.13.0" not found on crates.io
Job: sglang-runtime / Build multi-arch cuda13.0 · Failed step: Build and Push Image · Logs: gh run view --job 118900576135 -R ai-dynamo/dynamo --log-failed
error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
Same failure as planner: the runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.
❌ trtllm-runtime / Build multi-arch cuda13.1: aisimulate-core = "=0.13.0" not found on crates.io
Job: trtllm-runtime / Build multi-arch cuda13.1 · Failed step: Build and Push Image · Logs: gh run view --job 118900576211 -R ai-dynamo/dynamo --log-failed
error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
Same failure as planner: the runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.
❌ triton-runtime / Build multi-arch cpu: aisimulate-core = "=0.13.0" not found on crates.io
Job: triton-runtime / Build multi-arch cpu · Failed step: Build and Push Image · Logs: gh run view --job 118900576160 -R ai-dynamo/dynamo --log-failed
error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
Same failure as planner: the runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.
❌ dynamo-runtime / image / Build multi-arch cuda13.0: aisimulate-core = "=0.13.0" not found on crates.io
Job: dynamo-runtime / image / Build multi-arch cuda13.0 · Failed step: Build and Push Image · Logs: gh run view --job 118900576494 -R ai-dynamo/dynamo --log-failed
error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, 0.13.0-dev.202610160000000073, ...
location searched: crates.io index
required by package `dynamo-py3 v1.6.0 (/opt/dynamo/lib/bindings/python)`
help: if you are looking for the prerelease package it needs to be specified explicitly
💥 maturin failed
Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
Same failure as planner: the runtime_wheel_builder stage fails in maturin/cargo metadata: dynamo-py3 (lib/bindings/python) pins aisimulate-core = "=0.13.0", but crates.io only has 0.13.0-dev.* prerelease versions, so dependency resolution fails before anything compiles.
❌ dynamo-sidecar / Build multi-arch cpu: aisimulate-core = "=0.13.0" not found on crates.io
Job: dynamo-sidecar / Build multi-arch cpu · Failed step: Build and Push Image · Logs: gh run view --job 118900576255 -R ai-dynamo/dynamo --log-failed
#32 1.147 error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
#33 1.160 error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
#32 ERROR: process "/bin/sh -c set -e; RUSTARCH=$(case ... (sidecar cargo build step)
ERROR: failed to build: failed to solve: process "/bin/sh -c set -e; RUSTARCH=... did not complete successfully
The sidecar's Rust build step fails on both arches with the same unresolvable aisimulate-core = "=0.13.0" requirement (only 0.13.0-dev.* exists on crates.io).
❌ frontend / Build multi-arch cpu: aisimulate-core = "=0.13.0" not found on crates.io
Job: frontend / Build multi-arch cpu · Failed step: Build EPP Image (For Frontend Only) · Logs: gh run view --job 118900576299 -R ai-dynamo/dynamo --log-failed
#30 0.752 error: failed to select a version for the requirement `aisimulate-core = "=0.13.0"`
#30 0.752 candidate versions found which didn't match: 0.13.0-dev.202610200000000077, 0.13.0-dev.202610170000000074, ...
#30 0.752 required by package `dynamo-mocker v1.6.0 (/dynamo/lib/mocker)`
#32 ERROR: failed to calculate checksum of ref ...: "/epp": not found
> [linux/amd64 stage-1 3/7] COPY --from=builder /epp /epp
ERROR: failed to build: failed to solve: failed to compute cache key: ... "/epp": not found
make: *** [Makefile:125: image-multiarch] Error 1
The EPP builder's cargo build fails resolving aisimulate-core = "=0.13.0" (required by dynamo-mocker), so no /epp binary is produced and the final COPY --from=builder /epp /epp fails with not found. The /epp not found error is a downstream symptom of the same pin.
❌ vllm-runtime / Build multi-arch cuda13.0: libvpx git clone HTTP 503 (chromium.googlesource.com)
Job: vllm-runtime / Build multi-arch cuda13.0 · Failed step: Build and Push Image · Logs: gh run view --job 118900576022 -R ai-dynamo/dynamo --log-failed
#81 26.82 Cloning into 'libvpx'...
#81 29.11 error: RPC failed; HTTP 503 curl 22 The requested URL returned error: 503
#81 29.11 fatal: expected 'packfile'
> [linux/amd64 wheel_builder_base 17/20] RUN ... git clone --depth 1 --branch v1.14.1 https://chromium.googlesource.com/webm/libvpx.git && ...
ERROR: failed to build: failed to solve: process "/bin/sh -c ... git clone ... libvpx.git ..." did not complete successfully: exit code: 128
Infrastructure: the wheel_builder_base step's git clone of libvpx from chromium.googlesource.com got HTTP 503, so this build died before reaching the Rust wheel step. On its own a rerun would likely clear it, but the build uses the same Rust wheel step that fails on the aisimulate-core pin in the jobs above.
For agents
{"pr": 16949, "run_id": 39501659607, "run_attempt": 1, "head_sha": "76df58657eba37e28003ea78566e735edba3c8ba", "failures": [{"job": "planner / Build multi-arch cpu", "job_id": 118900576088, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576088 -R ai-dynamo/dynamo --log-failed"}, {"job": "sglang-runtime / Build multi-arch cuda13.0", "job_id": 118900576135, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576135 -R ai-dynamo/dynamo --log-failed"}, {"job": "trtllm-runtime / Build multi-arch cuda13.1", "job_id": 118900576211, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576211 -R ai-dynamo/dynamo --log-failed"}, {"job": "triton-runtime / Build multi-arch cpu", "job_id": 118900576160, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576160 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-runtime / image / Build multi-arch cuda13.0", "job_id": 118900576494, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576494 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-sidecar / Build multi-arch cpu", "job_id": 118900576255, "failed_step": "Build and Push Image", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576255 -R ai-dynamo/dynamo --log-failed"}, {"job": "frontend / Build multi-arch cpu", "job_id": 118900576299, "failed_step": "Build EPP Image (For Frontend Only)", "signature": "aisimulate-core = \"=0.13.0\" not found on crates.io", "tests": [], "log_cmd": "gh run view --job 118900576299 -R ai-dynamo/dynamo --log-failed"}, {"job": "vllm-runtime / Build multi-arch cuda13.0", "job_id": 118900576022, "failed_step": "Build and Push Image", "signature": "libvpx git clone HTTP 503 (chromium.googlesource.com)", "tests": [], "log_cmd": "gh run view --job 118900576022 -R ai-dynamo/dynamo --log-failed"}]}Posted automatically by Devin for run 39501659607. Updated on every full-CI run of this PR.
Regenerate all three Cargo lockfiles from the published GA crate and record its registry checksum. No other package versions change. Refs OPS-8866. Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
| pyproject, cargo = _root_configs() | ||
| assert cargo_version == Version("0.13.0") | ||
| assert python_versions == SpecifierSet("==0.13.0") | ||
| assert python_versions.contains(cargo_version) |
There was a problem hiding this comment.
These membership checks cannot distinguish any regression beyond the retained exact assertions that Cargo is 0.13.0 and the Python specifier is ==0.13.0: those assertions already reject a Cargo/Python mismatch, every listed prerelease, 0.13.1, and 0.14.0. Remove these duplicate cases; the exact pins plus the existing lockfile/source checks still protect the published-version contract.
🤖 AI Fix
Delete the contains(cargo_version) assertion and the four-version exclusion loop, retaining the two exact-version assertions.
| cargo["workspace"]["dependencies"]["aisimulate-core"]["version"].removeprefix( | ||
| "=" | ||
| ) | ||
| assert "requirements.aisimulate.txt" in wheel_builder |
There was a problem hiding this comment.
This filename-only substring assertion cannot fail while the immediately following assertion for the complete --requirement /opt/dynamo/container/deps/requirements.aisimulate.txt argument passes. The full argument check already protects both use of the requirements file and its expected location.
🤖 AI Fix
Remove the standalone requirements.aisimulate.txt substring assertion and retain the full --requirement argument assertion.
Summary
Switch
release/1.6.0from the temporary AISimulate RC source build to the published 0.13.0 GA wheel and crate. GA publication is verified on PyPI, NVIDIA's wheel index, and crates.io.==0.13.0in the package, benchmark, and container requirements, preserving the supported Python markers and excluding nightlies.aisimulate-coreCargo source overrides and regenerate the root, Python-binding, and KVBM-binding lockfiles against crates.io. All three now recordaisimulate-core 0.13.0with checksum1d2deb095416b9ff555cbf1053f1eb7832c610b9cae06be37944cd939a6ae2b5. No other package versions or checksums change; Cargo also normalizes existingwindows-sysdependency references.container/deps/build_aisimulate.pyand stage the published abi3 wheel through the existing binary-only download path, retaining explicit installation across image targets.Release-only change with no Dynamo
maincounterpart. Refs OPS-8866. Requested in Ben's UAT thread.Validation
Validated the GA handoff on 2026-10-21, including lockfile follow-up commit
352403888aad5a043332d51fcef0883129f542c5.cargo check --locked -p dynamo-mocker --no-default-featurespassed with the repository's Rust 1.96.1 toolchain and compiler explicitly selected.git diff --checkpassed.Full container builds and integration/GPU tests remain hosted-CI validation. The previous draft's Rust and sampled planner-build failures reported the then-unpublished GA crate; those old runs do not qualify the updated head. The old lychee run separately reported a 404 for
fleet-ai/miles-fleetin an unchanged documentation page. Fresh CI and required review remain necessary before merge.