Repository navigation
test(replay): [cherry-pick 1.7.0] detect AISimulate under the aisim distribution name (#18286) - #18287
test(replay): [cherry-pick 1.7.0] detect AISimulate under the aisim distribution name (#18286)#18287BenHamm wants to merge 1 commit into
Conversation
AISimulate 0.14.0 ships as the `aisim` distribution (the import name is
still `aisimulate`), and release/1.7.0 pins `aisim==0.14.0`. The replay
integration gate in tests/replay/replay_utils.py only looked up
`distribution("aisimulate")`, so with `aisim` installed it raised
PackageNotFoundError and skipped: test_replay_planner_scaling.py was
skipped at module level and
test_replay_smoke.py::test_planner_replay_accepts_multi_shard_dynamo_trace
skipped in its body, with no failure to flag the lost coverage.
Accept either distribution name, `aisim` first and then `aisimulate`, so
the gate works on main (still pinned to `aisimulate==0.13.0.dev...`) and
on branches that pin `aisim`. The tests still skip when neither is
installed, and the skip reason now names both distributions.
Internal tracking: Linear AISIM-2278.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ben Hamm <ben.hamm@gmail.com>
(cherry picked from commit 45de897)
❌ Dynamo PR CI failed — run 42153664182 (attempt 1) on
|
| Framework | Build |
|---|---|
| vLLM | ❌ 1 ✅ 2 |
| SGLang | ❌ 1 ✅ 2 |
| TRT-LLM | ❌ 1 ✅ 2 |
| Other | Jobs |
|---|---|
| frontend | ❌ 1 ✅ 2 |
Failure details
4 jobs failed, all in the Copy image to target registry step: every push of the built image to ACR is rejected with unknown: The operation is disallowed on this registry, repository or image on blob upload, across all retries. The image builds themselves passed.
❌ sglang-runtime / Copy to ACR cuda13.0: ACR blob upload disallowed
Job: sglang-runtime / Copy to ACR cuda13.0 · Failed step: Copy image to target registry · Logs: gh run view --job 127687105121 -R ai-dynamo/dynamo --log-failed
level=warning msg="Failed, retrying in 8s ... (4/4). Error: copying image 1/4 from manifest list: writing blob: initiating layer upload to /v2/ai-dynamo/dynamo/blobs/uploads/ in ***: unknown: The operation is disallowed on this registry, repository or image. View troubleshooting steps at https://aka.ms/acr/faq/#why-does-my-pull-or-push-request-fail-with-disallowed-operation-"
⚠️ Attempt 1 failed, retrying in 10s...
⚠️ Attempt 2 failed, retrying in 20s...
❌ All attempts failed
The copy of the built image to the Azure Container Registry is refused by the registry on layer upload (ACR "disallowed operation", typically a registry-side lock/write-disabled repository or permission policy), and all retries fail the same way. This is a registry/infrastructure-side rejection, not a build or test failure; once the registry accepts writes, a rerun of failed jobs is likely enough.
❌ vllm-runtime / Copy to ACR cuda13.0: ACR blob upload disallowed
Job: vllm-runtime / Copy to ACR cuda13.0 · Failed step: Copy image to target registry · Logs: gh run view --job 127688516110 -R ai-dynamo/dynamo --log-failed
level=warning msg="Failed, retrying in 8s ... (4/4). Error: copying image 1/4 from manifest list: writing blob: initiating layer upload to /v2/ai-dynamo/dynamo/blobs/uploads/ in ***: unknown: The operation is disallowed on this registry, repository or image. View troubleshooting steps at https://aka.ms/acr/faq/#why-does-my-pull-or-push-request-fail-with-disallowed-operation-"
⚠️ Attempt 1 failed, retrying in 10s...
⚠️ Attempt 2 failed, retrying in 20s...
❌ All attempts failed
The copy of the built image to the Azure Container Registry is refused by the registry on layer upload (ACR "disallowed operation", typically a registry-side lock/write-disabled repository or permission policy), and all retries fail the same way. This is a registry/infrastructure-side rejection, not a build or test failure; once the registry accepts writes, a rerun of failed jobs is likely enough.
❌ frontend / Copy to ACR cuda, amd64: ACR blob upload disallowed
Job: frontend / Copy to ACR cuda, amd64 · Failed step: Copy image to target registry · Logs: gh run view --job 127690636183 -R ai-dynamo/dynamo --log-failed
level=warning msg="Failed, retrying in 8s ... (4/4). Error: copying system image from manifest list: writing blob: initiating layer upload to /v2/ai-dynamo/dynamo/blobs/uploads/ in ***: unknown: The operation is disallowed on this registry, repository or image. View troubleshooting steps at https://aka.ms/acr/faq/#why-does-my-pull-or-push-request-fail-with-disallowed-operation-"
⚠️ Attempt 1 failed, retrying in 10s...
⚠️ Attempt 2 failed, retrying in 20s...
❌ All attempts failed
The copy of the built image to the Azure Container Registry is refused by the registry on layer upload (ACR "disallowed operation", typically a registry-side lock/write-disabled repository or permission policy), and all retries fail the same way. This is a registry/infrastructure-side rejection, not a build or test failure; once the registry accepts writes, a rerun of failed jobs is likely enough.
❌ trtllm-runtime / Copy to ACR cuda13.1: ACR blob upload disallowed
Job: trtllm-runtime / Copy to ACR cuda13.1 · Failed step: Copy image to target registry · Logs: gh run view --job 127691828684 -R ai-dynamo/dynamo --log-failed
level=warning msg="Failed, retrying in 8s ... (4/4). Error: copying image 1/4 from manifest list: writing blob: initiating layer upload to /v2/ai-dynamo/dynamo/blobs/uploads/ in ***: unknown: The operation is disallowed on this registry, repository or image. View troubleshooting steps at https://aka.ms/acr/faq/#why-does-my-pull-or-push-request-fail-with-disallowed-operation-"
⚠️ Attempt 1 failed, retrying in 10s...
⚠️ Attempt 2 failed, retrying in 20s...
❌ All attempts failed
The copy of the built image to the Azure Container Registry is refused by the registry on layer upload (ACR "disallowed operation", typically a registry-side lock/write-disabled repository or permission policy), and all retries fail the same way. This is a registry/infrastructure-side rejection, not a build or test failure; once the registry accepts writes, a rerun of failed jobs is likely enough.
For agents
{"pr": 18287, "run_id": 42153664182, "run_attempt": 1, "head_sha": "c9bf05799f69d96efd95ab44d82c3d0354b94e34", "failures": [{"job": "sglang-runtime / Copy to ACR cuda13.0", "job_id": 127687105121, "failed_step": "Copy image to target registry", "signature": "ACR push disallowed: \"The operation is disallowed on this registry, repository or image\"", "tests": [], "log_cmd": "gh run view --job 127687105121 -R ai-dynamo/dynamo --log-failed"}, {"job": "vllm-runtime / Copy to ACR cuda13.0", "job_id": 127688516110, "failed_step": "Copy image to target registry", "signature": "ACR push disallowed: \"The operation is disallowed on this registry, repository or image\"", "tests": [], "log_cmd": "gh run view --job 127688516110 -R ai-dynamo/dynamo --log-failed"}, {"job": "frontend / Copy to ACR cuda, amd64", "job_id": 127690636183, "failed_step": "Copy image to target registry", "signature": "ACR push disallowed: \"The operation is disallowed on this registry, repository or image\"", "tests": [], "log_cmd": "gh run view --job 127690636183 -R ai-dynamo/dynamo --log-failed"}, {"job": "trtllm-runtime / Copy to ACR cuda13.1", "job_id": 127691828684, "failed_step": "Copy image to target registry", "signature": "ACR push disallowed: \"The operation is disallowed on this registry, repository or image\"", "tests": [], "log_cmd": "gh run view --job 127691828684 -R ai-dynamo/dynamo --log-failed"}]}Posted automatically by Devin for run 42153664182. Updated on every full-CI run of this PR.
Overview
Cherry-pick of #18286 to
release/1.7.0. This branch pinsaisim==0.14.0, so without this change the replay integration gate never finds theaisimulatedistribution and silently skipstest_replay_planner_scaling.pyandtest_replay_smoke.py::test_planner_replay_accepts_multi_shard_dynamo_trace. Unlike main, the coverage is lost on this branch today.Key Changes
_require_aisimulate_distributionaccepts either distribution name (aisimfirst, thenaisimulate). Cleancherry-pick -x; the resulting file is identical to main.Validation
aisimcases skip); withaisimpresent the two files collect 18 tests (was 1, module skipped), and the smoke test gets past the gate instead of skipping.pre-commit run --files lib/bindings/python/tests/replay/replay_utils.py: passed.Issues and Ongoing Work
🤖 Generated with Claude Code