chore: upgrade the aiperf version in dynamo repo - #11061
Conversation
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
github-actions
Bot
added
chore
documentation
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
WalkthroughAll references to Changesaiperf bump to 0.10.0
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
recipes/qwen3-32b-fp8/trtllm/agg/perf.yaml (1)
27-33: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winMake this shell block fail fast before switching to the PyPI pin.
This script still runs under
/bin/shwithoutset -e, so a failedpip installor lateraiperf profilecan fall through to the trailingecho/lsand leave the Job green with no valid benchmark result. Please enable fail-fast at the top of the block (or explicitly|| exit 1the critical commands) before landing the new package version.Minimal fix
- | + set -eu apt-get update && apt-get install -y curl jq procps git && apt-get clean pip install "aiperf==0.10.0";🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3-32b-fp8/trtllm/agg/perf.yaml` around lines 27 - 33, The shell block in the command script can continue after a failed install or benchmark step because it runs under /bin/sh without fail-fast behavior. Update the block that installs aiperf and runs the benchmark so it exits immediately on error, either by enabling strict shell mode at the start or by explicitly guarding the critical commands with immediate exits. Use the existing command block and the aiperf install/profile steps as the key places to apply the fix.recipes/qwen3-235b-a22b-fp8/trtllm/disagg/hopper/perf.yaml (1)
27-33: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winMake this shell block fail fast before switching to the PyPI pin.
This script still runs under
/bin/shwithoutset -e, so a failedpip installor lateraiperf profilecan fall through to the trailingecho/lsand leave the Job green with no valid benchmark result. Please enable fail-fast at the top of the block (or explicitly|| exit 1the critical commands) before landing the new package version.Minimal fix
- | + set -eu apt-get update && apt-get install -y curl jq procps git && apt-get clean pip install "aiperf==0.10.0";🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3-235b-a22b-fp8/trtllm/disagg/hopper/perf.yaml` around lines 27 - 33, The shell block in the command script can continue past failed setup or profiling steps because it runs under /bin/sh without fail-fast behavior. Update the block in the perf config to enable immediate failure at the start of the script body, or explicitly guard the critical commands so that the apt-get, pip install of aiperf, and any subsequent aiperf profile invocation cannot fall through to later echo/ls steps; use the shell command block itself as the fix point.recipes/deepseek-r1/trtllm/disagg/wide_ep/gb200/perf.yaml (1)
28-34: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winMake this shell block fail fast before bumping the pin.
This script still runs under
/bin/shwithoutset -e, so a failedpip installor lateraiperf profilecan fall through to the trailingecho/lsand leave the Job green with no valid benchmark result. Please enable fail-fast at the top of the block (or explicitly|| exit 1the critical commands) before landing the new package version.Minimal fix
- | + set -eu apt-get update && apt-get install -y curl jq procps git && apt-get clean pip install aiperf==0.10.0;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/deepseek-r1/trtllm/disagg/wide_ep/gb200/perf.yaml` around lines 28 - 34, The shell block in the benchmark command needs fail-fast behavior so installation or profiling errors do not get masked by later commands. Update the script body used by the command sequence to enable immediate exit on failure (or add explicit exits around the critical install/profile steps) before the new aiperf pin is used, so failures in the pip install or subsequent benchmark commands stop the Job instead of reaching the trailing echo/ls. Use the command block under the perf YAML entry to apply the fix consistently.recipes/qwen3-32b-fp8/trtllm/disagg/perf.yaml (1)
27-33: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winMake this shell block fail fast before switching to the PyPI pin.
This script still runs under
/bin/shwithoutset -e, so a failedpip installor lateraiperf profilecan fall through to the trailingecho/lsand leave the Job green with no valid benchmark result. Please enable fail-fast at the top of the block (or explicitly|| exit 1the critical commands) before landing the new package version.Minimal fix
- | + set -eu apt-get update && apt-get install -y curl jq procps git && apt-get clean pip install "aiperf==0.10.0";🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@recipes/qwen3-32b-fp8/trtllm/disagg/perf.yaml` around lines 27 - 33, The shell block in the perf command currently runs under /bin/sh without fail-fast behavior, so failures in the apt-get or pip install steps can still reach later commands and mask a bad benchmark run. Update the command block to fail immediately before the aiperf pin is used by adding fail-fast handling at the top of the script block in the command array (or explicitly guarding the critical commands with exit-on-failure) so the benchmark job cannot continue after a failed install or profile step.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@recipes/glm-5-nvfp4/sglang/disagg/perf.yaml`:
- Line 69: The pip install step for aiperf in the benchmark setup is not
arm64-safe because aiperf==0.10.0 pulls source-only dependencies on Linux
aarch64 and will fail in the non-root user install path. Update the perf setup
to either use an arm64-capable base image that already includes the required
build toolchain or switch the aiperf dependency to a version with an arm64
wheel, keeping the change in the benchmark install block that also installs
transformers and tokenizers.
In `@recipes/gpt-oss-120b/trtllm/agg/perf.yaml`:
- Line 32: The perf recipe installs aiperf directly in the runtime image, but on
arm64 this can fail because crick is built from sdist and the base image lacks a
compiler toolchain. Update the setup around the pip install in the perf.yaml
recipe to ensure gcc/build-essential (or an equivalent build toolchain) is
installed before invoking pip install aiperf, or otherwise use a prebuilt image
with aiperf already baked in; keep the change localized to the aiperf install
step in this recipe.
In `@recipes/gpt-oss-120b/trtllm/disagg/perf.yaml`:
- Line 32: Add the compiler toolchain step before the aiperf installation in the
perf setup so Linux arm64 can build crick from source on GB200 nodes. Update the
startup sequence around the pip install for aiperf==0.10.0 to ensure
build-essential/gcc is present, or otherwise use a prebuilt image/dependency
path, so the install in this perf.yaml job no longer fails during startup.
In `@recipes/kimi-k2.5/trtllm/agg-eagle-kv-router/perf.yaml`:
- Line 54: The startup command in the perf job installs a package set that can
fail on arm64 because aiperf==0.10.0 brings in crick, which needs to build from
source on Linux aarch64. Update the job setup around the pip install step to
either install the required build toolchain and compiler headers before running
pip, or constrain this workload to amd64 via the relevant job/pod configuration.
Use the pip install command in the perf.yaml startup sequence as the anchor for
the fix.
In `@recipes/qwen3-32b-fp8/vllm/disagg/perf.yaml`:
- Line 22: The perf job installs aiperf in the Qwen disagg setup, but the base
image lacks a native build toolchain needed when crick is built from source on
Linux arm64. Update the setup step around the pip install in the disagg perf
workflow to install build-essential (or equivalent compiler tooling) before
invoking aiperf, or otherwise constrain the job to amd64 so the install path
does not require source compilation.
In `@recipes/qwen3-32b/vllm/agg-round-robin/perf.yaml`:
- Line 23: The perf job setup in the aiperf install step assumes aarch64 can
build source-only dependencies, but the container currently only installs
runtime tools and will fail on arm64. Update the job definition around the pip
install aiperf==0.10.0 step by either adding the required Linux build toolchain
to the image/setup or constraining this manifest to amd64 so the install path is
only used on supported architecture.
In `@recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml`:
- Line 31: The benchmark setup in perf.yaml installs aiperf without ensuring
arm64 build dependencies are present, which can break on python:3.12-slim when
crick==0.0.8 falls back to a source build. Update the install step around the
aiperf command to add gcc and libc6-dev for arm64 before pip install, or
otherwise constrain the job to amd64; use the existing benchmark setup block to
keep the change localized.
---
Outside diff comments:
In `@recipes/deepseek-r1/trtllm/disagg/wide_ep/gb200/perf.yaml`:
- Around line 28-34: The shell block in the benchmark command needs fail-fast
behavior so installation or profiling errors do not get masked by later
commands. Update the script body used by the command sequence to enable
immediate exit on failure (or add explicit exits around the critical
install/profile steps) before the new aiperf pin is used, so failures in the pip
install or subsequent benchmark commands stop the Job instead of reaching the
trailing echo/ls. Use the command block under the perf YAML entry to apply the
fix consistently.
In `@recipes/qwen3-235b-a22b-fp8/trtllm/disagg/hopper/perf.yaml`:
- Around line 27-33: The shell block in the command script can continue past
failed setup or profiling steps because it runs under /bin/sh without fail-fast
behavior. Update the block in the perf config to enable immediate failure at the
start of the script body, or explicitly guard the critical commands so that the
apt-get, pip install of aiperf, and any subsequent aiperf profile invocation
cannot fall through to later echo/ls steps; use the shell command block itself
as the fix point.
In `@recipes/qwen3-32b-fp8/trtllm/agg/perf.yaml`:
- Around line 27-33: The shell block in the command script can continue after a
failed install or benchmark step because it runs under /bin/sh without fail-fast
behavior. Update the block that installs aiperf and runs the benchmark so it
exits immediately on error, either by enabling strict shell mode at the start or
by explicitly guarding the critical commands with immediate exits. Use the
existing command block and the aiperf install/profile steps as the key places to
apply the fix.
In `@recipes/qwen3-32b-fp8/trtllm/disagg/perf.yaml`:
- Around line 27-33: The shell block in the perf command currently runs under
/bin/sh without fail-fast behavior, so failures in the apt-get or pip install
steps can still reach later commands and mask a bad benchmark run. Update the
command block to fail immediately before the aiperf pin is used by adding
fail-fast handling at the top of the script block in the command array (or
explicitly guarding the critical commands with exit-on-failure) so the benchmark
job cannot continue after a failed install or profile step.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 0ac9c808-0e4b-4167-a66d-c5d93b44e27a
📒 Files selected for processing (41)
benchmarks/frontend/scripts/README.mdbenchmarks/frontend/scripts/sweep_k8s/aiperf.pybenchmarks/pyproject.tomlcontainer/deps/requirements.benchmark.txtcontainer/templates/frontend.Dockerfiledocs/benchmarks/deepseek-v3-2-wideep-routing.mdxdocs/benchmarks/kv-router-ab-testing.mddocs/recipes/deepseek-v3-2-nvfp4.mdxrecipes/deepseek-r1/trtllm/disagg/wide_ep/gb200/perf.yamlrecipes/deepseek-v32-fp4/trtllm/agg-round-robin/perf.yamlrecipes/deepseek-v32-fp4/trtllm/disagg-kv-router/perf.yamlrecipes/deepseek-v4/deepseek-v4-pro/vllm/disagg/gb200/perf.yamlrecipes/glm-5-nvfp4/sglang/disagg/efa/perf.yamlrecipes/glm-5-nvfp4/sglang/disagg/perf.yamlrecipes/gpt-oss-120b/trtllm/agg/perf.yamlrecipes/gpt-oss-120b/trtllm/disagg/perf.yamlrecipes/kimi-k2.5/trtllm/agg-eagle-kv-router/perf.yamlrecipes/kimi-k2.5/trtllm/agg-eagle-round-robin/perf.yamlrecipes/kimi-k2.5/trtllm/agg-round-robin/perf.yamlrecipes/kimi-k2.5/trtllm/disagg-eagle-kv-router/perf.yamlrecipes/kimi-k2.6/perf/perf.yamlrecipes/llama-3-70b/vllm/agg/perf.yamlrecipes/llama-3-70b/vllm/disagg-multi-node/perf.yamlrecipes/llama-3-70b/vllm/disagg-single-node/perf.yamlrecipes/nemotron-3-super/perf/perf.yamlrecipes/nemotron-3-ultra/perf/README.mdrecipes/nemotron-3-ultra/perf/perf.yamlrecipes/qwen3-235b-a22b-fp8/trtllm/agg/blackwell/perf.yamlrecipes/qwen3-235b-a22b-fp8/trtllm/agg/hopper/perf.yamlrecipes/qwen3-235b-a22b-fp8/trtllm/disagg/blackwell/perf.yamlrecipes/qwen3-235b-a22b-fp8/trtllm/disagg/hopper/perf.yamlrecipes/qwen3-32b-fp8/trtllm/agg/perf.yamlrecipes/qwen3-32b-fp8/trtllm/disagg/perf.yamlrecipes/qwen3-32b-fp8/vllm/disagg/perf.yamlrecipes/qwen3-32b/vllm/agg-round-robin/perf.yamlrecipes/qwen3-32b/vllm/disagg-kv-router/perf.yamlrecipes/qwen3-vl-30b/vllm/agg-embedding-cache/perf.yamlrecipes/qwen3-vl-32b-fp8/benchmark/perf.yamlrecipes/qwen3.6-35b/README.mdrecipes/qwen3.6-35b/perf.yamltests/fault_tolerance/deploy/client.py
| - | | ||
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||
| pip install git+https://github.com/ai-dynamo/aiperf.git@54cd6dc820bff8bfebc875da104e59d745e14f75; | ||
| pip install "aiperf==0.10.0"; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '--- perf.yaml context ---\n'
sed -n '1,120p' recipes/gpt-oss-120b/trtllm/agg/perf.yaml
printf '\n--- aiperf references in repo ---\n'
rg -n "aiperf|crick|build-essential|gcc|python:3.12-slim|arm64|aarch64" recipes/gpt-oss-120b -S || trueRepository: ai-dynamo/dynamo
Length of output: 245
🌐 Web query:
aiperf 0.10.0 PyPI Linux aarch64 crick sdist build toolchain install notes
💡 Result:
When installing AIPerf version 0.10.0 on Linux aarch64 (arm64) systems, you must install the system build toolchain before running pip install because the dependency crick is provided only as a source distribution (sdist) and requires a C compiler at installation time [1][2][3]. For Debian or Ubuntu-based distributions, install the necessary tools using [1][2][3]: sudo apt install build-essential For RHEL or CentOS-based distributions, use [1][2][3]: sudo yum groupinstall "Development Tools" Once the build toolchain is present, you can proceed with the standard installation in a virtual environment [1][2][3]: python3 -m venv venv source venv/bin/activate pip install aiperf This requirement is specific to Linux aarch64 environments. Other platforms, such as Linux x86_64, macOS, and Windows, typically install from pre-built wheels and do not require a system build toolchain [1][2][3].
Citations:
- 1: https://pypi.org/project/aiperf/0.10.0/
- 2: https://github.com/ai-dynamo/aiperf
- 3: https://github.com/ai-dynamo/aiperf/blob/main/README.md
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,120p' recipes/gpt-oss-120b/trtllm/agg/perf.yaml
printf '%s\n' '--- nearby aiperf references ---'
rg -n "aiperf|python:3.12-slim|build-essential|gcc|arm64|aarch64" recipes/gpt-oss-120b/trtllm/agg/perf.yaml recipes/gpt-oss-120b -S || trueRepository: ai-dynamo/dynamo
Length of output: 6733
Install a build toolchain before pip install aiperf on arm64.
This recipe targets ARM64 nodes, and aiperf==0.10.0 still needs a system compiler there because crick installs from sdist. python:3.12-slim doesn’t include gcc/build-essential, so the runtime install will fail unless the toolchain is added or aiperf is baked into the image.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@recipes/gpt-oss-120b/trtllm/agg/perf.yaml` at line 32, The perf recipe
installs aiperf directly in the runtime image, but on arm64 this can fail
because crick is built from sdist and the base image lacks a compiler toolchain.
Update the setup around the pip install in the perf.yaml recipe to ensure
gcc/build-essential (or an equivalent build toolchain) is installed before
invoking pip install aiperf, or otherwise use a prebuilt image with aiperf
already baked in; keep the change localized to the aiperf install step in this
recipe.
| ulimit -n 600000 | ||
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||
| pip install "aiperf==0.7.0" protobuf "transformers==4.57.3" | ||
| pip install "aiperf==0.10.0" protobuf "transformers==4.57.3" |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== perf.yaml context =="
sed -n '1,140p' recipes/kimi-k2.5/trtllm/agg-eagle-kv-router/perf.yaml | cat -n
echo
echo "== search for aiperf/crick references =="
rg -n "aiperf|crick|arm64|aarch64|python:3.12-slim" recipes/kimi-k2.5/trtllm/agg-eagle-kv-router -S || trueRepository: ai-dynamo/dynamo
Length of output: 8115
🌐 Web query:
aiperf 0.10.0 PyPI Linux aarch64 crick sdist build toolchain arm64
💡 Result:
AIPerf version 0.10.0 is available on PyPI [1]. When installing aiperf on Linux aarch64 (ARM64) systems, you must install a system build toolchain before running pip install aiperf [2][3]. This requirement exists because one of the package's dependencies, crick, does not provide pre-built wheels for the aarch64 architecture and instead ships only a source distribution (sdist) [2][3]. Consequently, pip must compile the extension at install time, which requires a C compiler and development headers present on the host system [2][3]. To prepare your environment, install the appropriate build toolchain for your distribution: - Debian/Ubuntu: sudo apt install build-essential [2][3] - RHEL/CentOS: sudo yum groupinstall "Development Tools" [2][3] In contrast, Linux x86_64, macOS, and Windows environments typically install aiperf using pre-built wheels, which do not require a local build toolchain [2][3].
Citations:
- 1: https://pypi.org/project/aiperf/0.10.0/
- 2: https://github.com/ai-dynamo/aiperf/blob/main/README.md
- 3: https://pypi.org/project/aiperf-nightly/
Install a build toolchain before pip install on arm64. aiperf==0.10.0 pulls in crick, which only ships an sdist on Linux aarch64, so this python:3.12-slim job will fail during startup on arm64 unless you add build-essential/compiler headers first or pin the pod to amd64.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@recipes/kimi-k2.5/trtllm/agg-eagle-kv-router/perf.yaml` at line 54, The
startup command in the perf job installs a package set that can fail on arm64
because aiperf==0.10.0 brings in crick, which needs to build from source on
Linux aarch64. Update the job setup around the pip install step to either
install the required build toolchain and compiler headers before running pip, or
constrain this workload to amd64 via the relevant job/pod configuration. Use the
pip install command in the perf.yaml startup sequence as the anchor for the fix.
| - | | ||
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||
| pip install git+https://github.com/ai-dynamo/aiperf.git@54cd6dc820bff8bfebc875da104e59d745e14f75; | ||
| pip install "aiperf==0.10.0"; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== perf.yaml =="
cat -n recipes/qwen3-32b-fp8/vllm/disagg/perf.yaml
echo
echo "== file stats =="
wc -l recipes/qwen3-32b-fp8/vllm/disagg/perf.yaml
echo
echo "== nearby references to aiperf =="
rg -n "aiperf|crick|aarch64|arm64|gcc|build-essential" recipes/qwen3-32b-fp8 -S || trueRepository: ai-dynamo/dynamo
Length of output: 7872
🌐 Web query:
aiperf 0.10.0 aarch64 crick sdist build toolchain pip install
💡 Result:
When installing AIPerf version 0.10.0 on Linux aarch64 (arm64) systems, you must install the system build toolchain before running pip install because the dependency crick ships only as an sdist (source distribution) and requires a C compiler to build during the installation process [1][2][3]. For common Linux distributions, you can install the necessary build toolchain using the following commands: - Debian/Ubuntu: sudo apt install build-essential [1][3] - RHEL/CentOS: sudo yum groupinstall "Development Tools" [1][3] This requirement is specific to Linux aarch64 architectures. On Linux x86_64, macOS, and Windows, AIPerf is installed via pre-built wheels and does not require a system build toolchain [1][2][4].
Citations:
- 1: https://pypi.org/project/aiperf/0.10.0/
- 2: https://github.com/ai-dynamo/aiperf/blob/main/README.md
- 3: https://pypi.org/project/aiperf/
- 4: https://github.com/ai-dynamo/aiperf
🌐 Web query:
PyPI aiperf 0.10.0 crick aarch64 build dependencies
💡 Result:
When installing aiperf 0.10.0 on aarch64 (ARM64) Linux, the dependency crick does not provide a pre-built wheel, forcing an installation from the source distribution (sdist) [1][2][3]. As a result, you must have a C compiler and the system build toolchain installed on your host system before running pip install aiperf [1][2][4]. To satisfy these build requirements, you should install the necessary development tools for your specific Linux distribution: - For Debian or Ubuntu-based systems: sudo apt install build-essential [1][3] - For RHEL, CentOS, or compatible systems: sudo yum groupinstall "Development Tools" [1][3] For other platforms, such as Linux x86_64, macOS, and Windows, aiperf and its dependencies typically install from pre-built wheels, meaning no additional system-level C compiler is required [1][4].
Citations:
- 1: https://pypi.org/project/aiperf/0.10.0/
- 2: https://pypi.org/project/aiperf/
- 3: https://github.com/ai-dynamo/aiperf/blob/main/README.md
- 4: https://github.com/ai-dynamo/aiperf
Install a build toolchain before pip install aiperf aiperf==0.10.0 pulls in crick, which builds from source on Linux arm64. python:3.12-slim only installs curl/jq/procps/git here, so this job will fail on arm64 unless you add build-essential or pin the job to amd64.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@recipes/qwen3-32b-fp8/vllm/disagg/perf.yaml` at line 22, The perf job
installs aiperf in the Qwen disagg setup, but the base image lacks a native
build toolchain needed when crick is built from source on Linux arm64. Update
the setup step around the pip install in the disagg perf workflow to install
build-essential (or equivalent compiler tooling) before invoking aiperf, or
otherwise constrain the job to amd64 so the install path does not require source
compilation.
There was a problem hiding this comment.
added a comment
#NOTE: aiperf>=0.8.0 depends on crick (no prebuilt arm64 wheel); on arm64
# nodes the apt-get step above must include gcc and libc6-dev so pip can
# compile crick from source.
There was a problem hiding this comment.
@pvijayakrish, thanks for adding the note — that helps document the caveat.
I don’t think this fully resolves the issue, though. At Line 22 the job still installs aiperf==0.10.0 after only installing curl jq procps git, so on an arm64 node it would still fail unless the command also installs the compiler headers/toolchain.
If this job can run on arm64, please update the apt-get install line to include the required packages (for example gcc and libc6-dev, or build-essential). If it is guaranteed to run only on amd64, then constraining the job to amd64 would also address it.
🧠 Learnings used
Learnt from: karen-sy
Repo: ai-dynamo/dynamo PR: 7915
File: recipes/qwen3-32b-fp8/vllm/disagg/deploy.yaml:13-25
Timestamp: 2026-04-09T05:39:40.946Z
Learning: For vLLM disaggregated deployment recipes under recipes/**/vllm/disagg, avoid setting `envFromSecret: hf-token-secret` (HuggingFace token) on the Frontend service. The HuggingFace token secret should only be provided to worker components (e.g., `VllmPrefillWorker`, `VllmDecodeWorker` or any component that actually pulls/loads model weights). Since the Frontend only routes requests, it should not receive the HF token secret. In reviews, flag Frontend/HPA/frontend pods that include `envFromSecret: hf-token-secret` unless there is a clear, automated need to access model assets in the Frontend.
Learnt from: karen-sy
Repo: ai-dynamo/dynamo PR: 7915
File: recipes/qwen3-32b-fp8/vllm/disagg/deploy.yaml:13-25
Timestamp: 2026-04-09T05:39:40.946Z
Learning: For vLLM disaggregated deployment recipes in ai-dynamo/dynamo (files under recipes/**/vllm/disagg/), do not add the `--disaggregation-mode decode` flag to the `VllmDecodeWorker` args unless there is a confirmed routing/handler behavior change. Dynamo routes requests to the same decode handler regardless of whether the flag is explicitly set, so omitting the flag in these recipes is intentional and should not be treated as a bug during review.
Learnt from: tanmayv25
Repo: ai-dynamo/dynamo PR: 8584
File: lib/backend-common/examples/rotating_tokens/docker-compose.yml:41-48
Timestamp: 2026-04-23T01:22:51.416Z
Learning: In this repository, `bitnamilegacy/etcd` is an intentional, repo-wide convention for deployments. When reviewing YAML files (e.g., docker-compose, Helm `values.yaml`, ECS task/compose manifests), do not flag occurrences of `bitnamilegacy/etcd` as issues or deviations from the standard. Treat any future migration away from it as a holistic, coordinated change documented in `deploy/helm/charts/platform/README.md`, not as isolated per-file replacements.
Learnt from: ritazh
Repo: ai-dynamo/dynamo PR: 8355
File: recipes/qwen3.6-35b-a3b/vllm/agg/deploy.yaml:43-55
Timestamp: 2026-05-03T17:54:20.684Z
Learning: In this repository, vLLM deployment recipe YAMLs that reference the vLLM runtime image should use the planned stable image tag `nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.1.0` (which bundles vLLM 0.19.0+), not the dev tag `nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.1.0-dev.1`. Update image references accordingly in recipes under `recipes/qwen3.6-35b-a3b/` and other similar recipe paths once `1.1.0` is published.
Learnt from: hutm
Repo: ai-dynamo/dynamo PR: 8927
File: recipes/deepseek-r1/trtllm/agg/b200/perf.yaml:35-35
Timestamp: 2026-05-29T00:27:15.634Z
Learning: For benchmark/performance Kubernetes Job manifests named `perf.yaml` in the ai-dynamo/dynamo repo, reviewers should treat using `sysctl -w net.ipv4.ip_local_port_range="1024 65000"` inside the container’s shell command/entrypoint together with `securityContext.privileged: true` as an intentional repo-wide convention. Do not flag or suggest changing either setting on a per-file basis, since using pod-level `securityContext.sysctls` for `net.*` would require kubelet `--allowed-unsafe-sysctls` allowlisting that may not be available on target clusters. If you want to harden this behavior, propose it as a coordinated change across all `perf.yaml` files holistically (not a single-file deviation).
|
|
||
| # Install benchmarking tool | ||
| pip install aiperf==0.6.0 | ||
| pip install aiperf==0.10.0 |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
sed -n '1,80p' recipes/qwen3-32b/vllm/agg-round-robin/perf.yamlRepository: ai-dynamo/dynamo
Length of output: 2939
🌐 Web query:
aiperf 0.10.0 Linux aarch64 crick sdist build toolchain install notes
💡 Result:
When installing AIPerf 0.10.0 on a Linux aarch64 (ARM64) system, you must manually install a C build toolchain before running the installation command. This requirement exists because one of AIPerf's dependencies, crick, is distributed only as a source distribution (sdist) and requires compilation at install time [1][2][3]. To prepare your environment, install the appropriate build toolchain for your distribution: For Debian/Ubuntu systems: sudo apt install build-essential For RHEL/CentOS systems: sudo yum groupinstall "Development Tools" Once the toolchain is installed, you can proceed with the standard installation of AIPerf using pip: pip install aiperf Please note that this step is specific to Linux aarch64 environments. Other platforms, such as Linux x86_64, macOS, and Windows, receive pre-built wheels and do not require a local C compiler toolchain during the installation process [1][2][4].
Citations:
- 1: https://pypi.org/project/aiperf/0.10.0/
- 2: https://github.com/ai-dynamo/aiperf
- 3: https://github.com/ai-dynamo/aiperf/blob/main/README.md
- 4: https://pypi.org/project/aiperf/
Add a build toolchain or pin this job to amd64
aiperf 0.10.0 needs a local C toolchain on Linux aarch64 because crick is source-only there. This pod only installs tmux, wget, curl, and jq into python:3.11, so arm64 runs will fail at pip install aiperf==0.10.0 unless you bake in the compiler tools or restrict the manifest to amd64.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@recipes/qwen3-32b/vllm/agg-round-robin/perf.yaml` at line 23, The perf job
setup in the aiperf install step assumes aarch64 can build source-only
dependencies, but the container currently only installs runtime tools and will
fail on arm64. Update the job definition around the pip install aiperf==0.10.0
step by either adding the required Linux build toolchain to the image/setup or
constraining this manifest to amd64 so the install path is only used on
supported architecture.
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||
|
|
||
| pip install aiperf==0.9.0 | ||
| pip install aiperf==0.10.0 |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== perf.yaml relevant lines ==\n'
sed -n '1,260p' recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml | nl -ba | sed -n '1,120p'
printf '\n== sweep_k8s/aiperf.py relevant lines ==\n'
sed -n '1,220p' benchmarks/frontend/scripts/sweep_k8s/aiperf.py | nl -ba | sed -n '1,140p'
printf '\n== frontend.Dockerfile relevant lines ==\n'
sed -n '1,120p' container/templates/frontend.Dockerfile | nl -ba | sed -n '1,80p'
printf '\n== search for aiperf / crick references ==\n'
rg -n "aiperf|crick|libc6-dev|gcc" recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml benchmarks/frontend/scripts/sweep_k8s/aiperf.py container/templates/frontend.Dockerfile .ai -S || trueRepository: ai-dynamo/dynamo
Length of output: 226
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import json, urllib.request
urls = [
"https://pypi.org/pypi/aiperf/0.10.0/json",
"https://pypi.org/pypi/crick/0.0.8/json",
]
for url in urls:
try:
with urllib.request.urlopen(url, timeout=15) as r:
data = json.load(r)
print(f"== {url} ==")
print("name:", data["info"]["name"], "version:", data["info"]["version"])
files = data["releases"].get(data["info"]["version"], [])
for f in files:
print(f["filename"], f.get("packagetype"), f.get("python_version"))
except Exception as e:
print(f"FAILED {url}: {e}")
PYRepository: ai-dynamo/dynamo
Length of output: 518
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== perf.yaml relevant lines ==\n'
sed -n '1,260p' recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml | cat -n | sed -n '1,120p'
printf '\n== sweep_k8s/aiperf.py relevant lines ==\n'
sed -n '1,220p' benchmarks/frontend/scripts/sweep_k8s/aiperf.py | cat -n | sed -n '1,140p'
printf '\n== frontend.Dockerfile relevant lines ==\n'
sed -n '1,120p' container/templates/frontend.Dockerfile | cat -n | sed -n '1,80p'
printf '\n== search for aiperf / crick / arm64 install deps ==\n'
rg -n "aiperf|crick|libc6-dev|gcc|aarch64|arm64" recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml benchmarks/frontend/scripts/sweep_k8s/aiperf.py container/templates/frontend.Dockerfile .ai -S || trueRepository: ai-dynamo/dynamo
Length of output: 21825
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== perf.yaml container/spec tail ==\n'
sed -n '120,260p' recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml | cat -n
printf '\n== perf.yaml architecture/node scheduling hints ==\n'
rg -n "nodeSelector|affinity|arch|arm64|amd64|kubernetes.io/arch|node-name|tolerations" recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml -S || true
printf '\n== PyPI simple pages for crick==0.0.8 and aiperf==0.10.0 (wheel filenames) ==\n'
python3 - <<'PY'
import ssl, urllib.request, re
ctx = ssl._create_unverified_context()
for url in [
"https://pypi.org/simple/aiperf/",
"https://pypi.org/simple/crick/",
]:
try:
with urllib.request.urlopen(url, context=ctx, timeout=20) as r:
html = r.read().decode("utf-8", "replace")
print(f"URL: {url}")
for needle in ["aiperf-0.10.0", "crick-0.0.8"]:
if needle in html:
print(f" contains {needle}")
# print matching wheel links around versions of interest
for pattern in [r'href="([^"]*aiperf-0\.10\.0[^"]*)"', r'href="([^"]*crick-0\.0\.8[^"]*)"' ]:
hits = re.findall(pattern, html, flags=re.I)
for h in hits[:20]:
print(" ", h)
except Exception as e:
print(f"FAILED {url}: {e}")
PYRepository: ai-dynamo/dynamo
Length of output: 8518
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import ssl, urllib.request, json, re
ctx = ssl._create_unverified_context()
for pkg, ver in [("aiperf","0.10.0"), ("crick","0.0.8")]:
url = f"https://pypi.org/pypi/{pkg}/{ver}/json"
try:
with urllib.request.urlopen(url, context=ctx, timeout=20) as r:
data = json.load(r)
print(f"== {pkg} {ver} ==")
files = data.get("releases", {}).get(ver, [])
if not files:
files = data.get("urls", [])
for f in files:
fn = f.get("filename", "")
if fn:
print(fn)
except Exception as e:
print(f"FAILED {url}: {e}")
PYRepository: ai-dynamo/dynamo
Length of output: 1790
Install gcc/libc6-dev on arm64 before pip install aiperf==0.10.0.
aiperf is a pure-Python wheel, but its crick==0.0.8 dependency has no manylinux aarch64 wheel, so an unconstrained pod on python:3.12-slim can fail during the install step on arm64. Add the arm64 build deps here or pin the Job to amd64.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml` at line 31, The benchmark setup
in perf.yaml installs aiperf without ensuring arm64 build dependencies are
present, which can break on python:3.12-slim when crick==0.0.8 falls back to a
source build. Update the install step around the aiperf command to add gcc and
libc6-dev for arm64 before pip install, or otherwise constrain the job to amd64;
use the existing benchmark setup block to keep the change localized.
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
This comment has been minimized.
This comment has been minimized.
3 participants
Overview:
Upgrade the aiperf version to 0.10.0 in dynamo repo.
Summary by CodeRabbit
Bug Fixes
Documentation