Repository navigation
chore: upgrade the aiperf version in dynamo repo #11061
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,4 +3,4 @@ | |
|
|
||
| # Benchmark and profiling tool dependencies. | ||
|
|
||
| aiperf==0.8.0 | ||
| aiperf==0.10.0 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -29,7 +29,7 @@ spec: | |
| - -c | ||
| - | | ||
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||
| pip install git+https://github.com/ai-dynamo/aiperf.git@54cd6dc820bff8bfebc875da104e59d745e14f75; | ||
| pip install "aiperf==0.10.0"; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '--- perf.yaml context ---\n'
sed -n '1,120p' recipes/gpt-oss-120b/trtllm/agg/perf.yaml
printf '\n--- aiperf references in repo ---\n'
rg -n "aiperf|crick|build-essential|gcc|python:3.12-slim|arm64|aarch64" recipes/gpt-oss-120b -S || trueRepository: ai-dynamo/dynamo Length of output: 245 🌐 Web query:
💡 Result: When installing AIPerf version 0.10.0 on Linux aarch64 (arm64) systems, you must install the system build toolchain before running pip install because the dependency crick is provided only as a source distribution (sdist) and requires a C compiler at installation time [1][2][3]. For Debian or Ubuntu-based distributions, install the necessary tools using [1][2][3]: sudo apt install build-essential For RHEL or CentOS-based distributions, use [1][2][3]: sudo yum groupinstall "Development Tools" Once the build toolchain is present, you can proceed with the standard installation in a virtual environment [1][2][3]: python3 -m venv venv source venv/bin/activate pip install aiperf This requirement is specific to Linux aarch64 environments. Other platforms, such as Linux x86_64, macOS, and Windows, typically install from pre-built wheels and do not require a system build toolchain [1][2][3]. Citations:
🏁 Script executed: #!/bin/bash
set -euo pipefail
sed -n '1,120p' recipes/gpt-oss-120b/trtllm/agg/perf.yaml
printf '%s\n' '--- nearby aiperf references ---'
rg -n "aiperf|python:3.12-slim|build-essential|gcc|arm64|aarch64" recipes/gpt-oss-120b/trtllm/agg/perf.yaml recipes/gpt-oss-120b -S || trueRepository: ai-dynamo/dynamo Length of output: 6733 Install a build toolchain before This recipe targets ARM64 nodes, and 🤖 Prompt for AI Agents |
||
| echo "aiperf installation completed"; | ||
| sysctl -w net.ipv4.ip_local_port_range="1024 65000" | ||
| cat /proc/sys/net/ipv4/ip_local_port_range | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -51,7 +51,7 @@ spec: | |
| set -euo pipefail | ||
| ulimit -n 600000 | ||
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||
| pip install "aiperf==0.7.0" protobuf "transformers==4.57.3" | ||
| pip install "aiperf==0.10.0" protobuf "transformers==4.57.3" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== perf.yaml context =="
sed -n '1,140p' recipes/kimi-k2.5/trtllm/agg-eagle-kv-router/perf.yaml | cat -n
echo
echo "== search for aiperf/crick references =="
rg -n "aiperf|crick|arm64|aarch64|python:3.12-slim" recipes/kimi-k2.5/trtllm/agg-eagle-kv-router -S || trueRepository: ai-dynamo/dynamo Length of output: 8115 🌐 Web query:
💡 Result: AIPerf version 0.10.0 is available on PyPI [1]. When installing Citations:
Install a build toolchain before 🤖 Prompt for AI Agents |
||
| sysctl -w net.ipv4.ip_local_port_range="1024 65000" 2>/dev/null || true | ||
| export COLUMNS=200 | ||
| EPOCH=$(date +%s) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,7 +19,7 @@ spec: | |
| - -c | ||
| - | | ||
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||
| pip install git+https://github.com/ai-dynamo/aiperf.git@54cd6dc820bff8bfebc875da104e59d745e14f75; | ||
| pip install "aiperf==0.10.0"; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== perf.yaml =="
cat -n recipes/qwen3-32b-fp8/vllm/disagg/perf.yaml
echo
echo "== file stats =="
wc -l recipes/qwen3-32b-fp8/vllm/disagg/perf.yaml
echo
echo "== nearby references to aiperf =="
rg -n "aiperf|crick|aarch64|arm64|gcc|build-essential" recipes/qwen3-32b-fp8 -S || trueRepository: ai-dynamo/dynamo Length of output: 7872 🌐 Web query:
💡 Result: When installing AIPerf version 0.10.0 on Linux aarch64 (arm64) systems, you must install the system build toolchain before running pip install because the dependency crick ships only as an sdist (source distribution) and requires a C compiler to build during the installation process [1][2][3]. For common Linux distributions, you can install the necessary build toolchain using the following commands: - Debian/Ubuntu: sudo apt install build-essential [1][3] - RHEL/CentOS: sudo yum groupinstall "Development Tools" [1][3] This requirement is specific to Linux aarch64 architectures. On Linux x86_64, macOS, and Windows, AIPerf is installed via pre-built wheels and does not require a system build toolchain [1][2][4]. Citations:
🌐 Web query:
💡 Result: When installing aiperf 0.10.0 on aarch64 (ARM64) Linux, the dependency crick does not provide a pre-built wheel, forcing an installation from the source distribution (sdist) [1][2][3]. As a result, you must have a C compiler and the system build toolchain installed on your host system before running pip install aiperf [1][2][4]. To satisfy these build requirements, you should install the necessary development tools for your specific Linux distribution: - For Debian or Ubuntu-based systems: sudo apt install build-essential [1][3] - For RHEL, CentOS, or compatible systems: sudo yum groupinstall "Development Tools" [1][3] For other platforms, such as Linux x86_64, macOS, and Windows, aiperf and its dependencies typically install from pre-built wheels, meaning no additional system-level C compiler is required [1][4]. Citations:
Install a build toolchain before 🤖 Prompt for AI Agents
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added a comment
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I don’t think this fully resolves the issue, though. At Line 22 the job still installs If this job can run on arm64, please update the 🧠 Learnings used |
||
| echo "aiperf installation completed"; | ||
| sysctl -w net.ipv4.ip_local_port_range="1024 65000" | ||
| cat /proc/sys/net/ipv4/ip_local_port_range | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,7 +20,7 @@ spec: | |
| apt update && apt install tmux wget curl jq -y | ||
|
|
||
| # Install benchmarking tool | ||
| pip install aiperf==0.6.0 | ||
| pip install aiperf==0.10.0 | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: sed -n '1,80p' recipes/qwen3-32b/vllm/agg-round-robin/perf.yamlRepository: ai-dynamo/dynamo Length of output: 2939 🌐 Web query:
💡 Result: When installing AIPerf 0.10.0 on a Linux aarch64 (ARM64) system, you must manually install a C build toolchain before running the installation command. This requirement exists because one of AIPerf's dependencies, crick, is distributed only as a source distribution (sdist) and requires compilation at install time [1][2][3]. To prepare your environment, install the appropriate build toolchain for your distribution: For Debian/Ubuntu systems: sudo apt install build-essential For RHEL/CentOS systems: sudo yum groupinstall "Development Tools" Once the toolchain is installed, you can proceed with the standard installation of AIPerf using pip: pip install aiperf Please note that this step is specific to Linux aarch64 environments. Other platforms, such as Linux x86_64, macOS, and Windows, receive pre-built wheels and do not require a local C compiler toolchain during the installation process [1][2][4]. Citations:
Add a build toolchain or pin this job to amd64
🤖 Prompt for AI Agents |
||
|
|
||
| # Wait for model to be ready | ||
| echo "Waiting for model '${MODEL_NAME}' at http://${FRONTEND}:8000/v1/models..." | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -28,7 +28,7 @@ spec: | |
| set -e | ||
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||
|
|
||
| pip install aiperf==0.9.0 | ||
| pip install aiperf==0.10.0 | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== perf.yaml relevant lines ==\n'
sed -n '1,260p' recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml | nl -ba | sed -n '1,120p'
printf '\n== sweep_k8s/aiperf.py relevant lines ==\n'
sed -n '1,220p' benchmarks/frontend/scripts/sweep_k8s/aiperf.py | nl -ba | sed -n '1,140p'
printf '\n== frontend.Dockerfile relevant lines ==\n'
sed -n '1,120p' container/templates/frontend.Dockerfile | nl -ba | sed -n '1,80p'
printf '\n== search for aiperf / crick references ==\n'
rg -n "aiperf|crick|libc6-dev|gcc" recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml benchmarks/frontend/scripts/sweep_k8s/aiperf.py container/templates/frontend.Dockerfile .ai -S || trueRepository: ai-dynamo/dynamo Length of output: 226 🏁 Script executed: #!/bin/bash
set -euo pipefail
python3 - <<'PY'
import json, urllib.request
urls = [
"https://pypi.org/pypi/aiperf/0.10.0/json",
"https://pypi.org/pypi/crick/0.0.8/json",
]
for url in urls:
try:
with urllib.request.urlopen(url, timeout=15) as r:
data = json.load(r)
print(f"== {url} ==")
print("name:", data["info"]["name"], "version:", data["info"]["version"])
files = data["releases"].get(data["info"]["version"], [])
for f in files:
print(f["filename"], f.get("packagetype"), f.get("python_version"))
except Exception as e:
print(f"FAILED {url}: {e}")
PYRepository: ai-dynamo/dynamo Length of output: 518 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== perf.yaml relevant lines ==\n'
sed -n '1,260p' recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml | cat -n | sed -n '1,120p'
printf '\n== sweep_k8s/aiperf.py relevant lines ==\n'
sed -n '1,220p' benchmarks/frontend/scripts/sweep_k8s/aiperf.py | cat -n | sed -n '1,140p'
printf '\n== frontend.Dockerfile relevant lines ==\n'
sed -n '1,120p' container/templates/frontend.Dockerfile | cat -n | sed -n '1,80p'
printf '\n== search for aiperf / crick / arm64 install deps ==\n'
rg -n "aiperf|crick|libc6-dev|gcc|aarch64|arm64" recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml benchmarks/frontend/scripts/sweep_k8s/aiperf.py container/templates/frontend.Dockerfile .ai -S || trueRepository: ai-dynamo/dynamo Length of output: 21825 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== perf.yaml container/spec tail ==\n'
sed -n '120,260p' recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml | cat -n
printf '\n== perf.yaml architecture/node scheduling hints ==\n'
rg -n "nodeSelector|affinity|arch|arm64|amd64|kubernetes.io/arch|node-name|tolerations" recipes/qwen3-vl-32b-fp8/benchmark/perf.yaml -S || true
printf '\n== PyPI simple pages for crick==0.0.8 and aiperf==0.10.0 (wheel filenames) ==\n'
python3 - <<'PY'
import ssl, urllib.request, re
ctx = ssl._create_unverified_context()
for url in [
"https://pypi.org/simple/aiperf/",
"https://pypi.org/simple/crick/",
]:
try:
with urllib.request.urlopen(url, context=ctx, timeout=20) as r:
html = r.read().decode("utf-8", "replace")
print(f"URL: {url}")
for needle in ["aiperf-0.10.0", "crick-0.0.8"]:
if needle in html:
print(f" contains {needle}")
# print matching wheel links around versions of interest
for pattern in [r'href="([^"]*aiperf-0\.10\.0[^"]*)"', r'href="([^"]*crick-0\.0\.8[^"]*)"' ]:
hits = re.findall(pattern, html, flags=re.I)
for h in hits[:20]:
print(" ", h)
except Exception as e:
print(f"FAILED {url}: {e}")
PYRepository: ai-dynamo/dynamo Length of output: 8518 🏁 Script executed: #!/bin/bash
set -euo pipefail
python3 - <<'PY'
import ssl, urllib.request, json, re
ctx = ssl._create_unverified_context()
for pkg, ver in [("aiperf","0.10.0"), ("crick","0.0.8")]:
url = f"https://pypi.org/pypi/{pkg}/{ver}/json"
try:
with urllib.request.urlopen(url, context=ctx, timeout=20) as r:
data = json.load(r)
print(f"== {pkg} {ver} ==")
files = data.get("releases", {}).get(ver, [])
if not files:
files = data.get("urls", [])
for f in files:
fn = f.get("filename", "")
if fn:
print(fn)
except Exception as e:
print(f"FAILED {url}: {e}")
PYRepository: ai-dynamo/dynamo Length of output: 1790 Install
🤖 Prompt for AI Agents |
||
| echo "aiperf installation completed" | ||
|
|
||
| export COLUMNS=200 | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.