Repository navigation
fix(recipes): add build tools for ARM recipes on aiperf>=0.8.0 (#11278) #11333
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -47,15 +47,10 @@ spec: | |||||
| echo "File descriptor limit set to: $(ulimit -n)" | ||||||
| echo 2097152 > /proc/sys/fs/inotify/max_user_watches 2>/dev/null || true | ||||||
| echo 1024 > /proc/sys/fs/inotify/max_user_instances 2>/dev/null || true | ||||||
| apt-get update && apt-get install -y curl jq procps git && apt-get clean | ||||||
| # pip install git+https://github.com/ai-dynamo/aiperf.git | ||||||
| # Pin transformers==4.57.6 (verified to load the deepseek_v32 tokenizer). | ||||||
| # The trtllm-runtime base ships transformers 4.55.0; aiperf 0.6.0's | ||||||
| # `transformers>=4.56.0` floor would otherwise upgrade to 5.x, which lacks | ||||||
| # native support for model_type=deepseek_v32 (huggingface/transformers#41251) | ||||||
| # and fails AutoTokenizer.from_pretrained() before reading tokenizer.json. | ||||||
| # aiperf surfaces this as "Failed to load tokenizer". DYN-2878. | ||||||
| pip install "aiperf==0.10.0" "transformers==4.57.6" | ||||||
| # aiperf>=0.8.0 depends on crick, which has no Linux aarch64 wheel. | ||||||
| # GB200 benchmark pods need build tools to compile it from source. | ||||||
| apt-get update && apt-get install -y curl jq procps git build-essential && apt-get clean | ||||||
| pip install "aiperf==0.10.0" | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Tokenizer version pin removed for DeepSeek-V3.2 disaggregated recipe, likely breaking tokenizer loading The required version pin for the tokenizer library is removed ( Impact: The DeepSeek-V3.2 disaggregated benchmark job will crash at startup with a "Failed to load tokenizer" error. Detailed mechanism: same transformers 5.x incompatibility as the agg recipeThis is the same issue as in the agg-round-robin recipe. The previous code pinned
Suggested change
Was this helpful? React with 👍 or 👎 to provide feedback. |
||||||
| echo "aiperf installation completed" | ||||||
| sysctl -w net.ipv4.ip_local_port_range="1024 65000" 2>/dev/null || true | ||||||
| export COLUMNS=200 | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,9 +22,6 @@ spec: | |
| spec: | ||
| restartPolicy: Never | ||
| securityContext: | ||
| runAsNonRoot: true | ||
| runAsUser: 1000 | ||
| runAsGroup: 1000 | ||
| seccompProfile: | ||
| type: RuntimeDefault | ||
| containers: | ||
|
Comment on lines
25
to
27
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔍 Non-root security constraints removed to enable apt-get Two recipes ( Was this helpful? React with 👍 or 👎 to provide feedback. |
||
|
|
@@ -57,13 +54,6 @@ spec: | |
| value: "1" | ||
| - name: COLUMNS | ||
| value: "200" | ||
| # HOME defaults to "/" for uid 1000 with no /etc/passwd entry; pip --user | ||
| # then tries to write /.local (read-only). Point HOME at /tmp (1777 sticky) | ||
| # so ~/.local resolves under a writable path. | ||
| - name: HOME | ||
| value: /tmp | ||
| - name: PATH | ||
| value: /tmp/.local/bin:/usr/local/bin:/usr/bin:/bin | ||
| envFrom: | ||
| - secretRef: | ||
| name: hf-token-secret | ||
|
|
@@ -72,14 +62,10 @@ spec: | |
| - -c | ||
| - | | ||
| set -eu | ||
| # No apt-get: cluster runs us as non-root, and we don't actually need | ||
| # curl/jq — Python stdlib handles the readiness probe. Install aiperf | ||
| # into HOME=/tmp/.local via --user so it works under runAsNonRoot. | ||
| # NOTE: aiperf>=0.8.0 depends on crick, which has no prebuilt arm64 wheel | ||
| # and requires gcc/libc6-dev to build from source. This pod runs non-root | ||
| # with no apt-get access, so this install will fail on arm64 nodes. | ||
| # Ensure the cluster scheduler places this pod on an x86_64 node. | ||
| pip install --user -q aiperf==0.10.0 transformers==4.57.3 tokenizers==0.22.2 | ||
| # aiperf>=0.8.0 depends on crick, which has no Linux aarch64 wheel. | ||
| # GB200 benchmark pods need build tools to compile it from source. | ||
| apt-get update && apt-get install -y --no-install-recommends build-essential && apt-get clean | ||
| pip install -q aiperf==0.10.0 transformers==4.57.3 tokenizers==0.22.2 | ||
|
|
||
| echo "Waiting for model at http://$ENDPOINT/v1/models..." | ||
| python3 - <<'PY' | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 Tokenizer version pin removed for DeepSeek-V3.2, likely breaking tokenizer loading
The required version pin for the tokenizer library is removed (
pip install "aiperf==0.10.0"atrecipes/deepseek-v32-fp4/trtllm/agg-round-robin/perf.yaml:53) without addressing the known incompatibility, so the benchmark will fail to load the tokenizer.Impact: The DeepSeek-V3.2 aggregated benchmark job will crash at startup with a "Failed to load tokenizer" error.
Detailed mechanism: transformers 5.x lacks deepseek_v32 model_type support
The previous code at both
recipes/deepseek-v32-fp4/trtllm/agg-round-robin/perf.yamlandrecipes/deepseek-v32-fp4/trtllm/disagg-kv-router/perf.yamlexplicitly pinnedtransformers==4.57.6with a detailed comment explaining the issue (see the deleted lines in the diff):The new code removes this pin entirely, doing just
pip install "aiperf==0.10.0". Since aiperf has atransformers>=4.56.0floor dependency, pip will resolve to the latest available transformers (likely 5.x), which lacks native support formodel_type=deepseek_v32.The tokenizer is loaded from HuggingFace (
--tokenizer "nvidia/DeepSeek-V3.2-NVFP4"), meaningAutoTokenizer.from_pretrained()is called and the model_type issue applies.Notably, other recipes in this same PR still pin their transformers versions (e.g., kimi-k2.5 pins
4.57.3, glm-5 pins5.3.0/5.8.0, dsv4-pro pins4.57.3), suggesting this removal was unintentional.Was this helpful? React with 👍 or 👎 to provide feedback.