kimik2.5-fp4-b300-vllm: bump vLLM image to v0.22.0 and expand concurrency sweep - #1674
Conversation
Update the kimik2.5-fp4-b300-vllm vLLM image from v0.21.0 to v0.22.0. Appends a perf-changelog entry.
…m-image-v0.22.0 # Conflicts: # perf-changelog.yaml
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🔴
perf-changelog.yaml:3478-3482— The new perf-changelog entry has two issues to fix before merge: (1)pr-linkis still the literal placeholderhttps://github.com/SemiAnalysisAI/InferenceX/pull/<TODO>(the PR description acknowledges this) — should be/pull/1674. (2) The file no longer ends with a trailing newline (\ No newline at end of filein the diff), breaking the line-termination pattern AGENTS.md flags as load-bearing for this file.Extended reasoning...
Bug 1:
<TODO>placeholder in pr-linkThe newly appended entry in
perf-changelog.yaml(lines 3478–3482) ends with:pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/<TODO>
Every other entry in the file uses a real numeric PR number (e.g.
/pull/1544,/pull/1647,/pull/1648,/pull/1663). The PR description itself explicitly notes this: "documenting the image update (PR link still<TODO>)". The result is a dead URL in the changelog pointing tohttps://github.com/SemiAnalysisAI/InferenceX/pull/<TODO>, which is not a valid GitHub PR.Fix: replace
<TODO>with1674(this PR's number).Bug 2: Missing trailing newline at EOF
The diff shows
\ No newline at end of fileimmediately after the new entry. AGENTS.md is explicit about this file:perf-changelog.yaml— benchmark trigger log; append-only; preserve whitespace.Never delete or modify whitespace in
perf-changelog.yaml— CI depends on exact whitespace (including trailing spaces on blank separator lines). Altering it breaks CI.Before this PR the file ended with
...pull/1544\n(trailing newline preserved). After this PR it ends with the bytes...pull/<TODO>with no terminator. Every prior entry in the file is followed by a newline plus a blank separator line before the next entry begins.Step-by-step proof
- Check the previous file termination (HEAD~1):
tail -c 50 perf-changelog.yaml | od -cshowsp u l l / 1 5 4 4 \n. - After this PR's diff is applied, the same command shows
p u l l / < T O D O >with no\n. - Visit
https://github.com/SemiAnalysisAI/InferenceX/pull/<TODO>in a browser → 404, not a real PR. - If a follow-up PR appends another entry without first restoring the newline, the YAML stream will concatenate as
pull/<TODO>- config-keys:on a single line, producing a malformed entry.
Impact
- The changelog contains a broken URL that should point to PR #1674.
- The whitespace deviation contradicts AGENTS.md's load-bearing guidance for this exact file, and sets up the next appended entry to produce a malformed line concatenation.
- YAML parsing itself still succeeds today (
yaml.safe_loadtolerates a missing trailing newline), so this is unlikely to break CI immediately, but it violates the documented convention and the file's consistent pattern.
Fix
- config-keys: - kimik2.5-fp4-b300-vllm description: - "Update vLLM image from v0.21.0 to v0.22.0" pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1674
…followed by a trailing newline (and ideally the same blank-separator-line pattern used between prior entries).
- Check the previous file termination (HEAD~1):
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27042150694 |
TP4/EP1 conc 4-64 -> 1-128 and TP8/EP1 conc-start 4 -> 1 for the 1k/1k and 8k/1k cells. Appends a perf-changelog entry.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27044972506 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27045004683 |
RohitNagraj
changed the title
|
/reuse-sweep-run |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27077682128 |
2 participants
Two changes to
kimik2.5-fp4-b300-vllm:vllm/vllm-openai:v0.21.0→vllm/vllm-openai:v0.22.0.conc 4-64→1-128, TP8/EP1conc-start 4→1(conc-end 4unchanged).Appends a perf-changelog entry.
Note
Low Risk
Benchmark-only YAML and changelog updates; no runtime application or auth/data-path changes, though wider sweeps increase benchmark run time and cost.
Overview
Updates the
kimik2.5-fp4-b300-vllmbenchmark recipe innvidia-master.yaml: the vLLM container moves from v0.21.0 to v0.22.0, and the fixed-seq-len concurrency sweeps for the 1k/1k and 8k/1k cells are widened so low-concurrency points are included and TP4 can be searched up to 128 concurrent requests (previously TP4 started at 4 and capped at 64; TP8 now sweeps from 1 through 4 instead of starting at 4).A matching
perf-changelog.yamlentry documents the image bump and sweep changes forkimik2.5-fp4-b300-vllm.Reviewed by Cursor Bugbot for commit 014787e. Bugbot is set up for automated code reviews on this repo. Configure here.