[NVIDIA] Update vLLM version to latest v0.10.2 on H200. - #11
Merged
Conversation
Collaborator
|
lgtm |
cquil11
changed the title
Oseltamivir
added a commit
that referenced
this pull request
Aug 7, 2026
queries.md #11, the two concrete bugs. The registry consolidation the item also proposes is deliberately NOT here -- see below. Bug 1: BACKEND_PRECISIONS.get(target, ("bf16",)) meant a backend added to platform_config but missing from BACKEND_PRECISIONS silently produced a BF16-only matrix. That is a MISSING case, not a mislabelled one, which matters because the two fail differently: run_sweep's non-bf16-dispatch guard catches a case that ran under the wrong precision, but nothing anywhere can catch a case that was never emitted. The FP8 coverage would simply be absent and every gate would stay green. Now the matrix refuses to resolve and names the backend. Bug 2: the comment above the flashinfer-ep entry still read "FlashInfer one-sided is BF16-only this pass", directly above ("bf16", "fp8"). The pre-FP8 text had been left in place when the FP8 lines were appended, so the file asserted both. DEFERRED, with reasons, since the item asks for more than this: - The BACKEND_SPECS registry. The duplication is real -- BACKEND_PRECISIONS mirrors the adapters' SUPPORTED_PRECISIONS, and platform_config's backend_maturity carries a comment pointing at EPBackend.maturity. But the mirror has a structural cause, not laziness: sweep_matrix must resolve the matrix with no torch or vendor imports, while the authoritative values live on adapter classes that import deep_ep/mori/nccl. Fixing it properly means a vendor-free spec module plus a config-schema migration across the generator, adapters, workflow dispatch and tests -- no behavioural change, at the tail of a PR whose current shape is validated fleet-wide green. It belongs in its own PR where a re-validation is affordable. One note for whoever does it: ll_backends must STAY in the platform map. Low-latency needs NVSHMEM/IBGDA per SKU, so modes-per-backend is genuinely placement-dependent; only precisions and maturity belong in the spec module. Tests: the guard (verified failing when the entry is restored to a .get default), and a standing assertion that every scheduled backend declares its precisions -- which is the invariant the guard exists to protect, checked directly rather than only via the error path. 170 tests green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates the 70b_fp8_h200_slurm.sh benchmark script to use a configuration file approach and fixes critical dependency issues that were causing benchmark failures.