Repository navigation
[Fix] Validate FP8 KV scales before updating layers - #25
Open
rchalamala wants to merge 1 commit into
Open
rchalamala wants to merge 1 commit into
rchalamala wants to merge 1 commit into
Conversation
Reject non-scalar and non-finite checkpoint scales with explicit errors. Validate converted scales before storing them so FNUZ scale doubling cannot write infinities into the layer. Preserve existing default and single-scale handling. Backport the final scale-validation behavior from sgl-project#40243. Extend regression coverage to both scale tensors, empty tensors, unchanged parameters after rejection, and existing default-scale behavior. Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Author
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
rchalamala
marked this pull request as ready for review
September 25, 2026 06:37
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
rchalamala
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Invalid FP8 KV checkpoint scales can reach tensor comparisons or be stored as non-finite values. FNUZ conversion can also double an otherwise finite scale beyond float32 range.
Modifications
Backport the scale-validation behavior from sgl-project/sglang#40243: reject unsupported scale shapes and non-finite values, then verify converted scales fit the destination dtype before updating the layer. Preserve existing default-scale and single-scale handling.
Accuracy Tests
CPU validation:
The neighboring compiled CPU tests were run with the system C++ compiler. The default compiler produced a glibc compatibility error on both base and head. No GPU inference tests were run; the change is checkpoint-load validation.
Speed Tests and Profiling
Not applicable: no inference kernel or forward-path changes.
Checklist
CI States
Latest PR Test (Base): ❌ Run #36094995231
Latest PR Test (Extra): ❌ Run #36094995129
Latest PR Test (AMD ROCm 10): ❌ Run #36094995278