[NVIDIA] Added gpt-oss for H200 TRT. - #10
Merged
Merged
Conversation
Collaborator
Author
|
Verified the script works Run link |
cquil11
changed the title
Oseltamivir
added a commit
that referenced
this pull request
Aug 7, 2026
…alidation hook queries.md #10. _encode_dispatch returned (dispatch_payload, oracle_semantic) and existed to let an FP8 adapter send a prequantized payload with a matching oracle. Neither half of that is true any more: every one of the four overrides returns x unchanged, because each adapter quantizes inside dispatch() where production pays it, and each one's oracle half recomputes what semantic_payload already returns. Two code paths that must agree, kept in sync by hand. Verified equivalence per adapter before collapsing, since a mismatch here silently loosens the combine gate rather than failing: - deepep-v2 / uccl-ep normal: literally the same expression as semantic_payload. - deepep-v2 / uccl-ep low-latency: the LL branch called the EAGER helper (_to_fp8, per_token_cast_to_fp8) while semantic_payload calls self._quant -- equal because fused_quantize returns its argument unchanged in low-latency mode, so _quant IS the eager helper there. Same callable, not merely same values. - flashinfer-ep: normal only; same expression. - mori: same expression. So make_problem now sets dispatch_x = x and oracle_x = semantic_payload(x) unconditionally. Under BF16 semantic_payload is identity, so oracle_x is x and the oracle compares exactly what it compared before -- the sole consumer is getattr(problem, "oracle_x", problem.x) and nothing branches on hasattr, which is what makes always-setting it safe. The one behaviour _encode_dispatch had that semantic_payload lacks is the per-shape assert_quantize_identity in normal-mode FP8: the guard that the COMPILED quantizer is bit-identical to the eager one. That is preserved as _validate_quantizer, an explicit untimed hook, which is what it always was underneath. The "computes the oracle once, untimed, compiling this rung's shape" side effect survives too, because semantic_payload calls self._quant. Left alone deliberately: the "caller-prequantized" wording in the module docstrings, README and methodology. It was flagged as stale, but it describes the KERNEL contract -- MoRI and normal-mode deepep/uccl kernels do receive an already-quantized tensor -- which remains accurate now that the quantize sits in dispatch(). Rewriting only the adapter docstrings would have desynchronized them from the docs that say the same true thing. Tests: the base-identity test drops the tuple assertion, and a new one pins the contract that replaced it -- dispatch_x IS x, oracle_x comes from semantic_payload, and semantic_payload is called exactly once with x. 164 tests green.
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR with conflicts resolved