Conversation
jthomson04
pushed a commit
to jthomson04/InferenceMAX
that referenced
this pull request
Jan 21, 2026
Revert gb200 changes on main
Oseltamivir
added a commit
that referenced
this pull request
Aug 7, 2026
…on a bad consume mode queries.md #13 and #9. #13: flashinfer-ep declares maturity "production" -- true of the backend, since vLLM exposes --all2all-backend flashinfer_nvlink_one_sided -- while its FP8 dispatch is a precision no engine can select (vLLM takes nvfp4/mxfp8/bf16 on that transport). The only machine-readable marker was an "-offpath" suffix buried inside the dispatch_dtype STRING, which nothing groups or filters on, and summarize.py prints no dispatch_dtype column at all. So in the view most people read, a transport-only measurement was indistinguishable from a deployable result. maturity cannot carry this: it is per-backend, and the whole point is that one backend is deployed at BF16 and off-path at FP8. So path_status is per-case, defaults to "deployed", and flashinfer-ep sets "offpath" under FP8. The harness emits it beside maturity and summarize marks the precision column ("fp8 [offpath]"). Not excluded from the matrix, which was the other option on the table: the row exists precisely to compare FP8 wire cost against deepep-v2/uccl-ep at matching bytes and block size, and dropping it would throw that away to fix a labelling problem. The "-offpath" dispatch_dtype suffix stays for durable-store row continuity and can retire at the next version bump. #9: CX_FP8_CONSUME accepted any string and silently meant "native", so a typo measured the opposite model from the one requested AND tagged the artifact with the typo. It now fails closed. Because the value is read at class-body evaluation this raises at import, before any measurement runs, which is the right moment for it. The rest of #9 -- relocating the dequant hatch to a separate diagnostic script -- is declined. The review conditioned that on the dequant cost being derivable, and it is: dequant roundtrip is roundtrip + stage to within +2.4%/-0.1%, verified in-tree. But the hatch is what re-verifies that bound after a kernel change (302.0 vs 302.5us in run 30177021271), a separate script would need the identical staged-inline branch, and BF16 requires that branch regardless. Moving it deletes the instrument, not the mechanism, and would make the derivability claim unfalsifiable. Also repaired here: a comment I orphaned two commits ago. Deleting combine_input_attr left its comment with no declaration beneath it, so it ran straight into the fp8_consume block and read as one paragraph. It is now marked as what it is -- a handle contract, not an attribute of the class. Tests: the unrecognised-mode guard (verified failing when removed), the deployed default, and two render tests -- an off-path row is marked, a deployed row and a row predating the field are not. 168 tests green.
Oseltamivir
added a commit
that referenced
this pull request
Aug 8, 2026
queries.md #13, second half. A per-case `path_status` field (8068c41) made FlashInfer's FP8 row identifiable, but it still shipped in every production sweep — so the default matrix spent GPU time measuring, and the dashboard displayed, a configuration no serving engine can select. Labelling it was necessary and not sufficient. `OFF_PATH_PRECISIONS` now lists such (backend, precision) pairs and the generator drops them unless the caller names the precision explicitly. Default sweeps carry deployable configurations only; `--precisions fp8` still brings the row back, because it exists for a real purpose — comparing the FP8 wire against deepep-v2/uccl-ep at matching bytes and block size — and deleting it outright would throw that away to fix a scheduling problem. Declared in sweep_matrix rather than read from the adapter's `path_status`, because this generator must resolve the matrix with no torch or vendor imports; a test holds the two in step. One consequence stated rather than buried: a precision filter is no longer purely subtractive. Naming an off-path precision ADDS rows relative to the default. The test that pinned the subtractive property now pins the exception explicitly instead of quietly comparing against a different baseline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added success rate calculation to full sweep workflows.
results_*.jsonrun_stats.jsonSuccessful sample run