fix issue: If request.stream is false, the code after "yield out" will not be executed - #504
Closed
ZX-ModelCloud wants to merge 1 commit into
Closed
ZX-ModelCloud wants to merge 1 commit into
ZX-ModelCloud wants to merge 1 commit into
Conversation
…l not be executed.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
…each with a falsifier B3 (draft-KV-DCP now keys on the installer's own predicate uneven_weighted_dcp_enabled(), admitting the --rank-kv-ratio route the sibling speculation x DCP gate already accepted -- the -67% draft-KV win is no longer unreachable for flag-route boots); B8 (validate_breakable_boot's None arm split into a NO_SERVER_ARGS sentinel that skips vs. an unresolvable backend that refuses by name); B10 (verdict: the kvso x speculation refusal is DELIBERATE, not stale -- reason and the one unobserved round now in the message, KVSO_ALLOW_SPEC surfaced in --enable-kv-session-offload's CLI help for the first time); B2 (--rank-kv-ratio without a placement refuses by name instead of being accepted-and-inert); B18 (the marlin uneven-TP coarsening fold reads a per-backend marlin_packable_linear declaration instead of a class-name list; of the three configs the audit named, only MarlinConfig was a genuine latent gap, W8A8Fp8Config/QuarkConfig reach no marlin repack entry point); planner/flags.py's two inverted registry edges (I-2/I-3) plus a new contract test that drives every declared uneven-TP edge against the live ServerArgs validation instead of trusting the registry by inspection. Adds a status header to AUDIT_500_mechanism_reach.md §5 recording the six fixes and two corrections-from-execution. Manually extended that header with the sgl-project#503 B1-refutation (the sgl-project#503 branch predates B1's discovery entirely, so it couldn't record it): combined board is now B1 REFUTED (sgl-project#503), B2/B3/B8/B10/B18 + the flags.py registry FIXED (sgl-project#504), B4-B7, B9, B11-B17, B19, B20 still OPEN -- one place to read the whole status rather than two disjoint headers. Clean auto-merge, no conflict: FEATURE_CATALOG.md's touched regions (near the preamble, and in §1/§3/§12-adjacent prose) were untouched by every merge since the ac84f5d base; server_args.py was untouched by sgl-project#503/sgl-project#505/sgl-project#506/ sgl-project#509 (its only other change on this line, sgl-project#493, is already inside the base). All 17 catalog sections and the four previously-preserved additive paragraphs (two-axis §1, matrix doctrine §2, MERGE-DUTY owner-sites + SITREP §12, NVENC §13) verified present; the sgl-project#505 §16 corrections (gpu-arb convention-not-enforcement, forward_peak off-by-default, CollectiveClock prefill-only) verified present and unaffected. Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree): - test_flag_registry_contract_500.py alone: 15 passed (exact target). - test/registered/unit/server_args/: 532 passed, 86 subtests (up from 526 pre-merge -- the new/extended test files account for the delta). - test/registered/unit/planner/ (HTSGLANG_TEST_MODEL_DIR set): 2223 passed, 1 skipped, 352 subtests -- crossing-check against sgl-project#503 (which also worked the planner edge): both merges coexist without collision, count is above the sgl-project#503-alone baseline of 2205 as expected from sgl-project#504's added tests. - test/registered/unit/layers/moe/: 153 passed, 395 subtests (includes the new test_breakable_boot_gate_500.py). - test/registered/unit/distributed/ (touched via test_marlin_unit_coarsening.py): 24 failed, 2134 passed, 8 skipped, 721 subtests. Diffed the failing-test list against the sgl-project#503-merge state line by line: BYTE-IDENTICAL set of 24 failures, only the passed/subtests counts grew (from marlin coarsening's new cases) -- confirmed non-regression, not just a count match. - ruff check --select=F401,F821,UP037: 2 findings (planner/flags.py:1717 `"Profile"`, test_rank_kv_ratio_args.py's unused `math` import), both confirmed pre-existing at the ac84f5d base and outside sgl-project#504's diff hunks. No ruff format run (CLAUDE.md: do not reformat planner/* and other listed subtrees -- unrelated churn buries the semantic diff). - codespell clean on all 18 touched/added files.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 14, 2026
…d the stub to the real call graph Register 87 reported them; this closes them. All four were AttributeError: 'types.SimpleNamespace' object has no attribute '_auto_mamba_demand_active' out of model_runner_kv_cache_mixin.py:1913. MECHANISM. handle_max_mamba_cache is a mixin method that unit suites drive directly against a SimpleNamespace stub -- a usage the production code acknowledges in its own getattr(self, "pp_size", 1) comment. The stub bound a HAND-PICKED two of the nine mixin methods in that function's transitive call closure, so any new self._helper() on a reachable branch breaks it. ORIGIN, three landings inside one blind window: 2026-06-01 d8a5a25 (upstream sgl-project#25173, NIXL refactor) made the directory uncollectable -- THE GUARD GOES DOWN 2026-07-15 a0ed7dc "[DCP] Auto-size the mamba state pool by demand under uneven DCP": +132 mixin lines, NO test file touched, added `elif self._auto_mamba_demand_active():` -- break sgl-project#1 2026-07-22 8b48e32 "T156 stage 3": added server_args.max_speculative_num_draft_tokens at 1910/1922 -- break sgl-project#2, masked behind sgl-project#1 The ordering is the finding: the collection block landed SIX WEEKS BEFORE the first defect, so the guard was already down when both arrived. A third instance surfaced only once sgl-project#1 was removed -- the demand path dies on _mamba_pool_budget_cost_gb, which handle_max_mamba_cache never names (_fit_mamba_pool_to_budget calls it). Binding one level deep re-arms the trap one frame lower, so the fix takes the TRANSITIVE closure. NOT A LIVE SERVING DEFECT. ModelRunner INHERITS ModelRunnerKVCacheMixin (model_runner.py:426), so every real self has every one of these methods; the sole production caller is self.handle_max_mamba_cache(rest_memory) inside the same mixin (:725); and grep MethodType over python/sglang/srt/model_executor/ returns nothing, so nothing binds these onto a foreign object. Unreachable from a boot. ZERO PRODUCTION LINES CHANGED -- which is also why behaviour on the path where the methods ARE present is unchanged by construction rather than by test. FIXED FOR THE RIGHT REASON. The tempting repair -- getattr(self, "_auto_mamba_demand_active", lambda: False)() in production -- would silently pick the fixed-fraction branch on a real runner whose method had genuinely gone missing, turning a loud AttributeError into a wrong pool size. Instead: the gate is BOUND FROM THE REAL CLASS so the stub takes production branches, the declared list is checked against the AST of the real call graph (sgl-project#504-a one-source-of-truth), and both directions are pinned. TEST RESULTS (venv /spinning/htsglang-gpu/.venv, CUDA_VISIBLE_DEVICES=99, PYTHONPATH=<worktree>/python, pytest --color=no): test_mamba_checkpoint_interval.py 19 failed, 15 passed -> 15 failed, 25 passed 4 fixed (the AttributeError set: test_large_budget_keeps_requested_size, test_small_budget_caps_size_and_keeps_rest_positive, test_no_spec_dec_path, test_zero_budget_raises_actionable_error) + 6 new. All 15 remaining are "No accelerator ... is available". test/registered/unit/mem_cache/ 944F 738P 707S -> 940F 748P 707S ZERO remaining CPU-visible reds in the directory: every one of the 940 is accelerator-gated. BOTH DIRECTIONS PINNED (TestTheDemandGateIsConsultedBothWays): gate False - stock stub keeps the fixed-fraction path, and the four original tests' expectations are unchanged, so binding the method did not move them to the demand branch gate True - uneven vector [30,17,17] + radix on takes the demand path uniform - [1,1,1] is NOT uneven (even modulo fast path), stays False explicit - max_mamba_cache_size wins above the elif and never reaches it CAN-FAIL PROOF, executed: dropping "_mamba_pool_budget_cost_gb" from the declared list reproduces the original defect shape exactly -- FAILED ...TestTheStubTracksTheFunctionItDrives:: test_the_declared_list_matches_the_functions_real_self_calls FAILED ...test_an_uneven_vector_with_radix_on_takes_the_demand_path naming the missing method. Restored: 6 passed. CARRIED, named not hidden: the file is register_cuda_ci(stage="base-b"), so on a CPU-only CI arm these CPU-safe tests still would not run. Splitting the file by stage is not done here. Register entry 89.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 14, 2026
…d the stub to the real call graph Register 87 reported them; this closes them. All four were AttributeError: 'types.SimpleNamespace' object has no attribute '_auto_mamba_demand_active' out of model_runner_kv_cache_mixin.py:1913. MECHANISM. handle_max_mamba_cache is a mixin method that unit suites drive directly against a SimpleNamespace stub -- a usage the production code acknowledges in its own getattr(self, "pp_size", 1) comment. The stub bound a HAND-PICKED two of the nine mixin methods in that function's transitive call closure, so any new self._helper() on a reachable branch breaks it. ORIGIN, three landings inside one blind window: 2026-06-01 d8a5a25 (upstream sgl-project#25173, NIXL refactor) made the directory uncollectable -- THE GUARD GOES DOWN 2026-07-15 a0ed7dc "[DCP] Auto-size the mamba state pool by demand under uneven DCP": +132 mixin lines, NO test file touched, added `elif self._auto_mamba_demand_active():` -- break sgl-project#1 2026-07-22 8b48e32 "T156 stage 3": added server_args.max_speculative_num_draft_tokens at 1910/1922 -- break sgl-project#2, masked behind sgl-project#1 The ordering is the finding: the collection block landed SIX WEEKS BEFORE the first defect, so the guard was already down when both arrived. A third instance surfaced only once sgl-project#1 was removed -- the demand path dies on _mamba_pool_budget_cost_gb, which handle_max_mamba_cache never names (_fit_mamba_pool_to_budget calls it). Binding one level deep re-arms the trap one frame lower, so the fix takes the TRANSITIVE closure. NOT A LIVE SERVING DEFECT. ModelRunner INHERITS ModelRunnerKVCacheMixin (model_runner.py:426), so every real self has every one of these methods; the sole production caller is self.handle_max_mamba_cache(rest_memory) inside the same mixin (:725); and grep MethodType over python/sglang/srt/model_executor/ returns nothing, so nothing binds these onto a foreign object. Unreachable from a boot. ZERO PRODUCTION LINES CHANGED -- which is also why behaviour on the path where the methods ARE present is unchanged by construction rather than by test. FIXED FOR THE RIGHT REASON. The tempting repair -- getattr(self, "_auto_mamba_demand_active", lambda: False)() in production -- would silently pick the fixed-fraction branch on a real runner whose method had genuinely gone missing, turning a loud AttributeError into a wrong pool size. Instead: the gate is BOUND FROM THE REAL CLASS so the stub takes production branches, the declared list is checked against the AST of the real call graph (sgl-project#504-a one-source-of-truth), and both directions are pinned. TEST RESULTS (venv /spinning/htsglang-gpu/.venv, CUDA_VISIBLE_DEVICES=99, PYTHONPATH=<worktree>/python, pytest --color=no): test_mamba_checkpoint_interval.py 19 failed, 15 passed -> 15 failed, 25 passed 4 fixed (the AttributeError set: test_large_budget_keeps_requested_size, test_small_budget_caps_size_and_keeps_rest_positive, test_no_spec_dec_path, test_zero_budget_raises_actionable_error) + 6 new. All 15 remaining are "No accelerator ... is available". test/registered/unit/mem_cache/ 944F 738P 707S -> 940F 748P 707S ZERO remaining CPU-visible reds in the directory: every one of the 940 is accelerator-gated. BOTH DIRECTIONS PINNED (TestTheDemandGateIsConsultedBothWays): gate False - stock stub keeps the fixed-fraction path, and the four original tests' expectations are unchanged, so binding the method did not move them to the demand branch gate True - uneven vector [30,17,17] + radix on takes the demand path uniform - [1,1,1] is NOT uneven (even modulo fast path), stays False explicit - max_mamba_cache_size wins above the elif and never reaches it CAN-FAIL PROOF, executed: dropping "_mamba_pool_budget_cost_gb" from the declared list reproduces the original defect shape exactly -- FAILED ...TestTheStubTracksTheFunctionItDrives:: test_the_declared_list_matches_the_functions_real_self_calls FAILED ...test_an_uneven_vector_with_radix_on_takes_the_demand_path naming the missing method. Restored: 6 passed. CARRIED, named not hidden: the file is register_cuda_ci(stage="base-b"), so on a CPU-only CI arm these CPU-safe tests still would not run. Splitting the file by stage is not done here. Register entry 89.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
If request.stream is false, the code after "yield out" will not be executed.
Because return statement in generate_request() will interrupt the asynchronous generator.
![Uploading Screenshot_20240605_102103.png…]()