Improve error handling & abort disconnected requests - #449
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
HanHan009527
pushed a commit
to HanHan009527/sglang
that referenced
this pull request
Apr 7, 2026
Co-authored-by: zhuxingxin.0 <zhuxingxin.0@bytedance.com>
EdwardXuy
pushed a commit
to shun8686/sglang
that referenced
this pull request
Apr 29, 2026
Update run_aisbench.sh
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 2, 2026
…fill indexer (sgl-project#449) MiB-budgeted query-row loop over the existing KV-page loop (sgl-project#395 discipline, sgl-project#426 composition preserved); B-fold KV-gather duplication bounded from factor B to the budgeted row count; bit-identical (atol=0) against the unchunked path and the untouched single-pass reference. Collective audit clean under uneven-TP rank divergence. GPU measurement (peak VRAM per rank primary) pending — NOTE_449 §5. Full dedup of the B-fold gather stays open as ANALYSE_447 candidate C.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
…ually binds sgl-project#449 shipped the per-query-row chunking budget at 2048 MiB, above the peak it was meant to bound on this fork's serving geometry (--chunked-prefill-size 256: 903 rows fit the old cap against 256 asked for, so the cap discarded the whole query axis). Lowers the default to 256 MiB (the largest power of two that binds at both SEQ_CHUNK settings), makes the transient nameable at boot (dsv4_indexer_prefill_scratch_mib) and itemized in the pinned-reserve shortfall note. GPU arm packaged under scripts/dev/493_indexer_transient/, not run (falsifier: peak_bytes_max must fall ~326 MiB/rank between arms). FEATURE_CATALOG.md §15/§16 additions merged clean (no overlap with the sgl-project#500 audit merge's §1/§2 edits — adjacent but disjoint hunks). Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree): - test/registered/unit/layers/attention/: 152 passed, 9 skipped, 480 subtests passed (matches pre-merge expectation). - test/registered/unit/server_args/ + migration/cli-metadata/ratchet: 526 passed. test_server_args_mutation_ratchet fails (count 10 > baseline 0), but the offending lines are all in topology.py, dual_group_lane.py, cross_algo_utils.py and uneven_perf.py -- none of which this branch touches, and the same count (10) reproduces on 9fab358 before this merge. Pre-existing, not a regression from sgl-project#493. - ruff check --select=F401,F821,UP037 and codespell clean on all touched/added files.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
…ilent-wrongness sweep Adds docs/dev/AUDIT_505_silent_wrongness.md (1692 lines, 10 commits): axis A1 (warning-instead-of-error in loader/spec/MoE), A2 (warning-instead-of-error in memory/dist/pool), B (comment invariants as testable claims), C (numeric defaults without a binds-proof, the sgl-project#449/sgl-project#493 class generalised), D (catalog §14/§16 against their code predicates). Cross-cutting: the load-completeness check binds in only 2 of 186 model loaders; reconciles with the sgl-project#501 audit's A1-01 finding (two completeness remedies, neither general). Corrects FEATURE_CATALOG.md §14 (Dashboard) and §16 (Measurement/window infrastructure) with file:line-cited predicates in place of the old short prose: wizard refusals cite their source; the anonymization gate covers the rig-artifact share route only, NOT the sgl-project#152 result-share route, which renders argv verbatim (sgl-project#505-D3); energy metering is GPU board power only, not wall-socket (no knee-point probe actually exists, sgl-project#505-D6); self-update switching needs --serve-supervised (sgl-project#505-D8); gpu-arb is a CONVENTION, not an enforcement (sgl-project#438); forward_peak.py is off unless SGLANG_FORWARD_PEAK_PATH is set, and that env var has no environ.py entry (sgl-project#505-D11); CollectiveClock covers plain-prefill-only, cuda-only, pp_size==1-only -- no decode/verify-round equivalent (sgl-project#505-D14); the measured-KV-budget stale-boot trap only applies when the feature flag is on. Real conflict as briefed: HEAD's sgl-project#493 merge had appended a "transient-rate sampler" paragraph to the end of the OLD §16 prose; sgl-project#505's branch rewrote that same prose from base, sentence by sentence, with citations. Resolved by taking sgl-project#505's full rewrite (precision wins) and re-appending the sgl-project#493 paragraph after it unchanged, plus a one-line cross-reference noting forward_peak needs SGLANG_FORWARD_PEAK_PATH set before use (ties sgl-project#493's "use forward_peak's nvml_free_bytes_min" advice to sgl-project#505-D11's off-by-default finding). §14 conflict-free (git auto-merged; HEAD never touched that section). All 17 catalog sections and the four previously-preserved additive paragraphs (two-axis §1, matrix doctrine §2, MERGE-DUTY owner-sites + SITREP §12, NVENC §13) verified still present. Verified: 1f2b24e (sgl-project#501 fix commit) is confirmed an ancestor of the current line tip via the earlier fix/kvso-reclaim-decline-501 merge -- the sgl-project#505 audit's own warning that sgl-project#501 was absent from ITS base does not apply to this integration line. git diff --stat d653405..HEAD -- '*.py' empty (pure docs). codespell clean on both touched files. Scrubbed for personal email/PAT/private-IP leakage -- no hits in either audit document.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI States
Latest PR Test (Base): ❌ Missing
run-cilabel -- add it to run CI tests.Latest PR Test (Extra): ❌ Blocked --
run-ciis required first.