Repository navigation
Support Faster JSON decoding for llava - #137
Merged
Merged
Conversation
5 tasks done
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
When sending fast-forwarded reqs to model_rpc, re-calculate `pad_input_ids`
NorthmanPKU
pushed a commit
to NorthmanPKU/sglang
that referenced
this pull request
May 16, 2025
* init * new logic * del * fix * layout * major issue * evaluation * add missing parts * merge gap * del * merge conflict * del * clean
Garrybest
pushed a commit
to Garrybest/sglang
that referenced
this pull request
Jan 9, 2026
…update_kv_cache.py. [perf] 16bs_1kin_1kout; TTFT(Mean/Median): 224.06/229.12; ITL(Mean/Median): 12.48/12.36; [GSM8K] 0.9113
rucnyz
added a commit
to rucnyz/sglang
that referenced
this pull request
Jun 2, 2026
Same rename pattern as session_id_passthrough (sgl-project#135) — proxy + event emission is daemon I/O infrastructure, not a numbered PLAN task. Changes: - git mv verify/t4 -> verify/daemon_proxy_events (preserves blame on the 1116 LoC verify.py + the historical results/ logs) - README banner updated: dropped pre-round-9 NOTE; documented current contract (4 request-path event kinds; in-process daemon-internal pause gate); explicit TODO links to T21 (sglang PUT /aginfer/program_paused) and T39 (proxy disconnect-awareness) for when those land - verify/README.md top-level index: row 5 updated from 't4 / T4 / legacy banner' to 'daemon_proxy_events / (infra) / post-T33 pass' Test run on Qwen3-0.6B-grade STUB sglang (no GPU) post-T33+T20: 12 steps + 4 recovery probes ALL PASS. Proxy overhead p50 = 1.49 ± 0.01 ms over 5 runs × 50 reqs (latency budget unchanged from round-1 audit baseline). Test scope unchanged; only the dir name + README header refreshed. Pending Tn that extend this surface — not yet implemented, so probes deferred: T21 (PLAN §3) — sglang-side PUT /aginfer/program_paused so per_program_usage[p].state is authoritative on sglang T39 (PLAN §4) — proxy gate awaits BOTH the pause condition AND request.is_disconnected(); release with HTTP 499 + transition ENDED on disconnect Webhook-driven event kinds (memory_pressure / PRESSURE_CRITICAL / APPLY_FAILED / HASH_COLLISION / SESSION_END) are covered by old t5 (webhook router); cleanup is sgl-project#137 next.
rucnyz
added a commit
to rucnyz/sglang
that referenced
this pull request
Jun 2, 2026
Same infra-rename pattern (sgl-project#135/sgl-project#136/sgl-project#137). No code change in the verify itself — t6 already had no schema-stale refs (the program tracker is a pure FSM; doesn't touch /aginfer/state). README: - Banner rewritten as 'infrastructure regression-guard' status. - OPEN WORK kept (still applies — T33 didn't add ENDED state) but now references T31 (SESSION_END EventKind) + T41 (mark_ended + EventBus.forget) by Tn instead of the deleted N3_GAPS catalog. verify/README.md row 4 updated. Test result: 10 steps PASS in 213 ms. No defensive-fallback / stale- schema findings in this refresh.
Chosen-David
pushed a commit
to Chosen-David/sglang
that referenced
this pull request
Oct 6, 2026
- EXP1a r1c_canonical_config.json：PSI/mavg 主表臂全参数机器可读锁定 （α0.125/β0.375/γ0.625、K1=128/K2=1024/cmp4、L1 全维128无量化、 L2 tail32=pair48..63 4bit、sink/swa 128+128），每项标注论文节号/实验代号/代码行； 预算推导逐 S（4K..128K）闭式表 + GPU1 原型代码 TLI_DEBUG 实跑对拍 6/6 全过 （γ=0.625 截断坍缩实锤：far 恒为保底 64 token，near 吃满 K2_mid=768） - EXP1b r1c_token_bytes_ledger.json：bytes 读写口径账本（与 kernel 时间口径 1.46×@32k→5.09×@128k 显式分开）——索引 overhead 占 dense 读 8.9%@4k → 3.5%@32k → 2.1%@128k（渐近 1.56%）；总量读 6.6%@32k / 2.8%@128k （KV 读缩减 15×@32k / 35×@128k）；索引存量 384B/token/层 = dense KV 的 9.4% （生产 tail32-L1 口径 352B，论文 336B 系 L1 bounds fp16 假设，差异已如实标注） Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WhiteCmile
added a commit
to WhiteCmile/sglang
that referenced
this pull request
Oct 31, 2026
…gl-project#137) CI fast tier flagged 4 new failures on the sgl-project#137 patch: - Hoisting _async_d2h to module level in dt_target broke the test seam that monkeypatches sglang.srt.managers.utils._async_d2h (d2h OOM injection and the pivot-copy scalar staging test). These call sites run once per wave, so keep the function-level import; the dt_protocol and msgspec hoists (the per-row/per-iteration ones) stay module-level. - test_direct_cancel_before_assignment_ack builds its arena via __new__ without __init__ and now must seed _scanned_slot_epoch itself, matching the other hand-set arena attributes. Full registered CPU suite: 177 tests, failures exactly the 10-entry known-failures allowlist.
WhiteCmile
added a commit
to WhiteCmile/sglang
that referenced
this pull request
Nov 1, 2026
Conflicts in dt_candidate_p2p.py and test_dt_candidate_completion_channel.py resolved in favour of dev (native record observer, sgl-project#122/sgl-project#137/sgl-project#150). Kept from sgl-project#25: dt_completion_source.py and the additive transport_endpoint / regions / capabilities fields in TargetCandidateArena.registration(). The arena does not route its record polling through CompletionSource for now; re-wiring it over the native observer is the first step of sgl-project#164. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.