config: retire the last process-global config field reads - #33338
Merged
Merged
Conversation
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
ch-wan
requested review from
BBuf,
CatherineSue,
DarkSharpness,
Edwardf0t1,
Fridge003,
HaiShaw,
HydraQYH,
JustinTong0323,
Qiaolin-Yu,
Ying1123,
alphabetc1,
celve,
hanming-lu,
hebiao064,
hnyls2002,
hzh0425,
ispobock,
kpham-sgl,
merrymercy,
mickqian,
pyc96,
slin1237,
xiezhq-hermann,
yhyang201,
yizhang2077 and
yuan-luo
as code owners
August 3, 2026 04:21
ch-wan
force-pushed
the
cheng/gc-global-read-sweep
branch
3 times, most recently
from
August 3, 2026 04:24
c365544 to
a4c0885
Compare
`get_server_args().<field>` reads one process's startup record. Nine sites still did that for a value that has a namespace: the attention backend (5), `skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` / `get_model()` / `get_schedule()`, so they see the resolved value including post-publish overrides. The multimodal processor's device selection moves to the instance it was constructed with rather than to a namespace: `base_gpu_id` differs per worker (the encode-server DP workers each specialise their own copy), so no process-global value can stand in for it, and engines sharing a tokenizer process each need their own. Branch order, the NPU preprocess patches, and the case that leaves "device" unset are unchanged. What stays on `get_server_args()` is the derived API — `@property` and method members computed from several fields plus the HF config (`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) — plus three config-intent reads of live-shadowed sizes, each of which needs an answer the live topology property cannot give (the DSA indexer's PP gate must short-circuit before touching the PP group, `allocation`'s DCP gate asks whether DCP was configured at all, and the CUDA-IPC recycler runs where no group exists). A new AST ratchet pins both shapes it can see — the direct call and an alias bound from it in the same function — at 0 and 12 respectively, exempting the derived APIs and those three sites by name. The alias-form baseline is not zero: those reads are mostly per-runner fields in model code, and lowering them is the next slice. Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound `allocation.get_server_args` to a SimpleNamespace, which silently stops intercepting the moment a reader migrates; it publishes a real config instead.
ch-wan
force-pushed
the
cheng/gc-global-read-sweep
branch
from
August 3, 2026 04:24
a4c0885 to
38acc74
Compare
This was referenced Aug 3, 2026
kfhfar
pushed a commit
to kfhfar/sglang
that referenced
this pull request
Aug 7, 2026
…t#33338) `get_server_args().<field>` reads one process's startup record. Nine sites still did that for a value that has a namespace: the attention backend (5), `skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` / `get_model()` / `get_schedule()`, so they see the resolved value including post-publish overrides. The multimodal processor's device selection moves to the instance it was constructed with rather than to a namespace: `base_gpu_id` differs per worker (the encode-server DP workers each specialise their own copy), so no process-global value can stand in for it, and engines sharing a tokenizer process each need their own. Branch order, the NPU preprocess patches, and the case that leaves "device" unset are unchanged. What stays on `get_server_args()` is the derived API — `@property` and method members computed from several fields plus the HF config (`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) — plus three config-intent reads of live-shadowed sizes, each of which needs an answer the live topology property cannot give (the DSA indexer's PP gate must short-circuit before touching the PP group, `allocation`'s DCP gate asks whether DCP was configured at all, and the CUDA-IPC recycler runs where no group exists). A new AST ratchet pins both shapes it can see — the direct call and an alias bound from it in the same function — at 0 and 12 respectively, exempting the derived APIs and those three sites by name. The alias-form baseline is not zero: those reads are mostly per-runner fields in model code, and lowering them is the next slice. Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound `allocation.get_server_args` to a SimpleNamespace, which silently stops intercepting the moment a reader migrates; it publishes a real config instead.
saturn-acc
pushed a commit
to saturn-acc/sglang
that referenced
this pull request
Aug 16, 2026
…t#33338) `get_server_args().<field>` reads one process's startup record. Nine sites still did that for a value that has a namespace: the attention backend (5), `skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` / `get_model()` / `get_schedule()`, so they see the resolved value including post-publish overrides. The multimodal processor's device selection moves to the instance it was constructed with rather than to a namespace: `base_gpu_id` differs per worker (the encode-server DP workers each specialise their own copy), so no process-global value can stand in for it, and engines sharing a tokenizer process each need their own. Branch order, the NPU preprocess patches, and the case that leaves "device" unset are unchanged. What stays on `get_server_args()` is the derived API — `@property` and method members computed from several fields plus the HF config (`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) — plus three config-intent reads of live-shadowed sizes, each of which needs an answer the live topology property cannot give (the DSA indexer's PP gate must short-circuit before touching the PP group, `allocation`'s DCP gate asks whether DCP was configured at all, and the CUDA-IPC recycler runs where no group exists). A new AST ratchet pins both shapes it can see — the direct call and an alias bound from it in the same function — at 0 and 12 respectively, exempting the derived APIs and those three sites by name. The alias-form baseline is not zero: those reads are mostly per-runner fields in model code, and lowering them is the next slice. Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound `allocation.get_server_args` to a SimpleNamespace, which silently stops intercepting the moment a reader migrates; it publishes a real config instead.
jakki-amd
pushed a commit
to jakki-amd/sglang
that referenced
this pull request
Sep 9, 2026
…t#33338) `get_server_args().<field>` reads one process's startup record. Nine sites still did that for a value that has a namespace: the attention backend (5), `skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` / `get_model()` / `get_schedule()`, so they see the resolved value including post-publish overrides. The multimodal processor's device selection moves to the instance it was constructed with rather than to a namespace: `base_gpu_id` differs per worker (the encode-server DP workers each specialise their own copy), so no process-global value can stand in for it, and engines sharing a tokenizer process each need their own. Branch order, the NPU preprocess patches, and the case that leaves "device" unset are unchanged. What stays on `get_server_args()` is the derived API — `@property` and method members computed from several fields plus the HF config (`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) — plus three config-intent reads of live-shadowed sizes, each of which needs an answer the live topology property cannot give (the DSA indexer's PP gate must short-circuit before touching the PP group, `allocation`'s DCP gate asks whether DCP was configured at all, and the CUDA-IPC recycler runs where no group exists). A new AST ratchet pins both shapes it can see — the direct call and an alias bound from it in the same function — at 0 and 12 respectively, exempting the derived APIs and those three sites by name. The alias-form baseline is not zero: those reads are mostly per-runner fields in model code, and lowering them is the next slice. Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound `allocation.get_server_args` to a SimpleNamespace, which silently stops intercepting the moment a reader migrates; it publishes a real config instead.
Atituiset
pushed a commit
to Atituiset/sglang
that referenced
this pull request
Sep 10, 2026
…t#33338) `get_server_args().<field>` reads one process's startup record. Nine sites still did that for a value that has a namespace: the attention backend (5), `skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` / `get_model()` / `get_schedule()`, so they see the resolved value including post-publish overrides. The multimodal processor's device selection moves to the instance it was constructed with rather than to a namespace: `base_gpu_id` differs per worker (the encode-server DP workers each specialise their own copy), so no process-global value can stand in for it, and engines sharing a tokenizer process each need their own. Branch order, the NPU preprocess patches, and the case that leaves "device" unset are unchanged. What stays on `get_server_args()` is the derived API — `@property` and method members computed from several fields plus the HF config (`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) — plus three config-intent reads of live-shadowed sizes, each of which needs an answer the live topology property cannot give (the DSA indexer's PP gate must short-circuit before touching the PP group, `allocation`'s DCP gate asks whether DCP was configured at all, and the CUDA-IPC recycler runs where no group exists). A new AST ratchet pins both shapes it can see — the direct call and an alias bound from it in the same function — at 0 and 12 respectively, exempting the derived APIs and those three sites by name. The alias-form baseline is not zero: those reads are mostly per-runner fields in model code, and lowering them is the next slice. Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound `allocation.get_server_args` to a SimpleNamespace, which silently stops intercepting the moment a reader migrates; it publishes a real config instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
get_server_args().<field>reads the startup record of one process. Twenty-six ofthose reads were left in
srt; this removes every one that has somewhere better togo, and pins the rest.
Nine reads move to the namespace accessors — the value they want is the resolved
one, including post-publish overrides:
attention_backend×5 →get_exec().kernel,skip_tokenizer_init×2 →get_serving(), the draft-awareload_format→get_model(), and a PP size →get_parallel()(live topology, whichis what the DSA indexer wants).
Two reads move to the instance instead of a namespace. The multimodal processor
picked its fast-image-processor device from the process-global config, but
base_gpu_iddiffers per worker — the encode-server DP workers each specialise theirown copy — so no process-global value can stand in for it, and several
Engines canshare a tokenizer process. The decision moves into
BaseMultimodalProcessor._fast_image_processor_device, reading the instance theprocessor was constructed with, like the ten other config reads in that class. Branch
order, the NPU preprocess patches and the case that leaves
deviceunset areunchanged.
What stays is the derived API:
mamba_cache_chunk_size(@property),get_model_config(),enable_mamba_extra_buffer{,_lazy}()— computed from severalfields plus the HF config, so they are not namespace leaves and
ServerArgsis theironly home. Two config-intent reads of live-shadowed sizes also stay:
get_parallel()shadows
tp/pp/dcp/attn_cp/moe_dp_sizewith the live topology, and both sites runwhere no process group exists (
cuda_ipc_transport_utilseven guards for "notpublished yet").
A new AST ratchet pins field reads at zero and exempts the above by name, with the
reasoning in its docstring.
Test changes
test_dllm_fdfo_kv_reusereboundallocation.get_server_argsto aSimpleNamespace. That kind of stand-in stops intercepting the moment a readermigrates — which is exactly what happened here — so it publishes a real config now.
New
test_processor_device_selection.pypins the per-instance device resolutionagainst a conflicting published config.
Validation
Per-area unit suites (
mem_cache,multimodal,batch_overlap,layers/attention,managers) and all ratchets pass. Full registered CPU battery(16 partitions) against
main: seven branch-only failures, each re-run serially andgreen — two were port collisions from the parallel run itself, the rest are the
box's standing environment failures. No new failures.
Not covered locally: the flipped reads sit in model and attention paths that need a
GPU to execute; the flips are value-preserving (the bags are projected from the same
published config), but the speculative and model CI suites are the real check.
CI States
Latest PR Test (Base): 🚫 Run #30784352926
Latest PR Test (Extra): 🚫 Run #30784352743