Skip to content

config: six more runtime readers ask the bags - #36973

Merged
ch-wan merged 2 commits into
mainfrom
cheng/gc-r4c-bag-readers
Aug 29, 2026
Merged

config: six more runtime readers ask the bags#36973
ch-wan merged 2 commits into
mainfrom
cheng/gc-r4c-bag-readers

Conversation

@ch-wan

@ch-wan ch-wan commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

The series

Stacked, each PR based on the one above it. Read them in order; every boundary is
self-sufficient and green on its own.

  1. #36896 — the resolution pipeline's dispatcher leaves the record
  2. #36972 — the callbacks into the record go to zero
  3. #36973 — six more runtime readers ask the bags ← you are here
  4. #36974 — the dead record parameters go
  5. #36975 — the lazy imports that buy nothing become eager
  • #36925 — CI vehicle — runs the whole series against main, not for merge

Motivation

The previous PR took the resolution package off the record. This one is about the runtime:
code that runs after publish, holds a ServerArgs, and reads raw fields off it.

A census of srt/ finds 295 such reads in 53 files, every field of them in a published namespace.
Not all are convertible, and the criterion is not the field — it is the process. The launcher trio
(http_server, engine, data_parallel_controller) reads the record before its process
publishes; those 117 reads have to stay, and this PR does not touch them.

expert_distribution.py is the clearest case of the rest. ExpertDistributionRecorder was handed
a ServerArgs and threaded it through six constructors — recorder, gatherer, accumulator and
their subclasses — to reach four reads. Two of the classes stored it; one subclass took it and
never touched it. The file already asked get_exec().moe for the recorder mode, so the record was
travelling alongside a bag that already answered.

Modifications

The EPLB recorder. The four reads go to the bag and the parameter goes with them: five call
sites in model_runner.py, six signatures, two stored attributes.

eplb_min_rebalancing_utilization_threshold is an exec.moe leaf. The three balancedness
predicates were members of the record — should_log_expert_balancedness_to_server_log and its
Prometheus sibling, read from four places across three files, each through a record someone was
holding for that purpose. They become published accessors in runtime_context, the way
uses_mla_backend and cutedsl_moe_max_num_tokens already pair with their pre-publish
equivalents. Both members are gone from ServerArgs.

resolve_collector_class was the one thing keeping the record in the chain: it took a ServerArgs
to read stat_loggers, an observability leaf, and three of its six callers were reaching for
get_server_args() to pass it straight back. It asks the bag now, and server_args=None — which
meant "no config, use the default" — becomes "the namespace is not published", the same case in
the context's own terms.

Six more runtime readers, each already reading its other configuration from a bag:

file reads bag
hybrid_pool_assembler 6 × hicache_host_memory_mode memory
scheduler 3 × hicache_host_memory_mode memory
kt_ep_wrapper 7 × six kt_* leaves exec.moe
expert_backup_client nnodes, node_rank parallel
dspark_planner three speculative_dspark_* spec
prefill_delayer three prefill-delay leaves schedule

Three of them were carrying the record only for those reads, so it goes: PrefillDelayer and
ExpertBackupClient lose the constructor parameter and the stored attribute, and
build_sps_cost_table — whose body was already entirely get_spec() and get_schedule() — loses
its keyword-only server_args, which takes it out of DsparkPlanner too.

Removing a constructor parameter reaches further than the file that declares it. Three call sites
outside srt/ had to follow: the DSpark worker constructs DSparkVerifyPlanner (a P1 — every
DSPARK startup would have raised TypeError), and the registered prefill-delayer and SPS-table
tests both passed the keyword. The delayer test now publishes its per-case schedule leaves through
override_server_args instead of handing in a stand-in.

Dropping the argument leaves the local that fed it: base_prefix_cache.init_metrics_collector
still bound server_args = get_server_args() for a call that no longer takes one. Its two sibling
sites had already lost both.

The accessor docstrings say what a caller needs and no more: reports_expert_balancedness names
overrides.should_report_expert_balancedness as its pre-publish equivalent, the way
cutedsl_moe_max_num_tokens already did, and drops the sentence about why the accessors exist.

TestResolveCollectorClass drove resolve_collector_class with a _StubArgs stand-in. A stand-in
proves nothing about a function that reads a bag, so the cases force the leaf through
override_server_args and the stand-in is deleted.

Accuracy Tests

The 62-shape resolution probe is byte-identical to the base commit. The affected suites
(observability, mem_cache, managers, spec, server_args) report the same failure set as
main, file for file — 3038 passed on both sides, with the same three pre-existing failures.

Speed Tests and Profiling

None. A bag read is a dataclass attribute read.

Checklist

Review and Merge Process

Small and readable in full. The judgement call worth checking is which files were left alone: the
launcher trio publishes, so its reads are pre-publish by necessity, and roughly 150 reads across
thirty smaller files still need that per-file timing check before they can move.


CI States

Latest PR Test (Base): 🚫 Run #33249843844
Latest PR Test (Extra): 🚫 Run #33249843729
Latest PR Test (AMD ROCm 7.2): ❌ Run #33249843794

@ch-wan
ch-wan force-pushed the cheng/gc-r4c-bag-readers branch from 97f9b5b to 53692cd Compare August 29, 2026 06:35
@ch-wan
ch-wan force-pushed the cheng/gc-r4b-callbacks branch from e961cff to 5f745be Compare August 29, 2026 07:14
@ch-wan
ch-wan force-pushed the cheng/gc-r4c-bag-readers branch from 53692cd to 8320872 Compare August 29, 2026 07:14
@ch-wan
ch-wan force-pushed the cheng/gc-r4b-callbacks branch from 5f745be to 71dc8ec Compare August 29, 2026 07:37
@ch-wan
ch-wan force-pushed the cheng/gc-r4c-bag-readers branch from 8320872 to dacb593 Compare August 29, 2026 07:37
@ch-wan
ch-wan force-pushed the cheng/gc-r4b-callbacks branch from 71dc8ec to ee652b2 Compare August 29, 2026 08:11
@ch-wan
ch-wan force-pushed the cheng/gc-r4c-bag-readers branch from dacb593 to 9842ba2 Compare August 29, 2026 08:11
@ch-wan
ch-wan force-pushed the cheng/gc-r4b-callbacks branch from ee652b2 to fce9c77 Compare August 29, 2026 08:32
@ch-wan
ch-wan force-pushed the cheng/gc-r4c-bag-readers branch from 9842ba2 to e714b72 Compare August 29, 2026 08:32
@ch-wan
ch-wan force-pushed the cheng/gc-r4b-callbacks branch from fce9c77 to 93788d5 Compare August 29, 2026 08:50
@ch-wan
ch-wan force-pushed the cheng/gc-r4c-bag-readers branch from e714b72 to dd4bc95 Compare August 29, 2026 08:50
@ch-wan
ch-wan force-pushed the cheng/gc-r4b-callbacks branch from 93788d5 to 8c0c14f Compare August 29, 2026 11:17
Base automatically changed from cheng/gc-r4b-callbacks to main August 29, 2026 11:18
ch-wan and others added 2 commits August 29, 2026 11:18
`ExpertDistributionRecorder` was handed a `ServerArgs` and threaded it through
six constructors -- recorder, gatherer, accumulator and their subclasses -- to
reach four reads. Two of the classes stored it; one subclass took it and never
touched it. The file already asked `get_exec().moe` for the recorder mode, so
the record was travelling alongside a bag that already answered.

The four reads go to the bag, and the parameter goes with them: five call sites
in `model_runner.py`, six signatures, two stored attributes.

`eplb_min_rebalancing_utilization_threshold` is an `exec.moe` leaf, so it is a
direct bag read. The three balancedness predicates were members of the record --
`should_log_expert_balancedness_to_server_log` and its Prometheus sibling, read
from four places across three files, each through a record someone was holding
for that purpose. They become published accessors in `runtime_context`
(`reports_expert_balancedness`, `logs_...`, `exports_...`), the way
`uses_mla_backend` and `cutedsl_moe_max_num_tokens` already pair with their
pre-publish equivalents. Both members are gone from `ServerArgs`.

`resolve_collector_class` was the one thing keeping the record in the chain: it
took a `ServerArgs` to read `stat_loggers`, which is an `observability` leaf.
Three of its six callers were reaching for `get_server_args()` to pass it
straight back. It asks the bag now, and `server_args=None` -- which meant "no
config, use the default" -- becomes "the namespace is not published", which is
the same case said in the context's own terms.

`TestResolveCollectorClass` drove that with a `_StubArgs` stand-in. A stand-in
proves nothing about a function that reads a bag, so the cases force the leaf
through `override_server_args` and the stand-in is deleted.

The 62-shape resolution probe is byte-identical to the base commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The EPLB recorder was one instance of a shape. A census of the whole tree finds
295 raw-field reads off a held `ServerArgs` in 53 files -- every field of them
in a published namespace. Not all are convertible: the launcher trio
(`http_server`, `engine`, `data_parallel_controller`) reads the record *before*
its process publishes, and those reads have to stay.

These six do not. Each already reads its other configuration from a bag; the
raw reads are what was left behind:

  hybrid_pool_assembler   6  hicache_host_memory_mode      -> memory
  scheduler               3  hicache_host_memory_mode      -> memory
  kt_ep_wrapper           7  six kt_* leaves               -> exec.moe
  expert_backup_client    4  nnodes, node_rank             -> parallel
  dspark_planner          3  three speculative_dspark_*    -> spec
  prefill_delayer         3  three prefill-delay leaves    -> schedule

Three of them were carrying the record only for those reads, so it goes:
`PrefillDelayer` and `ExpertBackupClient` lose the constructor parameter and the
stored attribute, and `build_sps_cost_table` -- whose body was already entirely
`get_spec()` and `get_schedule()` -- loses its keyword-only `server_args`, which
takes it out of `DsparkPlanner` too. Two `ServerArgs` imports were left with no
user.

`kt_ep_wrapper` and `hybrid_pool_assembler` keep theirs: the first still calls
`get_hf_config()` on the record, the second is handed one for other reasons.

The 62-shape resolution probe is byte-identical to the base commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ch-wan
ch-wan force-pushed the cheng/gc-r4c-bag-readers branch from dd4bc95 to 22f94e0 Compare August 29, 2026 11:19
@ch-wan
ch-wan merged commit 1a3e152 into main Aug 29, 2026
18 of 22 checks passed
@ch-wan
ch-wan deleted the cheng/gc-r4c-bag-readers branch August 29, 2026 11:19
kediwu0331 pushed a commit to Zhylkaaa/sglang that referenced this pull request Aug 30, 2026
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
saturn-acc pushed a commit to saturn-acc/sglang that referenced this pull request Aug 31, 2026
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@PaddyXj PaddyXj mentioned this pull request Aug 31, 2026
9 tasks
nzr-niu pushed a commit to nzr-niu/sglang that referenced this pull request Sep 1, 2026
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

amd deepseek documentation Improvements or additions to documentation hicache Hierarchical Caching for SGLang model-gateway quant LLM Quantization sgl-kernel speculative-decoding unified-radix-cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant