Skip to content

config: publishing is the process entry's job - #36251

Merged
ch-wan merged 3 commits into
mainfrom
cheng/gc-p2-publish-at-entry
Aug 26, 2026
Merged

config: publishing is the process entry's job#36251
ch-wan merged 3 commits into
mainfrom
cheng/gc-p2-publish-at-entry

Conversation

@ch-wan

@ch-wan ch-wan commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Publishing the config bags was happening inside constructors — TokenizerManager,
ModelRunner, the encoder server. That makes "is the config published?" depend on which
object a process happened to build first, and it means a process that legitimately builds
none of them (a spawned encoder worker, a benchmark entry) reads unpublished bags. It also
puts a process-wide side effect in an object's __init__, so a test that builds two of
them publishes twice.

Separately, whole-object readbacks (/server_info, the resolved-args dict) walked the
record's fields. Once a decision lives in the declaration stash rather than in the field,
walking fields reports the wrong shape.

What changes

  • publish(server_args, role=...) is called by the process entry: the HTTP entry, the
    encoder gRPC entry, the encoder DP and TP workers after spawn, and the one-batch
    benchmark entry.
  • Constructors call assert_published(server_args, role=...) instead of publishing. It
    fails closed with the role in the message, so an entry that forgot to publish says so at
    startup instead of returning a stale or empty namespace.
  • Whole-object readbacks go through resolution_projection(server_args) /
    ServerArgs.resolved_dict(), which report the declaration result for every leaf,
    expanding nested dataclasses and namedtuples the way asdict does.
  • test_publish_precedes_bag_reads.py walks each known entry and asserts publishing
    precedes the first bag read.

Nothing here changes what any field holds: resolution still writes as it declares, and the
end-of-resolution replay is untouched. Both go away together in the last PR of the stack.

How to verify

export PYTHONPATH=$PWD/python
python -m pytest -q test/registered/unit/test_publish_precedes_bag_reads.py
python -m pytest -q test/registered/unit/test_runtime_context.py
python -m pytest -q test/registered/unit/server_args/test_resolution_declarations.py

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ❌ Run #32965569690
Latest PR Test (Extra): ✅ Run #32966000247
Latest PR Test (AMD ROCm 7.2): ❌ Run #32965569687

@ch-wan
ch-wan force-pushed the cheng/gc-p2-publish-at-entry branch from 945cacf to 4083266 Compare August 25, 2026 03:47
@ch-wan
ch-wan force-pushed the cheng/gc-p1-parallel-tier branch from bd01f13 to 1303cea Compare August 26, 2026 06:48
@ch-wan
ch-wan force-pushed the cheng/gc-p2-publish-at-entry branch from 4083266 to d49cb9d Compare August 26, 2026 06:48
@ch-wan
ch-wan force-pushed the cheng/gc-p1-parallel-tier branch from 1303cea to 18e7fd8 Compare August 26, 2026 07:22
@ch-wan
ch-wan force-pushed the cheng/gc-p2-publish-at-entry branch from d49cb9d to b1d2a8b Compare August 26, 2026 07:22
@ch-wan
ch-wan force-pushed the cheng/gc-p1-parallel-tier branch from 18e7fd8 to 27cde8d Compare August 26, 2026 08:12
@ch-wan
ch-wan force-pushed the cheng/gc-p2-publish-at-entry branch 2 times, most recently from 048d3b8 to 8431d22 Compare August 26, 2026 08:21
@ch-wan
ch-wan force-pushed the cheng/gc-p1-parallel-tier branch from 27cde8d to 06caed3 Compare August 26, 2026 09:37
@ch-wan
ch-wan force-pushed the cheng/gc-p2-publish-at-entry branch from 8431d22 to 62d6018 Compare August 26, 2026 09:37
Base automatically changed from cheng/gc-p1-parallel-tier to main August 26, 2026 10:00
ch-wan added 3 commits August 26, 2026 11:51
`/server_info`, its gRPC and in-process twins, and the scheduler's
internal-state dump all handed out the record itself: three via
`dataclasses.asdict(server_args)` and one via `dict(vars(server_args))`. Both
read the fields, which carry resolution's result only for as long as
declarations materialize onto the record -- and the point of declaring is that
they will stop. Left alone, these endpoints would quietly start reporting what
the operator typed instead of what resolution decided.

`ServerArgs.resolved_dict()` is the whole-object shape of `resolution_result`:
every field, read through the declarations, nested dataclasses expanded the way
`asdict` expands them. The four exits report it. Values are unchanged today --
15 launch shapes agree field for field across all 476 -- so this is the
placement, not a new answer.

The `vars()` base was also leaking: the resolution bookkeeping (`_raw_input`,
the declaration stash, the materialization marker) and the `ModelConfig` memo
crossed IPC into `/server_info`'s `internal_states` block. The projection is
fields only, and a test pins that the dump is exactly the fields.
Three constructors published defensively because each could be built with
nothing published before it: `ModelRunner`, `TokenizerManager`, `MMEncoder`.
That made "are the config bags available here?" a question about which
constructor happened to have run, which is the wrong place for it: code between
the process entry and that constructor cannot read a bag, and a `publish` that
lands late re-projects the bags and silently drops any `override()` taken in
between.

The six entries that relied on it now publish for themselves:
`init_multi_tokenizer` (the multi-tokenizer worker reads its record from shared
memory), the two `benchmark/one_batch` work functions (run inline for
tp_size == 1 and spawned per rank otherwise), the encoder's gRPC entry, and its
spawned TP and DP workers. `publish` resolves through the idempotent gate, so a
spawned child that receives a resolved record is unaffected.

No behavior change: the constructors' `ensure_published` becomes a no-op at
each of these, and an override taken between an entry and its constructor now
survives instead of being discarded. `test_publish_precedes_bag_reads` pins all
six as publishing entries, so the walk checks each one's publish against the
bag reads it reaches.
…t publish

`ModelRunner`, `TokenizerManager` and `MMEncoder` published defensively
through `ensure_published`. With every process entry publishing for itself, a
constructor arriving unpublished no longer means "this is a standalone build" --
it means an entry was missed, and publishing here would work by accident while
re-projecting the bags over whatever the process had.

`assert_published` replaces it: same identity-and-role check, and it raises with
what is published instead, naming the entry as the place to publish. The draft
runner is unchanged (it deliberately does not publish, so it does not check
either).

The four manual tests that built one of these objects directly now publish in
their setup, which is what they are: the process entry for that test. The two
constructor entries leave `_KNOWN_ENTRIES`, and the constructor-publisher census
is down to the two that really are entries -- `Engine` and `SchedulerActor`.
@ch-wan
ch-wan force-pushed the cheng/gc-p2-publish-at-entry branch from 62d6018 to de39aea Compare August 26, 2026 11:52
@ch-wan ch-wan added the ready-to-merge The PR is ready to merge after the CI is green. label Aug 26, 2026
@ch-wan
ch-wan merged commit d7b144f into main Aug 26, 2026
128 of 145 checks passed
@ch-wan
ch-wan deleted the cheng/gc-p2-publish-at-entry branch August 26, 2026 11:58
wangfakang added a commit to wangfakang/sglang that referenced this pull request Aug 27, 2026
Resolve conflicts from the config-bag refactor (upstream sgl-project#36251-sgl-project#36255):
- kimi_k3_hook.py: keep disable_kimi_k3_symm_mem removed (revert intent).
- server_args.py _handle_cuda_graph_config: drop the disable_kimi_k3_symm_mem
  import/call (revert intent), keep upstream's cfg = resolving_view(self) the
  merged warning block at the end needs.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: wangfakang <fakangwang@gmail.com>
saturn-acc pushed a commit to saturn-acc/sglang that referenced this pull request Aug 31, 2026
nzr-niu pushed a commit to nzr-niu/sglang that referenced this pull request Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Multi-modal multi-modal language model ready-to-merge The PR is ready to merge after the CI is green.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant