Skip to content

fix(config): retain pre-engine resolution declarations - #37195

Merged
ishandhanani merged 3 commits into
mainfrom
idhanani/sgl-36255-late-resolution
Aug 31, 2026
Merged

fix(config): retain pre-engine resolution declarations#37195
ishandhanani merged 3 commits into
mainfrom
idhanani/sgl-36255-late-resolution

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

SGLang #36255 made ServerArgs preserve raw operator input, so Dynamo began declaring its pre-engine settings through declare_late_resolution(); the first engine resolution pass then cleared that declaration stash, silently dropping Dynamo's snapshot, GPU-memory-service, and forward-pass-metrics settings. This change preserves those declarations as inputs to resolution, allowing downstream launchers to retain their intended effective configuration while raw ServerArgs values remain unchanged.

How This Was Implemented

  • Keep pre-existing _resolved_overrides entries after snapshotting raw input, rather than replacing the stash at the start of run_resolution_pipeline().
  • Preserve ordering: launcher declarations are retained first, and pipeline declarations append afterward with the existing last-writer-wins behavior.
  • Add a regression that declares enable_forward_pass_metrics before resolve_once() and verifies the resolved projection observes it while the raw field does not change.
Walkthrough

Mental model

ServerArgs now has two intentionally separate states: raw operator input on the record and effective values in the declaration stash. A launcher that needs to influence the engine before it starts calls declare_late_resolution(); resolution must treat that declaration exactly like the declarations it derives internally.

sequenceDiagram
    participant D as Dynamo launcher
    participant SA as SGLang ServerArgs
    participant E as SGLang Engine resolution
    participant R as Resolved projection
    D->>SA: declare_late_resolution(FPM, snapshot, GMS)
    E->>SA: snapshot raw input
    E->>SA: retain launcher declarations
    E->>SA: append engine declarations
    R->>SA: project effective configuration
Loading

Downstream failure and ordering

The raw-input change exposed two separate Dynamo integration assumptions. First, Dynamo passed raw page_size=None to its KV publisher and hit a TypeError; Dynamo #13905 fixes that by reading SGLang's post-engine resolved view. During that same integration audit, Dynamo moved its own pre-engine settings to SGLang's declaration channel — but this pipeline reset discarded the entries before the engine projected them, leaving the engine to start without those settings.

This patch fixes the second failure at its source. It copies the existing stash before adding declarations derived during resolution, so a launcher's pre-engine decision remains visible to resolution_result() and the published config; declarations made later in the pipeline still win as before.

Boundaries

This does not materialize defaults back into raw fields, change post-publish mutation rules, or alter Dynamo's resolved-view handoff for page_size. declare_late_resolution() remains pre-publish only; post-publish configuration changes still use the runtime configuration bags.

Validation

  • python -m pytest -q test/registered/unit/server_args/test_resolution_declarations.py17 passed.
  • pre-commit run --files python/sglang/srt/arg_groups/pipeline.py test/registered/unit/server_args/test_resolution_declarations.py.

Summary

  • retain launcher declarations when the first resolution pass snapshots raw ServerArgs input
  • cover the pre-engine declaration path through the resolved projection

Validation

  • python -m pytest -q test/registered/unit/server_args/test_resolution_declarations.py
  • pre-commit run --files python/sglang/srt/arg_groups/pipeline.py test/registered/unit/server_args/test_resolution_declarations.py

CI States

Latest PR Test (Base): 🚫 Run #33425671954
Latest PR Test (Extra): ❌ Run #33425671224
Latest PR Test (AMD ROCm 7.2): 🚫 Run #33425672198

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@ishandhanani
ishandhanani marked this pull request as ready for review August 31, 2026 18:33
@ishandhanani

Copy link
Copy Markdown
Collaborator Author

relevant a-stage tests have passed

@ishandhanani
ishandhanani merged commit cf51650 into main Aug 31, 2026
80 of 105 checks passed
@ishandhanani
ishandhanani deleted the idhanani/sgl-36255-late-resolution branch August 31, 2026 19:24
nzr-niu pushed a commit to nzr-niu/sglang that referenced this pull request Sep 1, 2026
…7195)

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
StevenChenSE pushed a commit to StevenChenSE/sglang that referenced this pull request Sep 6, 2026
…7195)

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant