Skip to content

[Bug] Keep decision adaptations when the Builder deploys - #4535

Merged
wilsonwu merged 10 commits into
vllm-project:mainfrom
wilsonwu:fix-builder-deploy-bug
Oct 10, 2026
Merged

wilsonwu merged 10 commits into
vllm-project:mainfrom
wilsonwu:fix-builder-deploy-bug

Conversation

@wilsonwu

@wilsonwu wilsonwu commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator

Closes #4534

Purpose

Deploying from the Builder removed adaptations from every decision, even without edits. The Builder deploys DSL-compiled routing in replace mode. The DSL cannot express adaptations, and mergeDSLOwnedNodes replaced routing and recipes with the compiled nodes. Those decisions then fell back to mode: apply, so Router Learning started to act on routes that were set to bypass or observe it.

This change touches only the dashboard backend:

  • New preserveBaseDecisionAdaptations (deploy_policy_preservation.go):
    • Before the DSL-owned sections are replaced or merged, it copies each base decision's adaptations into the compiled decision with the same name.
    • Inside recipes it matches the recipe by name first, so identically named decisions in different recipes never share policy.
    • A value already in the compiled fragment wins.
  • mergeDSLOwnedNodes calls it in both deploy modes. Merge mode also replaces a supplied decisions or recipes list as a whole, so DSL-compiled fragments lost adaptations there too.
  • This is the rule dsl.MergeRoutingIntoBase already applies for sr-dsl compile --base.

Owner: Developer Experience & Ecosystem Workgroup (dashboard).

Test Plan

  • go test ./handlers -run 'TestMergeDeployPayload|TestDeploy' in dashboard/backend. New tests:
    • TestMergeDeployPayloadKeepsBaseDecisionAdaptations, in replace and merge mode:
      • top-level and recipe-scoped decisions keep their adaptations
      • a new decision and a new recipe do not inherit any
      • an explicit value in the fragment wins
    • TestMergeDeployPayloadReplaceKeepsMaintainedDecisionAdaptations:
      • runs the Builder's path (parse, decompile, compile, emit, replace deploy)
      • configs: config/config.yaml, the agent recipe, and both built-in mom-v1 configs
      • compares every decision's adaptations with the original
  • make dashboard-check
  • make check CHANGED_FILES="<the 3 changed files>"
  • Manual run without Docker: Go backend on :8700 and Vite on :3001.
    • Load the config, click Deploy without edits, check the Config Diff, click Deploy Now, and read the written config.yaml.
    • Configs: the default config, the agent recipe and the built-in mom-v1 recipe.
    • Each config was checked on main and on this branch.

Test Result

  • The new tests fail on main (all 4 maintained configs, and both modes of the unit test) and pass here. All deploy handler tests pass.

  • make dashboard-check and make check pass.

  • Manual, main → this fix:

    Scenario main this fix
    Default config, Deploy preview: decisions with adaptations 4 → 0 4 → 4
    agent recipe, Deploy Now: decisions with adaptations in the written config 2 → 0 2 → 2
    Built-in mom-v1 (recipe-scoped), Deploy Now: written config 16 → 0 16 → 16
    Changes in the Deploy preview (default / agent / mom-v1) 44 / 11 / 28 40 / 9 / 14
  • The remaining preview changes are other DSL round-trip differences, such as list order and fields the DSL does not carry. This PR does not change them.

  • With the default config, Deploy Now fails validation on main and here for a separate reason. The DSL round trip adds the static_business_route candidate iteration model to modelRefs, and its LoRA business-adapter is not declared in modelCards. A config merged by sr-dsl compile --base fails the same validation.

  • A renamed decision does not keep the old name's adaptations, as with sr-dsl compile --base.

  • I did not run the optional dashboard E2E profile locally because it needs a kind cluster. CI selects it.

Before fix:
image

After fix:
image


Semantic Router PR Checklist
  • PR title begins with exactly one bracketed category, such as [Feature], [Bug], [Docs], [Test], [Research], [Community], or [CI/Build]
  • The title does not stack prefixes such as [Router][Docs]; affected modules belong in labels and the PR body
  • The PR links an accepted issue with exactly one owner: one wg/* label for project work or owner/maintainers for repository governance
  • Commits in this PR are signed off with git commit -s
  • The Purpose, Test Plan, and Test Result sections reflect the actual scope, commands, and blockers for this change

See CONTRIBUTING.md for the full contributor workflow and commit guidance.

The Builder decompiles the config to DSL, compiles it back, and
deploys the compiled routing in replace mode. The DSL cannot express
decision adaptations, and mergeDSLOwnedNodes replaced routing and
recipes with the compiled nodes, so a Deploy without edits removed
every decision's adaptations: 4 decisions in the default config, 2
in the agent recipe and all 16 in the built-in mom-v1 recipe. Those
decisions then fall back to mode apply, and Router Learning acts on
routes that were set to bypass or observe it.

Before the DSL-owned sections are replaced or merged, copy each base
decision's adaptations into the compiled decision with the same name,
matching recipes by name first. A value already in the compiled
fragment wins. This is the rule dsl.MergeRoutingIntoBase applies when
sr-dsl compiles with --base.

Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
@github-actions github-actions Bot added the pr/blocked Blocked on a named decision, dependency, or required check. label Oct 4, 2026
@github-actions github-actions Bot added pr/needs-rebase Needs rebase or conflict resolution. wg/developer-experience-ecosystem Owned by the Developer Experience and Ecosystem Workgroup. and removed pr/blocked Blocked on a named decision, dependency, or required check. labels Oct 6, 2026
@github-actions github-actions Bot added pr/needs-review Ready for reviewer attention. pr/needs-rebase Needs rebase or conflict resolution. and removed pr/needs-rebase Needs rebase or conflict resolution. pr/needs-review Ready for reviewer attention. labels Oct 7, 2026
@github-actions github-actions Bot added pr/needs-review Ready for reviewer attention. pr/needs-rebase Needs rebase or conflict resolution. and removed pr/needs-rebase Needs rebase or conflict resolution. pr/needs-review Ready for reviewer attention. labels Oct 8, 2026
@github-actions github-actions Bot added pr/blocked Blocked on a named decision, dependency, or required check. and removed pr/needs-review Ready for reviewer attention. labels Oct 9, 2026

@Xunzhuo Xunzhuo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for preserving decision adaptations across Builder deploys. The new round-trip test includes an archived v0.4 config that the current parser rejects before checking adaptations; please use a compatible fixture and rerun the Dashboard suite.

Comment thread dashboard/backend/handlers/deploy_policy_preservation_test.go Outdated
Since vllm-project#4734 removed routing.data_policy from the schema, the current
parser rejects the immutable v0.4 mom-v1 snapshot before the test
reaches any adaptations check. Round-trip only the maintained configs:
config/config.yaml, the agent recipe and the latest built-in mom-v1,
which carries the same 16 decision adaptations as the snapshot.

Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
@wilsonwu
wilsonwu requested a review from Xunzhuo October 9, 2026 18:59
@github-actions github-actions Bot added pr/needs-author Waiting for author changes or response. and removed pr/blocked Blocked on a named decision, dependency, or required check. labels Oct 9, 2026

@Xunzhuo Xunzhuo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating the fixture. The maintained-config round trips and Dashboard suite now pass, including preserving decision adaptations across both deploy modes.

@mergify

mergify Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

@github-actions github-actions Bot added pr/needs-rebase Needs rebase or conflict resolution. and removed pr/needs-author Waiting for author changes or response. labels Oct 9, 2026
@mergify

mergify Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

🛑 Queue command has been cancelled

@github-actions github-actions Bot added pr/blocked Blocked on a named decision, dependency, or required check. pr/merge-ready Approved and passing required checks. pr/needs-rebase Needs rebase or conflict resolution. and removed pr/needs-rebase Needs rebase or conflict resolution. pr/blocked Blocked on a named decision, dependency, or required check. pr/merge-ready Approved and passing required checks. labels Oct 10, 2026
@mergify

mergify Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

🛑 Queue command has been cancelled

@mergify

mergify Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

🛑 Queue command has been cancelled

@mergify

mergify Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • ✅ Entered queue — 2026-10-10 12:55 UTC · Rule: default · triggered by @wilsonwu with the merge queue checkbox
  • 🚫 Left the queue — 2026-10-10 12:58 UTC · at a23f230b880574fb0a95a0272ee2b6fd7b74495b

This pull request spent 3 minutes 37 seconds in the queue, with no time running CI.

Reason

Pull request #4535 has been dequeued

Pull request from fork cannot be queued. This pull request comes from a fork, and Mergify needs the author's permission to update its branch.

The author needs to enable "Allow edits from maintainers" on this pull request.

Hint

You should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it.
If you do update this pull request, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio queue comment.

Tick the box to put this pull request back in the merge queue (same as @mergifyio queue).

  • Requeue this pull request

@github-actions github-actions Bot added pr/blocked Blocked on a named decision, dependency, or required check. and removed pr/needs-rebase Needs rebase or conflict resolution. labels Oct 10, 2026
@wilsonwu
wilsonwu merged commit d10d5ad into vllm-project:main Oct 10, 2026
31 checks passed
@github-actions github-actions Bot removed the pr/blocked Blocked on a named decision, dependency, or required check. label Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

wg/developer-experience-ecosystem Owned by the Developer Experience and Ecosystem Workgroup.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Builder Deploy without edits removes decision adaptations

2 participants