Skip to content

fix(sglang): attach the KVCR linker to the external state agent - #16827

Closed
karya0 wants to merge 1 commit into
ai-dynamo:codex/glm53-kvcr-hicachefrom
karya0:kvcr-guard-sglang
Closed

fix(sglang): attach the KVCR linker to the external state agent#16827
karya0 wants to merge 1 commit into
ai-dynamo:codex/glm53-kvcr-hicachefrom
karya0:kvcr-guard-sglang

Conversation

@karya0

@karya0 karya0 commented Oct 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Stacked on #16117 (base branch codex/glm53-kvcr-hicache). The first commit cherry-picks the lychee link fix from #16573 that the base branch has not picked up yet; the second commit is this PR.

An aggregated SGLang worker running the direct KVCR linker can opt in with --kvcr-state-agent-owner <CacheOwnerId>. The worker then registers one attachment with the external KV state agent after engine init and withdraws it on shutdown. The attachment carries the raw ZMQ event endpoint, the KV block size, and a router_hint_source naming the linker's source-control endpoint, so the state agent indexes the cache under a stable owner and the router can keep hinting at it after the worker is gone and the KVCR Guard holds the pool.

With the flag set, the worker registers kv_event_source_mode = "state_agent_v2" and skips the legacy KV event publisher; use_kv_events stays advertised. Without the flag nothing changes.

Validation rejects anything other than an aggregated linker worker with KV events, a ZMQ TCP event publisher on an empty topic, and remote linker hints enabled. kvcr_hint_sources is split out of publish_kvcr_hint_capabilities so the attachment and registration resolve the same endpoint.

Cancelling startup needs no Python-side cleanup: the native owner withdraws its own registrations when its waiter is dropped, as the vLLM attachment relies on already.

🤖 Generated with Claude Code

@copy-pr-bot

copy-pr-bot Bot commented Oct 19, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@karya0
karya0 temporarily deployed to external_collaborator October 19, 2026 21:04 — with GitHub Actions Inactive
@karya0
karya0 temporarily deployed to external_collaborator October 19, 2026 21:04 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Oct 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🌿 Fern Docs Preview: https://nvidia-preview-01a15b73-7112-724c-a730-0a573386e925.docs.buildwithfern.com/dynamo/dev

@github-actions github-actions Bot added external-contribution Pull request is from an external contributor backend::sglang Relates to the sglang backend labels Oct 19, 2026
@karya0
karya0 force-pushed the kvcr-guard-sglang branch from a8ef01c to 3dfb227 Compare October 20, 2026 21:13
@karya0
karya0 temporarily deployed to external_collaborator October 20, 2026 21:13 — with GitHub Actions Inactive
@dynamo-ops

dynamo-ops commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

/ok to test 3dfb227

@karya0 karya0 changed the title Attach TP1 SGLang KVCR workers to the external state agent fix(sglang): attach TP1 KVCR linker to the external state agent Oct 20, 2026
@github-actions github-actions Bot added the fix label Oct 20, 2026
@karya0
karya0 force-pushed the kvcr-guard-sglang branch from 3dfb227 to ec5db30 Compare October 20, 2026 21:32
@karya0 karya0 changed the title fix(sglang): attach TP1 KVCR linker to the external state agent fix(sglang): attach the KVCR linker to the external state agent Oct 20, 2026
@karya0
karya0 temporarily deployed to external_collaborator October 20, 2026 21:32 — with GitHub Actions Inactive
@dynamo-ops

dynamo-ops commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

/ok to test ec5db30

@karya0
karya0 force-pushed the kvcr-guard-sglang branch from ec5db30 to bd4799b Compare October 20, 2026 22:26
@karya0
karya0 temporarily deployed to external_collaborator October 20, 2026 22:26 — with GitHub Actions Inactive
@dynamo-ops

dynamo-ops commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

/ok to test bd4799b

@karya0
karya0 force-pushed the kvcr-guard-sglang branch from bd4799b to a310f33 Compare October 20, 2026 22:29
@karya0
karya0 temporarily deployed to external_collaborator October 20, 2026 22:29 — with GitHub Actions Inactive
@dynamo-ops

dynamo-ops commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

/ok to test a310f33

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 20, 2026
@devin-ai-integration

devin-ai-integration Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

✅ Dynamo PR CI passed — run 40448994647 (attempt 1) on a7f1bc54c3

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

Posted automatically by Devin for run 40448994647. Updated on every full-CI run of this PR.

This was referenced Oct 21, 2026
Closed
Closed
Draft
Merged
@ziqifan617 ziqifan617 mentioned this pull request Oct 21, 2026
Open
3 of 4 tasks
@karya0
fix(sglang): attach the KVCR linker to the external state agent
a7f1bc5 
Opt in with --kvcr-state-agent-owner on an aggregated direct-linker
worker: register one state-agent attachment after engine init, advertise
kv_event_source_mode=state_agent_v2, and skip the legacy KV event
publisher. Native registration rolls itself back when its waiter is
dropped, so startup cancellation needs no Python-side cleanup.

Signed-off-by: Kapil Arya <kapila@nvidia.com>
@karya0
karya0 force-pushed the kvcr-guard-sglang branch from a310f33 to a7f1bc5 Compare October 27, 2026 04:18
@karya0
karya0 temporarily deployed to external_collaborator October 27, 2026 04:18 — with GitHub Actions Inactive
@dynamo-ops

dynamo-ops commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

/ok to test a7f1bc5

@karya0
karya0 marked this pull request as ready for review October 27, 2026 04:23
@karya0
karya0 requested review from a team as code owners October 27, 2026 04:23
Copilot AI balanced review requested due to automatic review settings October 27, 2026 04:24
Copilot AI reviewed Oct 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@ziqifan617 ziqifan617 mentioned this pull request Oct 27, 2026
Merged
@ziqifan617
ziqifan617 deleted the branch ai-dynamo:codex/glm53-kvcr-hicache October 28, 2026 06:14
@ziqifan617 ziqifan617 closed this Oct 28, 2026
@callanjfox callanjfox mentioned this pull request Oct 28, 2026
Draft
@ziqifan617 ziqifan617 mentioned this pull request Oct 30, 2026
Merged

This branch was previously deployed

1 inactive deployment
external_collaborator — a7f1bc54 Deployed Oct 27, 2026 by karya0 via ok-to-test #39319
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

No one assigned

Labels

backend::sglang Relates to the sglang backend documentation Improvements or additions to documentation external-contribution Pull request is from an external contributor fix size/L

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@karya0 @dynamo-ops @ziqifan617