Skip to content

docs(sglang): correct session radix availability - #11241

Merged
ishandhanani merged 2 commits into
ai-dynamo:mainfrom
ishandhanani:idhanani/dyn-3400-session-radix-wiring
Jul 6, 2026
Merged

docs(sglang): correct session radix availability#11241
ishandhanani merged 2 commits into
ai-dynamo:mainfrom
ishandhanani:idhanani/dyn-3400-session-radix-wiring

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Correct the SGLang agent documentation and launcher commentary to match the capabilities shipped in SGLang 0.5.14. This prevents users from expecting passive session-radix behavior that the release cannot provide.

CLOSES: DYN-3400

How This Was Implemented

  • Replaced session-radix availability claims with the shipped session-header behavior.
  • Clarified that SGLang 0.5.14 session_params requires an explicitly opened session and is not a passive KV ownership tag.
  • Kept priority-based KV eviction guidance while removing session-tagging claims from launchers, cross-links, and the agentic inference digest.
Walkthrough

Mental model

SGLang 0.5.14 accepts session_params, but its scheduler routes those requests through the explicit session controller and rejects unknown IDs. The passive session-radix implementation and first-class session identity landed separately from the v0.5.14 release tag, so Dynamo's disabled compatibility gate must not be described as an active feature.

The updated guidance distinguishes three behaviors: session headers remain available for tracing and router affinity, --radix-eviction-policy priority controls eviction ordering only, and session_params belongs to SGLang's explicit session lifecycle.

Boundaries and limitations

This PR does not change runtime request handling or enable session-radix ownership. Keeping the current runtime no-op avoids forwarding arbitrary header IDs as session_params.id, which would make SGLang 0.5.14 reject requests whose sessions were not opened first.

Validation

  • pre-commit run --files <changed files>
  • fern check
  • fern docs broken-links
  • bash -n examples/backends/sglang/launch/agg_agent.sh examples/backends/sglang/launch/_test_agg.sh
  • git diff --check origin/main...HEAD

Summary by CodeRabbit

  • Documentation
    • Clarified SGLang guidance around session handling, tracing, and router affinity.
    • Updated references across backend and frontend docs to focus on priority scheduling and KV cache eviction.
    • Removed outdated wording that suggested session-based radix cache tagging was available.
  • Chores
    • Aligned example launch text and inline notes with the latest SGLang behavior and terminology.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani
ishandhanani had a problem deploying to external_collaborator July 6, 2026 15:35 — with GitHub Actions Failure
@ishandhanani
ishandhanani temporarily deployed to external_collaborator July 6, 2026 15:35 — with GitHub Actions Inactive
@github-actions github-actions Bot added docs documentation Improvements or additions to documentation backend::sglang Relates to the sglang backend labels Jul 6, 2026
@datadog-official

datadog-official Bot commented Jul 6, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 16.25% (-29.85%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 3ef8e83 | Docs | Give us feedback!

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

@ishandhanani
ishandhanani marked this pull request as ready for review July 6, 2026 15:42
@ishandhanani
ishandhanani requested review from a team as code owners July 6, 2026 15:42
@ishandhanani
ishandhanani requested a review from a team July 6, 2026 15:42
@ishandhanani
ishandhanani requested review from a team as code owners July 6, 2026 15:42
@ishandhanani

Copy link
Copy Markdown
Contributor Author

/ok-to-test e3a988e

devin-ai-integration[bot]

This comment was marked as resolved.

@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Documentation and inline comments across CLAUDE.md, several docs files, and SGLang launch scripts are updated to remove claims that SGLang supports passive session-aware radix KV ownership, replacing them with descriptions of priority-based KV eviction and clarifying that session headers remain used for tracing and router affinity only.

Changes

Session Identity Documentation Correction

Layer / File(s) Summary
Core session identity docs update
components/src/dynamo/sglang/CLAUDE.md, docs/backends/sglang/agents.md
Replaces "session radix cache" pitfall/section content with "session identity" content stating SGLang 0.5.14 lacks passive session-aware radix ownership, clarifies session_params ties to explicit open_session lifecycle, and notes session headers/priority eviction flag do not tag KV by session.
Cross-referencing doc updates
docs/backends/sglang/sglang-chat-processor.md, docs/components/frontend/nvext.md, docs/digest/agentic-inference/agentic-inference.md
Updates "See Also" descriptions and related text to reference priority scheduling/KV eviction policies instead of session-aware radix KV tagging.
Launch script comment and banner updates
examples/backends/sglang/launch/_test_agg.sh, examples/backends/sglang/launch/agg_agent.sh
Removes session-aware radix ownership wording from printed banners and inline comments, replacing with priority-based radix eviction descriptions; no runtime logic changed.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Related PRs: None identified.

Suggested labels: documentation

Suggested reviewers: None identified.

🐰 A rabbit hops through docs so fine,
Erasing claims of session's shine,
Priority now leads the KV way,
Headers just trace, come what may,
Comments and banners align.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is detailed, but it does not follow the required template and the Related Issues section is incomplete. Add the required Overview, Details, Where should reviewer start?, and Related Issues sections, and use the template's Closes #XXXX or no-issue format.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately reflects the main change: correcting SGLang session-radix availability docs.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani
ishandhanani temporarily deployed to external_collaborator July 6, 2026 16:12 — with GitHub Actions Inactive
@ishandhanani
ishandhanani enabled auto-merge (squash) July 6, 2026 16:41
@ishandhanani
ishandhanani merged commit d245a5b into ai-dynamo:main Jul 6, 2026
89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::sglang Relates to the sglang backend docs documentation Improvements or additions to documentation size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants