fix(sglang): preserve cached token usage

[ishandhanani]

Summary

The unified SGLang backend now forwards SGLang's cached-token count into OpenAI completion usage. This restores cache observability for warm requests, including HiCache; the existing dynamo.sglang path already returns this field.

CLOSES: DYN-3275

How This Was Implemented

• Populate prompt_tokens_details.cached_tokens from final SGLang response metadata when the count is nonzero.

Walkthrough

• SglangLLMEngine.generate includes cached-token details in terminal completion usage.

Validation

• ruff check components/src/dynamo/sglang/llm_engine.py
• Live before/after with Qwen/Qwen3-0.6B, SGLang 0.5.13.post1, and HiCache enabled. The base backend logged #cached-token: 512 on the warm request.

Before (origin/main) — exact warm-response usage:

{"completion_tokens":4,"prompt_tokens":516,"total_tokens":520}

After — exact warm-response usage:

{"completion_tokens":4,"prompt_tokens":516,"prompt_tokens_details":{"audio_tokens":null,"cached_tokens":512},"total_tokens":520}

• Default python -m dynamo.sglang with --enable-cache-report --enable-hierarchical-cache already returns the same warm-response usage as the patched unified backend.
• Bare sglang.launch_server returned prompt_tokens_details.cached_tokens: 512 on the warm request.

Summary by CodeRabbit

• New Features
  • Generation responses now include additional prompt token details in usage data when available.
  • Cached prompt token counts are surfaced for completed generation chunks, improving usage visibility.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e8910803-dcc5-471f-8927-953cb1b89a0f

📥 Commits

Reviewing files that changed from the base of the PR and between 1ede0bb and fc78393.

📒 Files selected for processing (1)

• components/src/dynamo/sglang/llm_engine.py

---

Walkthrough

SglangLLMEngine.generate() extends the completion_usage dict emitted for finished generation chunks with a new prompt_tokens_details field. The field is set to {"cached_tokens": <value>} when meta_info contains cached_tokens, otherwise None.

Changes

Cached token details in completion_usage

Layer / File(s)	Summary
Add prompt_tokens_details to terminal completion_usage components/src/dynamo/sglang/llm_engine.py	When a generation chunk finishes, completion_usage now conditionally includes prompt_tokens_details derived from meta_info.cached_tokens (or None if absent), alongside the existing prompt_tokens, completion_tokens, and total_tokens fields.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the change: preserving cached token usage in the SGLang backend.
Description check	✅ Passed	The description includes summary, implementation, validation, and a related issue reference, though it omits the reviewer-start section.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands.}