feat(sglang): enforce stream_output=True for optimal streaming performance

[MatejKosec]

Summary

• Enforce stream_output=True in SGLang ServerArgs for Dynamo
• Update streaming handlers to pass through disjoint token segments directly (no more cumulative-to-delta conversion)
• Applies to both LLM decode handler and multimodal worker handler

Description

With stream_output=True, SGLang sends only new tokens since the last output (disjoint segments) rather than all tokens generated so far (cumulative). This change:

1. Forces stream_output=True in args.py after parsing ServerArgs
2. Simplifies _process_token_stream in decode_handler - removes tracking/slicing logic
3. Simplifies process_sglang_stream in multimodal worker_handler - same fix

This aligns Dynamo with SGLang's efficient streaming mode, reducing redundant data transfer.

Summary by CodeRabbit

• Bug Fixes
  • Improved token streaming to deliver disjoint segments instead of cumulative tokens, ensuring more accurate and granular token delivery during streaming operations.
  • Enabled stream output mode in server configuration for consistent streaming behavior across LLM and multimodal handlers.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

The pull request enforces stream output with disjoint token segments across Dynamo's SGLang integration. Stream output is now forcibly enabled in argument parsing, and both token stream handlers are refactored to forward token segments directly rather than computing them from running totals or offsets.

Changes

Cohort / File(s)	Summary
Stream Output Configuration components/src/dynamo/sglang/args.py	Unconditionally enables stream_output on server args after parsing, logging a message if it was previously disabled. Ensures consistent streaming behavior.
Token Stream Processing components/src/dynamo/sglang/request_handlers/llm/decode_handler.py, components/src/dynamo/sglang/request_handlers/multimodal/worker_handler.py	Refactored _process_token_stream and StreamProcessor.process_sglang_stream to use disjoint token segments from output_ids instead of slicing from running offsets. Removes accumulated offset tracking and clarifies streaming semantics. Error handling preserved.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 Streams now flow in segments bright,
No more offsets to recalculate each night,
Tokens hop along, disjoint and free,
SGLang streams as they should be! 🎉

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: enforcing stream_output=True in SGLang for optimal streaming, which is the primary objective across all three modified files.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description check	✅ Passed	The PR description follows the required template structure with all key sections present: Overview (Summary), Details, and related guidance. It clearly explains the changes, rationale, and includes a test plan.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}