Stream emulated tool-call prose incrementally

[i386]

What changed

• replace newline-based partial buffering with marker-aware buffering
• stream ordinary single-line prose immediately
• retain only a possible trailing TOOL_CALL marker prefix
• withhold a detected marker and payload until final structured parsing
• ignore markers inside completed <think> blocks

Why

The previous partial parser discarded the entire final line. Models commonly generate answers without newlines, so tool-enabled streaming requests that did not call a tool could produce no content until completion.

Impact

Normal prose regains incremental TTFT while incomplete emulated tool calls remain hidden from client-visible text.

Validation

• cargo test -p skippy-server --lib — 204 passed
• cargo clippy -p skippy-server --all-targets -- -D warnings

Summary by CodeRabbit

• Bug Fixes
  • Improved streaming responses involving emulated tool calls.
  • Prevented incomplete or marker-like text from appearing as tool calls before parsing is complete.
  • Preserved regular prose when streaming ends mid-line.
  • Ignored tool-call markers appearing within in-progress reasoning sections.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6a771479-efb3-4478-91c5-673db64589df

📥 Commits

Reviewing files that changed from the base of the PR and between 36a44b9 and c423013.

📒 Files selected for processing (3)

• crates/skippy-server/src/frontend/prompting.rs
• crates/skippy-server/src/frontend/tests.rs
• crates/skippy-server/src/frontend/tool_emulation.rs

---

📝 Walkthrough

Walkthrough

Partial emulated tool-call parsing now uses dedicated text preparation during streaming. New tests cover truncated prose, incomplete markers, and marker-like text inside incomplete thinking blocks.

Changes

Partial emulated tool-call parsing

Layer / File(s)	Summary
Partial scan preparation and validation crates/skippy-server/src/frontend/tool_emulation.rs, crates/skippy-server/src/frontend/prompting.rs, crates/skippy-server/src/frontend/tests.rs	partial_emulation_text removes completed thinking blocks, truncates at complete markers, and preserves only marker prefixes that may still form TOOL_CALL; partial parsing uses this text and tests cover the streaming cases.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• Mesh-LLM/mesh-llm#946: Adds the related emulated TOOL_CALL parsing and streaming behavior refined here.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: incremental streaming of emulated tool-call prose.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch jd/fix-emulated-tool-streaming

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

This pull request is currently a draft. Reviews will not take place until the PR is marked as ready for review.