perf(llm): avoid cloning jail metrics

[jthomson04]

Summary

• move the owned LLMMetricAnnotation out of each Nv stream item before adapting it for dynamo-parsers
• avoid cloning the optional worker-type strings on every jail input chunk while preserving accumulated metrics
• improve long ambiguous-prefix streams by 3.89% and ordinary text by 1.29% in paired current-main microbenchmarks

Validation

Investigation validation:

• cargo fmt --all -- --check
• cargo clippy --locked -p dynamo-llm --no-default-features --lib -- -D warnings
• cargo test --locked --release -p dynamo-llm --no-default-features --test test_streaming_tool_parsers --test test_reasoning_parser --test parallel_tool_call_integration --test tool_choice --test tool_choice_finish_reasons (99 passed, 1 ignored)
• 20-sample-per-role ABBA/BAAB benchmark matrix across seven stream shapes; no statistically resolved regression

Publication recheck on unchanged main:

• cargo fmt --all -- --check
• cargo test -p dynamo-llm --locked --no-default-features --test test_streaming_tool_parsers (37 passed)

Summary by CodeRabbit

• Bug Fixes
  • Improved handling of streamed chat responses so usage metrics are preserved more efficiently across tool-calling flows.
  • Reduced unnecessary copying during response processing, which may lower overhead in streaming scenarios.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[jthomson04]

/ok to test

[copy-pr-bot]

/ok to test

@jthomson04, there was an error processing your request: E1

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/1/

[jthomson04]

/ok to test d98b8a5

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 36b9e0e9-7710-49fc-9992-b6bf821667d3

📥 Commits

Reviewing files that changed from the base of the PR and between d245a5b and d98b8a5.

📒 Files selected for processing (1)

• lib/llm/src/preprocessor.rs

---

Walkthrough

The change modifies apply_tool_calling_jail in the LLM preprocessor to move llm_metrics out of streamed response data using take() instead of cloning it, while preserving existing buffering and re-attachment logic in PendingMetrics.

Changes

Tool-calling jail metrics handling

Layer / File(s)	Summary
Move instead of clone llm_metrics lib/llm/src/preprocessor.rs	The jail_input mapping closure now takes llm_metrics via .take() on a mutable reference instead of cloning it, requiring the closure to bind mut a.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Related PRs: None specified.

Suggested labels: None specified.

Suggested reviewers: None specified.

🤖 A tiny hop, a metric set free,
No more cloning, just take() and flee,
One line moved with care,
Through the jail's buffered air,
A small tweak for efficiency.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description omits required template sections, especially Details, Where should reviewer start?, and Related Issues.	Add the missing template sections, include a reviewer start file/path, and either link the related issue or confirm none.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: reducing cloning in jail metric handling for llm performance.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}