server: fix prefill speed regression on session resumption

[SakshamKapoor2911]

Overview

This PR resolves the prefill speed regression on session resumption (Issue #25213).

Previously, the prefill batch loop in server-context.cpp unconditionally split the batch at any user message start boundary. When resuming a session, this caused the prefill sequence to be fragmented into many tiny batches, severely bottlenecking GPU prefill throughput.

This PR optimizes prefill batching and adds dynamic tool start detection to prevent unnecessary batch splitting:

1. Conditional Batch Splitting: The prefill batch loop is refactored to conditionally break before a user message start boundary only when context checkpointing is enabled, and a checkpoint is actually scheduled to be taken (i.e., spacing respects checkpoint_min_step).
2. Tool Response Start Delimiter Detection: Added dynamic tool start detection using template diffing in detect_tool_response_start_marker to ensure tool outputs are not misclassified as user roles.
3. Delimiter Match Order Overlap: Parsed delimiters are sorted by length descending to ensure longer, specialized delimiters match first instead of partial overlapping short matches.
4. Restored Default: Reverted the temporary workaround value of checkpoint_min_step from 8192 back to 256 tokens.

Additional information

• Fixes Misc. bug: Substantial prefill speed regression when resuming long session from a coding agent #25213
• Related to PR server: tool role detection and create checkpoints only on user messages #25416. This PR provides a more optimal and correct solution by limiting prefill batch breaks strictly to when checkpointing spacing requires it, rather than unconditionally breaking at all user message boundaries.
• Verified by:
  • Running unit tests: ./build/bin/test-chat-auto-parser (98 tests, 537 assertions passed).
  • Running integration tests: bash tools/server/tests/tests.sh (290 tests passed).

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - AI was used in an assistive capacity to draft code changes for the conditional batch splitting in server-context.cpp, update delimiter sorting in chat.cpp, implement the tool response start detector in chat-diff-analyzer.cpp, and populate the expected test cases in test-chat-auto-parser.cpp. All changes have been compiled, run, and verified locally by the human contributor.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.