Skip to content

.NET: Preserve UTF-8 order in HeadTailBuffer#7128

Merged
rogerbarreto merged 2 commits into
microsoft:mainfrom
jstar0:fix/head-tail-buffer-utf8-boundary
Jul 16, 2026
Merged

.NET: Preserve UTF-8 order in HeadTailBuffer#7128
rogerbarreto merged 2 commits into
microsoft:mainfrom
jstar0:fix/head-tail-buffer-utf8-boundary

Conversation

@jstar0

@jstar0 jstar0 commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Motivation & Context

HeadTailBuffer can silently drop and reorder multi-byte UTF-8 output when a complete rune no longer fits the remaining head bytes but later smaller runes still do. At the exact overall byte cap, the previous logic could evict a tail rune and still report that the output was not truncated. This affects the bounded stdout and stderr capture shared by the local and Docker shell executors.

Description & Review Guide

  • What are the major changes? Seal the head after the first rune spills to the tail, defer whole-rune tail eviction until total input exceeds the overall cap, and add exact-cap and overflow regressions.
  • What is the impact of these changes? Inputs that fit the configured byte cap now round-trip in order without silent loss, while overflow remains bounded and retains only complete UTF-8 runes.
  • What do you want reviewers to focus on? The one-way transition from head to tail and the eviction boundary when total input first exceeds the cap.

Related Issue

Fixes #7112

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI review requested due to automatic review settings July 15, 2026 09:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a UTF-8 boundary bug in the .NET HeadTailBuffer used for bounded stdout/stderr capture, ensuring multi-byte runes are never reordered or silently dropped at the head/tail boundary and that truncation reporting matches actual byte loss.

Changes:

  • Seals the head buffer after the first rune that can’t fully fit, forcing all subsequent runes into the tail to preserve byte order.
  • Defers tail eviction until the overall byte cap is exceeded (not merely the tail budget), preventing silent loss when input is exactly at cap.
  • Adds unit tests covering exact-cap and minimal-overflow multi-byte UTF-8 regression scenarios.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
dotnet/src/Microsoft.Agents.AI.Tools.Shell/HeadTailBuffer.cs Updates head/tail append and eviction logic to preserve UTF-8 rune order and correct truncation behavior.
dotnet/tests/Microsoft.Agents.AI.Tools.Shell.UnitTests/HeadTailBufferTests.cs Adds regression tests for exact-cap and overflow cases involving 4-byte UTF-8 runes.

@rogerbarreto
rogerbarreto requested a review from westey-m July 15, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Usage: [Issues, PRs], Target: .Net

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.NET: [Bug]: Microsoft.Agents.AI.Tools.Shell.HeadTailBuffer can reorder and silently drop UTF-8 runes at the head/tail boundary

5 participants