fix(shell): bound pipe read to MaxOutputChars before truncating (#1293)

[Aaronontheweb]

Summary

• shell_execute previously called ReadToEndAsync on both stdout and stderr, buffering the entire child output before any size limit was applied. A command emitting hundreds of MB (e.g. kubectl logs, a large curl response) would allocate the full payload — plus several derived copies for StringBuilder, ToString(), Redact(), and the base64 inflation for LOH purposes — before TruncateOutput ran. The 32k MaxOutputChars cap protected the LLM's context window, not the process.
• Replaces ReadToEndAsync with BoundedDrainAsync: reads into a head + tail ring-buffer window capped at MaxOutputChars. Chars beyond the cap are discarded on read but the pipe continues to drain so a running child never deadlocks on a full pipe buffer (the existing deadlock-prevention comment and CancellationToken.None rationale are preserved).
• Redaction and result assembly now operate on the already-bounded strings — not the raw full output.
• The model sees a head+tail view (...\n...\n...) when truncated, so tail-of-log content (error summaries, final exit status) survives instead of being silently dropped by head-only truncation.

Test plan

• All existing ShellToolTests pass (30 total, including the updated Output_truncation_applies assertion)
• 6 new BoundedDrain_* unit tests covering: short output returned verbatim, empty input, output exactly at cap, long output with distinct head/tail markers, even head/tail split, disabled cap falls back to unbounded read
• dotnet slopwatch analyze — 0 issues

Closes #1293.

[Aaronontheweb]

Benchmark results

BenchmarkDotNet ([MemoryDiagnoser]) over the benchmarks/Netclaw.Benchmarks harness added in this PR. The benchmark isolates the drain loop — it feeds BoundedDrainAsync a synthetic in-memory TextReader (no process spawn, no real pipe I/O), so the numbers reflect only the cost of turning a stream into a bounded, redaction-ready string. Cap is the production default MaxOutputChars = 32_000. Baseline is the old path (ReadToEndAsync + TruncateOutput).

Run config: --job short (3 iterations) on the optimized drain. Allocation figures (MemoryDiagnoser) are deterministic; the time error bars are wide on the small inputs due to the short job, but the order-of-magnitude story holds. The post-review hardening (in-place reconstruction on the truncated path) only reduces allocation further than shown here.

TotalChars	Implementation	Mean	Allocated	Gen2
1,000	ReadToEnd + Truncate (old)	1,971 ns	10.53 KB	—
1,000	BoundedDrainAsync (new)	521 ns	10.23 KB	—
32,000	ReadToEnd + Truncate (old)	12,409 ns	127.00 KB	—
32,000	BoundedDrainAsync (new)	8,146 ns	156.76 KB	—
1,000,000	ReadToEnd + Truncate (old)	1,502,686 ns	4,005.85 KB	498
1,000,000	BoundedDrainAsync (new)	51,219 ns	188.41 KB	—
50,000,000	ReadToEnd + Truncate (old)	54,004,217 ns	197,197.51 KB	2,600
50,000,000	BoundedDrainAsync (new)	2,042,472 ns	188.41 KB	—

What the numbers show

• Allocation is bounded by the cap, not the output. The new path allocates 188.41 KB at both 1M and 50M chars — identical to the byte. The old path scales with total output (193 MB at 50M chars). That O(cap) flatness is the property that stops the cgroup OOM-kill on a memory-limited daemon (shell_execute buffers entire child output into memory before truncating (LOH/OOM risk) #1293).
• Zero Gen2 / LOH collections on the new path at every size. The old path forces 2,600 Gen2 collections per 1k ops at 50M chars — that GC pressure was the kill mechanism.
• Faster across the board, despite doing more bookkeeping: ~26× at 50M chars (54 ms → 2 ms), ~29× at 1M.
• The common small-output case is faster and lighter than the old ReadToEnd path (521 ns / 10.23 KB vs 1,971 ns / 10.53 KB at 1,000 chars).

The one spot slightly above baseline is exactly at the cap (32k: 156.76 KB vs 127 KB) — head and tail buffers both allocate right at the boundary where nothing is truncated. It's bounded (~30 KB), only at that precise size, and the path is still ~34% faster there.

Reproduce: dotnet run -c Release --project benchmarks/Netclaw.Benchmarks -- --filter '*ShellDrain*' --job short