Skip to content

fix(agent): auto-continue output-cap truncations below compaction threshold - #1684

Merged
lavaman131 merged 1 commit into
mainfrom
fix/output-cap-auto-continue
Jul 9, 2026
Merged

fix(agent): auto-continue output-cap truncations below compaction threshold#1684
lavaman131 merged 1 commit into
mainfrom
fix/output-cap-auto-continue

Conversation

@lavaman131

@lavaman131 lavaman131 commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Problem

When a model response is truncated at the provider's per-turn output-token cap (stopReason: "length"), the model's work dead-ends with the red "Error: Model stopped because it reached the maximum output token limit. The response may be incomplete." banner, and the task is left half-finished.

The existing recovery only kicks in as a side effect of threshold auto-compaction: _checkCompaction continues a length-truncated turn (willRetry: true) only when the context also crosses the compaction threshold. When the context is still below the compaction budget — genuine long output with input room to spare — _checkCompaction did nothing, so the truncation was never continued.

This closes the gap left by the compaction-path fix (#1662), which only handled length stops large enough to compact.

Fix

In _checkCompaction (packages/coding-agent/src/core/agent-session-auto-compaction.ts), when a response is length-truncated with real output (shouldRetryAfterThresholdCompaction) but the context is below the compaction budget, continue the generation directly without compacting via the new _resumeAfterLengthTruncation:

  • Drop the incomplete length-stopped assistant from retry context (agent.continue() rejects an assistant tail), keeping it in persisted history.
  • Resume via the existing post-compaction continuation probe (_schedulePostAutoCompactionContinuationProbe) so the model finishes where it left off, reusing the normal continuation lifecycle.
  • Bound consecutive direct continuations with MAX_LENGTH_CONTINUATION_ATTEMPTS (3) so a turn that keeps exceeding the per-turn output cap still terminates. The counter (_lengthContinuationAttempts) resets in agent-session-events.ts on any non-length assistant completion.
  • Gate the resume to the live turn-completion path (agent_end, skipAbortedCheck = true), so a fresh user prompt never resumes a previously truncated turn.

Compaction-driven continuation and overflow recovery are unchanged.

Tests

Added coverage in agent-session-auto-compaction-queue-03.suite.ts:

  • continues a below-threshold length stop without calling _runAutoCompaction
  • does not continue a below-threshold zero-output length stop
  • does not resume before a fresh user prompt (non-live path)
  • stops continuing after MAX_LENGTH_CONTINUATION_ATTEMPTS

All existing auto-compaction / retry / safety-refusal suites pass; typecheck, lint, check:file-length, and test:unit are green. Docs (packages/coding-agent/docs/compaction.md) and CHANGELOG.md updated.

…eshold

A response truncated at the provider's per-turn output-token cap
(stopReason "length") only auto-continued as a side effect of threshold
compaction. When the context was still below the compaction budget,
_checkCompaction did nothing, so the model's work dead-ended with the red
"maximum output token limit" error and the task was left half-finished.

Continue length-truncated turns directly when compaction is not needed:
drop the incomplete assistant from retry context and resume generation via
the existing post-compaction continuation probe so the model finishes where
it left off. A small consecutive-continuation cap
(MAX_LENGTH_CONTINUATION_ATTEMPTS) prevents runaway loops when a turn keeps
exceeding the per-turn output cap, and the resume is gated to the live
turn-completion path so a fresh user prompt never resumes an old turn.

Adds coverage for the below-threshold continue, zero-output no-op, the
non-live pre-prompt guard, and the attempt bound.
@mintlify

mintlify Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
bastani 🟢 Ready View Preview Jul 9, 2026, 12:27 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@claude claude Bot changed the title fix(agent): auto-continue output-cap truncations below the compaction threshold fix(agent): auto-continue output-cap truncations below compaction threshold Jul 9, 2026
@claude

claude Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review: auto-continue output-cap truncations below the compaction threshold

Nice, well-scoped fix that closes a real gap left by #1662. The design reuses the existing post-compaction continuation lifecycle rather than inventing a new path, the live-turn gating (isLiveTurnCompletion) is a clean way to distinguish agent_end from the pre-prompt check, and the doc/CHANGELOG updates are thorough. Test coverage for the four core behaviors is good.

A couple of things worth considering before merge:

1. _lengthContinuationAttempts is not reset at the start of a new user prompt (correctness)

The counter only resets when a non-length assistant message is processed (agent-session-events.ts:159-161). But when a turn genuinely exhausts the cap, its last assistant message is a length stop and no non-length completion follows — so the counter stays at MAX_LENGTH_CONTINUATION_ATTEMPTS (3) and leaks into the next turn.

Scenario:

  1. Turn A hits the per-turn output cap and dead-ends after 3 continuations → _lengthContinuationAttempts === 3, last assistant is length.
  2. User submits an unrelated Turn B whose first response is also long enough to hit the output cap (stopReason: "length").
  3. message_end sees stopReason === "length", so it does not reset. agent_end_checkCompaction_resumeAfterLengthTruncation, where 3 >= 3 returns immediately.
  4. Turn B's legitimate long output is never auto-continued — the very bug this PR fixes.

The pre-prompt path (agent-session-prompt.ts:122) doesn't reset it either. Consider resetting _lengthContinuationAttempts = 0 when a fresh user prompt is submitted in prompt(), and adding a test for "a new prompt after an exhausted turn still auto-continues." Conceptually the cap is meant to bound continuations within a single turn, so tying the reset to turn boundaries (not just to observing a non-length completion) matches the intent.

2. Counter increments even if the scheduled probe skips the continuation (minor)

_resumeAfterLengthTruncation increments the counter synchronously and drops the trailing assistant, but the actual continuation is deferred 100ms and is skipped when isCompacting || isStreaming (_schedulePostAutoCompactionContinuationProbe, lines 191-194). In that case an attempt is "spent" without any continuation occurring, so a turn could exhaust its budget faster than intended. Low severity given the narrow race window, but worth a comment or a guard.

Minor / nits

  • The prompt() promise for a below-threshold length continuation resolves before the async continuation runs (the threshold branch doesn't set _pendingOverflowPostCompactionContinuation). This matches existing threshold-compaction behavior, so it's consistent — just flagging that a workflow caller awaiting prompt() won't observe the continuation, unlike the overflow path documented in compaction.md.
  • Double blank lines before _resumeAfterLengthTruncation and around its export in agent-session-auto-compaction.ts — cosmetic.

Summary

Solid fix; the reuse of the existing continuation machinery is the right call. The cross-turn counter reset (#1) is the one item I'd recommend addressing (plus a regression test) before merge; the rest are minor.

@lavaman131
lavaman131 merged commit aabefd8 into main Jul 9, 2026
11 checks passed
@lavaman131
lavaman131 deleted the fix/output-cap-auto-continue branch July 9, 2026 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant