diff --git a/packages/coding-agent/CHANGELOG.md b/packages/coding-agent/CHANGELOG.md index 9b56a829e..be93d198d 100644 --- a/packages/coding-agent/CHANGELOG.md +++ b/packages/coding-agent/CHANGELOG.md @@ -9,7 +9,8 @@ ### Fixed - Fixed Windows native filesystem watchers to canonicalize watched paths before calling `fs.watch`, reject unresolved 8.3 short-name paths, and fall back to polling for unsafe watcher paths. This prevents libuv fs-event assertion crashes when temp, session, footer, or theme paths contain short-name components such as `USERNA~1`. -- Fixed responses truncated at the output-token cap (`stopReason: "length"`) dead-ending the model's work with a "maximum output token limit" error when the context was still below the auto-compaction threshold. Previously auto-continuation only happened as a side effect of threshold compaction, so a length truncation with input room to spare left the task half-finished. Length-truncated turns now continue directly without compacting: the incomplete assistant is removed from retry context and the generation resumes automatically so the model finishes where it left off, bounded by a small consecutive-continuation cap so a turn that keeps exceeding the per-turn output cap still terminates. Compaction-driven continuation and fresh user prompts are unaffected. +- Fixed responses truncated at the output-token cap (`stopReason: "length"`) dead-ending the model's work with a "maximum output token limit" error when the context was still below the auto-compaction threshold. Previously auto-continuation only happened as a side effect of threshold compaction, so a length truncation with input room to spare left the task half-finished. Length-truncated turns now continue directly without compacting: the incomplete assistant is removed from retry context and the generation resumes automatically so the model finishes where it left off, bounded by a small consecutive-continuation cap so a turn that keeps exceeding the per-turn output cap still terminates. Compaction-driven `willRetry: true` continuations are now consistently tracked through the normal prompt lifecycle, so `AgentSession.prompt()` waits for threshold length-stop continuation before resolving instead of treating it as fire-and-forget. +- Fixed auto-compaction after OpenAI Responses output-budget underflow errors such as `Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 1 instead.` Atomic now classifies that specific context-pressure failure as retry-worthy when the live context crosses the compaction threshold, removes the empty error assistant from retry context after compaction, and automatically continues from the preceding work anchor instead of waiting for the user to type `Continue`. Generic `invalid_request_body` errors such as malformed tool schemas are still not auto-retried, OpenAI Responses payload sanitization now prevents `max_output_tokens` values below the provider minimum of 16 from being sent, and repeated output-budget underflow continuation is capped at one compact-and-retry attempt so an unrecoverable context terminates visibly instead of looping or stalling. - Fixed bundled workflow durable resume for reusable `git_worktree_dir` worktrees so resumed runs reuse the original invocation repository/cwd and report slow Git subprocess timeouts as Git timeouts instead of repository-detection failures. ## [0.9.5-alpha.8] - 2026-07-08 diff --git a/packages/coding-agent/docs/compaction.md b/packages/coding-agent/docs/compaction.md index 927040e2f..8dec0f692 100644 --- a/packages/coding-agent/docs/compaction.md +++ b/packages/coding-agent/docs/compaction.md @@ -108,9 +108,11 @@ You can also trigger compaction manually with `/compact`. Custom summary instruc If auto-compaction runs while a turn still has queued work (for example a failed tool-call result or a follow-up queued during compaction), Atomic resumes through the same continuation lifecycle as a normal queued turn: provider retry handling runs, additional queued messages drain, and any post-compaction resume failure is surfaced instead of being swallowed silently. -For provider-overflow recovery that succeeds with `willRetry: true`, the public `AgentSession.prompt()` promise remains pending until the post-compaction retry continuation has run through the normal continuation lifecycle. If that continuation exhausts the one compact-and-retry attempt and emits `compaction_end` with `unresolvedOverflow: true`, workflow callers can observe the signal before deciding whether the prompt succeeded or should advance model fallback. +For any compaction event that succeeds with `willRetry: true`, the public `AgentSession.prompt()` promise remains pending until the post-compaction retry continuation has run through the normal continuation lifecycle. This includes overflow recovery, threshold recovery after output-token length stops, and threshold recovery after retry-worthy OpenAI Responses output-budget errors. If overflow continuation exhausts the one compact-and-retry attempt and emits `compaction_end` with `unresolvedOverflow: true`, workflow callers can observe the signal before deciding whether the prompt succeeded or should advance model fallback. -When an assistant response is truncated at the provider's per-turn output-token cap (`stopReason: "length"`) with real output produced, Atomic treats it as work cut off mid-flight and continues it automatically instead of leaving the turn dead-ended on the "maximum output token limit" error. If the context is at or above the compaction threshold, the truncation is recovered through the normal compact-and-continue path (the incomplete assistant is dropped from retry context, then generation resumes). If the context is still below the threshold — genuine long output with input room to spare — compaction would free no room, so Atomic continues the generation directly without compacting. Consecutive direct continuations are bounded by a small cap, so a turn that keeps exceeding the per-turn output cap still terminates rather than looping. This resume applies only to the live turn-completion path; a fresh user prompt never resumes a previously truncated turn. +When an assistant response is truncated at the provider's per-turn output-token cap (`stopReason: "length"`) with real output produced, Atomic treats it as work cut off mid-flight and continues it automatically instead of leaving the turn dead-ended on the "maximum output token limit" error. If the context is at or above the compaction threshold, the truncation is recovered through the normal compact-and-continue path (the incomplete assistant is dropped from retry context, then generation resumes, and `AgentSession.prompt()` waits for that continuation). If the context is still below the threshold — genuine long output with input room to spare — compaction would free no room, so Atomic continues the generation directly without compacting. Consecutive direct continuations are bounded by a small cap, so a turn that keeps exceeding the per-turn output cap still terminates rather than looping. This resume applies only to the live turn-completion path; a fresh user prompt never resumes a previously truncated turn. + +OpenAI Responses providers can also report context pressure as a request-budget underflow instead of a normal context-overflow stop, for example `Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 1 instead.` When that exact output-budget family of errors arrives on a live, threshold-sized context, Atomic treats it as retry-worthy interrupted work: auto-compaction records the `context_compaction` entry, removes the empty error assistant from retry context, and automatically continues from the preceding user/tool-result anchor. Other `invalid_request_body` errors, such as malformed tool schemas, remain visible and are not auto-retried through compaction. Output-budget underflow uses a separate one-attempt guard and intentionally does not set `unresolvedOverflow`; if the compact-and-continue attempt still cannot produce a non-error assistant turn, the session leaves the visible terminal provider error in place instead of looping or advancing overflow-specific fallback. ### Image Context and Compaction diff --git a/packages/coding-agent/docs/json.md b/packages/coding-agent/docs/json.md index 0401cbc76..5e9769a8b 100644 --- a/packages/coding-agent/docs/json.md +++ b/packages/coding-agent/docs/json.md @@ -26,7 +26,7 @@ type AgentSessionEvent = `queue_update` emits the full pending steering and follow-up queues whenever they change. `session_info_changed`, `model_changed`, `thinking_level_changed`, and `context_window_changed` report interactive session metadata changes. `context_window_changed` carries the active token budget after `AgentSession.setContextWindow()` or branch navigation replay applies a branch-scoped `context_window_change`; branch replay does not add another session journal entry or write settings. `compaction_start` and `compaction_end` cover both manual and automatic Verbatim Compaction, Atomic's transcript-bound, deletion-only Context Compaction approach inspired by [Morph's Context Compaction](https://www.morphllm.com/context-compaction). -For overflow auto-compaction, `compaction_end.willRetry === true` means the agent is retrying the interrupted turn after compaction; `AgentSession.prompt()` waits for that continuation before resolving. If the same-model compact-and-retry path is exhausted, `compaction_end` includes `unresolvedOverflow: true` plus an `errorMessage` so orchestration layers can fallback to another model instead of treating the prompt as successful. +For automatic compaction, `compaction_end.willRetry === true` means the agent is retrying the interrupted turn after compaction; `AgentSession.prompt()` waits for that continuation before resolving. This includes overflow recovery and live threshold compaction for retry-worthy interrupted work such as output-token truncation or OpenAI Responses output-budget underflow errors. Generic provider `invalid_request_body` failures still compact with `willRetry: false` when threshold compaction is warranted. If the same-model compact-and-retry overflow path is exhausted, `compaction_end` includes `unresolvedOverflow: true` plus an `errorMessage` so orchestration layers can fallback to another model instead of treating the prompt as successful. Base events come from `AgentEvent` in `@earendil-works/pi-agent-core` (installed as an Atomic dependency): diff --git a/packages/coding-agent/src/core/agent-session-auto-compaction.ts b/packages/coding-agent/src/core/agent-session-auto-compaction.ts index 9998e4447..262c7ceed 100644 --- a/packages/coding-agent/src/core/agent-session-auto-compaction.ts +++ b/packages/coding-agent/src/core/agent-session-auto-compaction.ts @@ -4,6 +4,7 @@ import { getEffectiveInputBudget } from "./context-window.ts"; import { parseCopilotPromptLimitError } from "./copilot-errors.ts"; import { calculateContextTokens, estimateContextTokens, shouldCompact } from "./compaction/index.ts"; import { getLatestCompactionBoundaryEntry } from "./session-manager.ts"; +import { MIN_RESPONSES_MAX_OUTPUT_TOKENS } from "./openai-responses-payload-sanitizer.ts"; import type { AgentSessionInternalSurface as AgentSession } from "./agent-session-methods.ts"; /** @@ -15,6 +16,20 @@ import type { AgentSessionInternalSurface as AgentSession } from "./agent-sessio */ export const MAX_LENGTH_CONTINUATION_ATTEMPTS = 3; +export const MAX_OUTPUT_BUDGET_ERROR_CONTINUATION_ATTEMPTS = 1; + +type JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue }; +type ProviderErrorDetails = { + message?: string; + code?: string; + param?: string; +}; + +const OUTPUT_BUDGET_PARAMETER_PATTERN = /\bmax_output_tokens\b/; +const OUTPUT_BUDGET_UNDERFLOW_PATTERN = new RegExp( + `(?:integer\\s+below\\s+minimum\\s+value|expected\\s+(?:a\\s+)?value\\s*>=\\s*${MIN_RESPONSES_MAX_OUTPUT_TOKENS}|got\\s+1\\s+instead)`, +); + export async function _checkCompaction(this: AgentSession, assistantMessage: AssistantMessage, skipAbortedCheck = true): Promise { // The agent_end path passes skipAbortedCheck=true; the pre-prompt path passes // false. Only the live turn-completion path may auto-continue a truncated @@ -113,7 +128,23 @@ export async function _checkCompaction(this: AgentSession, assistantMessage: Ass // rather than relying on reactive overflow recovery near the cap. const compactionBudget = this.model ? getEffectiveInputBudget(this.model) : contextWindow; if (shouldCompact(contextTokens, compactionBudget, settings)) { - await this._runAutoCompaction("threshold", shouldRetryAfterThresholdCompaction(assistantMessage)); + const willRetry = shouldRetryAfterThresholdCompaction(assistantMessage); + if (willRetry && isRetryWorthyOutputBudgetError(assistantMessage)) { + if (this._outputBudgetErrorContinuationAttempts >= MAX_OUTPUT_BUDGET_ERROR_CONTINUATION_ATTEMPTS) { + this._emit({ + type: "compaction_end", + reason: "threshold", + result: undefined, + aborted: false, + willRetry: false, + errorMessage: + "Output-budget recovery stopped after a compact-and-retry attempt. Try reducing context or switching to a larger-context model.", + }); + return; + } + this._outputBudgetErrorContinuationAttempts += 1; + } + await this._runAutoCompaction("threshold", willRetry); return; } @@ -122,7 +153,7 @@ export async function _checkCompaction(this: AgentSession, assistantMessage: Ass // context overflow. Compaction would not free any room, so continue the // generation directly instead of dead-ending on the truncation and leaving // the task half-finished. - if (isLiveTurnCompletion && shouldRetryAfterThresholdCompaction(assistantMessage)) { + if (isLiveTurnCompletion && isRetryWorthyLengthStop(assistantMessage)) { this._resumeAfterLengthTruncation(); } } @@ -138,10 +169,73 @@ export function _isCopilotServerCapBelowSelectedContextWindow(this: AgentSession return promptLimitError !== undefined && getEffectiveInputBudget(this.model) > promptLimitError.limitTokens; } -export function shouldRetryAfterThresholdCompaction(assistantMessage: AssistantMessage): boolean { +export function isRetryWorthyLengthStop(assistantMessage: AssistantMessage): boolean { return assistantMessage.stopReason === "length" && assistantMessage.usage.output > 0; } +function isJsonRecord(value: JsonValue): value is { [key: string]: JsonValue } { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function stringField(record: { [key: string]: JsonValue }, key: string): string | undefined { + const value = record[key]; + return typeof value === "string" ? value : undefined; +} + +function parseProviderErrorDetails(errorMessage: string): ProviderErrorDetails | undefined { + const start = errorMessage.indexOf("{"); + const end = errorMessage.lastIndexOf("}"); + if (start === -1 || end <= start) return undefined; + + try { + const parsed = JSON.parse(errorMessage.slice(start, end + 1)) as JsonValue; + if (!isJsonRecord(parsed)) return undefined; + + const nestedError = parsed.error; + if (nestedError !== undefined && isJsonRecord(nestedError)) { + return { + message: stringField(nestedError, "message"), + code: stringField(nestedError, "code") ?? stringField(parsed, "code"), + param: stringField(nestedError, "param") ?? stringField(parsed, "param"), + }; + } + + return { + message: stringField(parsed, "message"), + code: stringField(parsed, "code"), + param: stringField(parsed, "param"), + }; + } catch { + return undefined; + } +} + +function isOutputBudgetUnderflowText(text: string): boolean { + const message = text.toLowerCase(); + return OUTPUT_BUDGET_PARAMETER_PATTERN.test(message) && OUTPUT_BUDGET_UNDERFLOW_PATTERN.test(message); +} + +function isStructuredOutputBudgetUnderflow(details: ProviderErrorDetails): boolean { + const message = details.message?.toLowerCase() ?? ""; + const param = details.param?.toLowerCase(); + if (!OUTPUT_BUDGET_UNDERFLOW_PATTERN.test(message)) return false; + return param !== undefined ? OUTPUT_BUDGET_PARAMETER_PATTERN.test(param) : OUTPUT_BUDGET_PARAMETER_PATTERN.test(message); +} + +export function isRetryWorthyOutputBudgetError(assistantMessage: AssistantMessage): boolean { + if (assistantMessage.stopReason !== "error" || !assistantMessage.errorMessage) return false; + if (assistantMessage.api !== "openai-responses") return false; + + const structuredDetails = parseProviderErrorDetails(assistantMessage.errorMessage); + if (structuredDetails && isStructuredOutputBudgetUnderflow(structuredDetails)) return true; + + return isOutputBudgetUnderflowText(assistantMessage.errorMessage); +} + +export function shouldRetryAfterThresholdCompaction(assistantMessage: AssistantMessage): boolean { + return isRetryWorthyLengthStop(assistantMessage) || isRetryWorthyOutputBudgetError(assistantMessage); +} + /** * Internal: remove an incomplete assistant from retry context before auto-continuing after compaction. */ @@ -161,30 +255,30 @@ export function _dropTrailingAutoCompactionRetryAssistantIfPresent(this: AgentSe */ export function _schedulePostAutoCompactionContinuationProbe(this: AgentSession, - reason: "overflow" | "threshold", + _reason: "overflow" | "threshold", willRetry: boolean, ): void { - if (reason === "overflow" && willRetry) { - const token = this._overflowPostCompactionContinuationToken + 1; - this._overflowPostCompactionContinuationToken = token; + if (willRetry) { + const token = this._postCompactionContinuationToken + 1; + this._postCompactionContinuationToken = token; let pending: Promise; pending = new Promise((resolve) => { setTimeout(() => { void (async () => { try { - if (this._overflowPostCompactionContinuationToken !== token) return; + if (this._postCompactionContinuationToken !== token) return; if (this.isCompacting || this.isStreaming) return; await this._resumeAfterAutoCompaction(); } finally { - if (this._pendingOverflowPostCompactionContinuation === pending) { - this._pendingOverflowPostCompactionContinuation = undefined; + if (this._pendingPostCompactionContinuation === pending) { + this._pendingPostCompactionContinuation = undefined; } resolve(); } })(); }, 100); }); - this._pendingOverflowPostCompactionContinuation = pending; + this._pendingPostCompactionContinuation = pending; return; } @@ -193,11 +287,6 @@ export function _schedulePostAutoCompactionContinuationProbe(this: AgentSession, return; } - if (willRetry) { - void this._resumeAfterAutoCompaction(); - return; - } - if (!this.agent.hasQueuedMessages()) { return; } @@ -206,8 +295,8 @@ export function _schedulePostAutoCompactionContinuationProbe(this: AgentSession, }, 100); } -export async function _awaitPendingOverflowPostCompactionContinuation(this: AgentSession): Promise { - const pending = this._pendingOverflowPostCompactionContinuation; +export async function _awaitPendingPostCompactionContinuation(this: AgentSession): Promise { + const pending = this._pendingPostCompactionContinuation; if (pending === undefined) return; await pending; } @@ -332,7 +421,7 @@ export async function _runAutoCompaction(this: AgentSession, reason: "overflow" */ export const agentSessionAutoCompactionMethods = { - _awaitPendingOverflowPostCompactionContinuation, + _awaitPendingPostCompactionContinuation, _checkCompaction, _isCopilotServerCapBelowSelectedContextWindow, _dropTrailingAutoCompactionRetryAssistantIfPresent, diff --git a/packages/coding-agent/src/core/agent-session-events.ts b/packages/coding-agent/src/core/agent-session-events.ts index 8ffc7f27d..c1e98554f 100644 --- a/packages/coding-agent/src/core/agent-session-events.ts +++ b/packages/coding-agent/src/core/agent-session-events.ts @@ -151,6 +151,7 @@ export async function _processAgentEvent(this: AgentSession, event: AgentEvent): if (!assistantFailed) { this._fallbackAttemptedKeys.clear(); this._overflowRecoveryAttempted = false; + this._outputBudgetErrorContinuationAttempts = 0; } // A non-truncated assistant response means the length-continuation loop diff --git a/packages/coding-agent/src/core/agent-session-methods.ts b/packages/coding-agent/src/core/agent-session-methods.ts index 013d7337d..74150fd26 100644 --- a/packages/coding-agent/src/core/agent-session-methods.ts +++ b/packages/coding-agent/src/core/agent-session-methods.ts @@ -201,7 +201,7 @@ export interface AgentSessionMethodSurface { _isCopilotServerCapBelowSelectedContextWindow(assistantMessage: AssistantMessage): boolean; _dropTrailingAutoCompactionRetryAssistantIfPresent(): void; _schedulePostAutoCompactionContinuationProbe(reason: "overflow" | "threshold", willRetry: boolean): void; - _awaitPendingOverflowPostCompactionContinuation(): Promise; + _awaitPendingPostCompactionContinuation(): Promise; _resumeAfterAutoCompaction(): Promise; _resumeAfterLengthTruncation(): void; _runAutoCompaction(reason: "overflow" | "threshold", willRetry: boolean): Promise; @@ -341,9 +341,10 @@ export interface AgentSessionInternalSurface extends AgentSessionMethodSurface, _steeringMessages: string[]; _followUpMessages: string[]; _interruptDeliveryQueue: Promise; - _pendingOverflowPostCompactionContinuation: Promise | undefined; - _overflowPostCompactionContinuationToken: number; + _pendingPostCompactionContinuation: Promise | undefined; + _postCompactionContinuationToken: number; _lengthContinuationAttempts: number; + _outputBudgetErrorContinuationAttempts: number; _pendingInterruptDeliveries: number; _activeInterruptQueueHold: InterruptQueueHold | undefined; _activeInterruptAbortMessage: string | undefined; diff --git a/packages/coding-agent/src/core/agent-session-prompt.ts b/packages/coding-agent/src/core/agent-session-prompt.ts index fa6939bea..6470a8126 100644 --- a/packages/coding-agent/src/core/agent-session-prompt.ts +++ b/packages/coding-agent/src/core/agent-session-prompt.ts @@ -186,7 +186,7 @@ export async function _runAgentPrompt(this: AgentSession, messages: AgentMessage await this.agent.prompt(messages); await this.waitForRetry(); await this._continueQueuedAgentMessages(); - await this._awaitPendingOverflowPostCompactionContinuation(); + await this._awaitPendingPostCompactionContinuation(); } finally { this._systemPromptOverride = undefined; } diff --git a/packages/coding-agent/src/core/agent-session.ts b/packages/coding-agent/src/core/agent-session.ts index 0a127805a..b85ceb39a 100644 --- a/packages/coding-agent/src/core/agent-session.ts +++ b/packages/coding-agent/src/core/agent-session.ts @@ -86,9 +86,10 @@ export class AgentSession { protected _compactionAbortController: AbortController | undefined = undefined; protected _autoCompactionAbortController: AbortController | undefined = undefined; protected _overflowRecoveryAttempted = false; - protected _pendingOverflowPostCompactionContinuation: Promise | undefined = undefined; - protected _overflowPostCompactionContinuationToken = 0; + protected _pendingPostCompactionContinuation: Promise | undefined = undefined; + protected _postCompactionContinuationToken = 0; protected _lengthContinuationAttempts = 0; + protected _outputBudgetErrorContinuationAttempts = 0; protected _branchSummaryAbortController: AbortController | undefined = undefined; protected _retryAbortController: AbortController | undefined = undefined; protected _retryAttempt = 0; diff --git a/packages/coding-agent/src/core/openai-responses-payload-sanitizer.ts b/packages/coding-agent/src/core/openai-responses-payload-sanitizer.ts index 89348ace1..efe777a54 100644 --- a/packages/coding-agent/src/core/openai-responses-payload-sanitizer.ts +++ b/packages/coding-agent/src/core/openai-responses-payload-sanitizer.ts @@ -7,6 +7,7 @@ type JsonValue = string | number | boolean | null | JsonValue[] | JsonObject; const RESPONSES_FUNCTION_CALL_ID = /^fc_[A-Za-z0-9_-]{1,61}$/; const RESPONSES_FUNCTION_CALL_ID_PREFIX = "fc_"; const MAX_RESPONSES_FUNCTION_CALL_ID_LENGTH = 64; +export const MIN_RESPONSES_MAX_OUTPUT_TOKENS = 16; function isPlainObject(value: unknown): value is JsonObject { return typeof value === "object" && value !== null && !Array.isArray(value); @@ -54,15 +55,34 @@ function sanitizeResponsesFunctionCall(item: JsonObject): boolean { } export function sanitizeOpenAIResponsesPayload(payload: unknown, model: Pick, "api">): unknown { - if (!isOpenAIResponsesModel(model) || !isPlainObject(payload) || !Array.isArray(payload.input)) return payload; + if (!isOpenAIResponsesModel(model) || !isPlainObject(payload)) return payload; let changed = false; - const input = payload.input.map((item) => { - if (!isPlainObject(item)) return item; - const cloned = { ...item }; - changed = sanitizeResponsesFunctionCall(cloned) || changed; - return cloned; - }); - - return changed ? { ...payload, input } : payload; + let sanitizedPayload: JsonObject = payload; + + if ( + typeof payload.max_output_tokens === "number" && + Number.isFinite(payload.max_output_tokens) && + payload.max_output_tokens < MIN_RESPONSES_MAX_OUTPUT_TOKENS + ) { + changed = true; + sanitizedPayload = { ...sanitizedPayload, max_output_tokens: MIN_RESPONSES_MAX_OUTPUT_TOKENS }; + } + + if (Array.isArray(payload.input)) { + let inputChanged = false; + const input = payload.input.map((item) => { + if (!isPlainObject(item)) return item; + const cloned = { ...item }; + inputChanged = sanitizeResponsesFunctionCall(cloned) || inputChanged; + return cloned; + }); + + if (inputChanged) { + changed = true; + sanitizedPayload = { ...sanitizedPayload, input }; + } + } + + return changed ? sanitizedPayload : payload; } diff --git a/packages/coding-agent/test/agent-session-auto-compaction-queue-03.suite.ts b/packages/coding-agent/test/agent-session-auto-compaction-queue-03.suite.ts index 761f979ce..e1af55f04 100644 --- a/packages/coding-agent/test/agent-session-auto-compaction-queue-03.suite.ts +++ b/packages/coding-agent/test/agent-session-auto-compaction-queue-03.suite.ts @@ -5,7 +5,10 @@ import { Agent, type AgentMessage } from "@earendil-works/pi-agent-core"; import { type AssistantMessage, getModel } from "@earendil-works/pi-ai/compat"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { AgentSession } from "../src/core/agent-session.ts"; -import { MAX_LENGTH_CONTINUATION_ATTEMPTS } from "../src/core/agent-session-auto-compaction.ts"; +import { + MAX_LENGTH_CONTINUATION_ATTEMPTS, + MAX_OUTPUT_BUDGET_ERROR_CONTINUATION_ATTEMPTS, +} from "../src/core/agent-session-auto-compaction.ts"; import { AuthStorage } from "../src/core/auth-storage.ts"; import { ModelRegistry } from "../src/core/model-registry.ts"; import { SessionManager } from "../src/core/session-manager.ts"; @@ -29,6 +32,7 @@ const compactionMocks = vi.hoisted(() => ({ protectedEntryIds: [], stats: createContextCompactionStats(190_000, 120_000), })), + estimateContextTokens: vi.fn(() => ({ tokens: 0, usageTokens: 0, trailingTokens: 0, lastUsageIndex: null })), })); vi.mock("../src/core/compaction/index.js", () => ({ @@ -37,7 +41,7 @@ vi.mock("../src/core/compaction/index.js", () => ({ collectEntriesForBranchSummary: () => ({ entries: [], commonAncestorId: null }), compact: async () => ({ summary: "compacted", firstKeptEntryId: "entry-1", tokensBefore: 100, details: {} }), contextCompact: compactionMocks.contextCompact, - estimateContextTokens: () => ({ tokens: 0, usageTokens: 0, trailingTokens: 0, lastUsageIndex: null }), + estimateContextTokens: compactionMocks.estimateContextTokens, generateBranchSummary: async () => ({ summary: "", aborted: false, readFiles: [], modifiedFiles: [] }), prepareContextCompaction: () => ({ dummy: true }), shouldCompact: (contextTokens: number, contextWindow: number, settings: { enabled: boolean; reserveTokens: number }) => @@ -51,6 +55,8 @@ describe("AgentSession auto-compaction length-stop resume", () => { beforeEach(() => { compactionMocks.contextCompact.mockClear(); + compactionMocks.estimateContextTokens.mockReset(); + compactionMocks.estimateContextTokens.mockReturnValue({ tokens: 0, usageTokens: 0, trailingTokens: 0, lastUsageIndex: null }); tempDir = join(tmpdir(), `pi-auto-compaction-length-${Date.now()}`); mkdirSync(tempDir, { recursive: true }); vi.useFakeTimers(); @@ -114,6 +120,53 @@ describe("AgentSession auto-compaction length-stop resume", () => { return assistant; } + function previousHighUsageAssistant(): AssistantMessage { + const model = session.model!; + return { + role: "assistant", + content: [{ type: "text", text: "previous complete response" }], + api: model.api, + provider: model.provider, + model: model.id, + usage: { + input: 180_000, + output: 10_000, + cacheRead: 0, + cacheWrite: 0, + totalTokens: 190_000, + cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 }, + }, + stopReason: "stop", + timestamp: Date.now() - 500, + }; + } + + function outputBudgetErrorAssistant( + errorMessage?: string, + api: AssistantMessage["api"] = "openai-responses", + ): AssistantMessage { + return { + role: "assistant", + content: [], + api, + provider: "github-copilot", + model: "gpt-5.5", + usage: { + input: 0, + output: 0, + cacheRead: 0, + cacheWrite: 0, + totalTokens: 0, + cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 }, + }, + stopReason: "error", + timestamp: Date.now(), + errorMessage: + errorMessage ?? + `OpenAI API error (400): {"message":"Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 1 instead.","code":"invalid_request_body"}`, + }; + } + it("compacts and retries threshold-sized length-stopped responses", async () => { const assistant = lengthStoppedAssistant(); session.agent.state.messages = [ @@ -148,6 +201,173 @@ describe("AgentSession auto-compaction length-stop resume", () => { expect(runAutoCompactionSpy).toHaveBeenCalledWith("threshold", false); }); + it("compacts and retries the reported OpenAI Responses output-budget underflow shape", async () => { + const previousAssistant = previousHighUsageAssistant(); + const assistant = outputBudgetErrorAssistant(); + session.agent.state.messages = [ + { role: "user", content: [{ type: "text", text: "continue the task" }], timestamp: Date.now() - 1000 }, + previousAssistant, + assistant, + ]; + compactionMocks.estimateContextTokens.mockReturnValue({ + tokens: 190_000, + usageTokens: 190_000, + trailingTokens: 0, + lastUsageIndex: 1, + }); + const runAutoCompactionSpy = vi + .spyOn(session as unknown as { _runAutoCompaction: (reason: "overflow" | "threshold", willRetry: boolean) => Promise }, "_runAutoCompaction") + .mockResolvedValue(); + const checkCompaction = (session as unknown as { _checkCompaction: (message: AssistantMessage) => Promise })._checkCompaction.bind(session); + + await checkCompaction(assistant); + + expect(runAutoCompactionSpy).toHaveBeenCalledWith("threshold", true); + }); + + it("stops retrying consecutive output-budget underflows after a compact-and-retry attempt", async () => { + const previousAssistant = previousHighUsageAssistant(); + const assistant = outputBudgetErrorAssistant(); + session.agent.state.messages = [ + { role: "user", content: [{ type: "text", text: "continue the task" }], timestamp: Date.now() - 1000 }, + previousAssistant, + assistant, + ]; + compactionMocks.estimateContextTokens.mockReturnValue({ + tokens: 190_000, + usageTokens: 190_000, + trailingTokens: 0, + lastUsageIndex: 1, + }); + (session as unknown as { _outputBudgetErrorContinuationAttempts: number })._outputBudgetErrorContinuationAttempts = + MAX_OUTPUT_BUDGET_ERROR_CONTINUATION_ATTEMPTS; + const runAutoCompactionSpy = vi + .spyOn(session as unknown as { _runAutoCompaction: (reason: "overflow" | "threshold", willRetry: boolean) => Promise }, "_runAutoCompaction") + .mockResolvedValue(); + const emitted: Array<{ type: string; reason?: string; willRetry?: boolean; errorMessage?: string }> = []; + vi.spyOn(session as unknown as { _emit: (event: { type: string; reason?: string; willRetry?: boolean; errorMessage?: string }) => void }, "_emit").mockImplementation((event) => { + emitted.push(event); + }); + const checkCompaction = (session as unknown as { _checkCompaction: (message: AssistantMessage) => Promise })._checkCompaction.bind(session); + + await checkCompaction(assistant); + + expect(runAutoCompactionSpy).not.toHaveBeenCalled(); + expect(emitted).toContainEqual( + expect.objectContaining({ + type: "compaction_end", + reason: "threshold", + willRetry: false, + errorMessage: expect.stringContaining("Output-budget recovery stopped"), + }), + ); + }); + + it("compacts and retries structured OpenAI Responses output-budget underflow errors", async () => { + const previousAssistant = previousHighUsageAssistant(); + const assistant = outputBudgetErrorAssistant( + `OpenAI API error (400): {"error":{"message":"Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 1 instead.","param":"max_output_tokens","code":"invalid_request_error"}}`, + ); + session.agent.state.messages = [ + { role: "user", content: [{ type: "text", text: "continue the task" }], timestamp: Date.now() - 1000 }, + previousAssistant, + assistant, + ]; + compactionMocks.estimateContextTokens.mockReturnValue({ + tokens: 190_000, + usageTokens: 190_000, + trailingTokens: 0, + lastUsageIndex: 1, + }); + const runAutoCompactionSpy = vi + .spyOn(session as unknown as { _runAutoCompaction: (reason: "overflow" | "threshold", willRetry: boolean) => Promise }, "_runAutoCompaction") + .mockResolvedValue(); + const checkCompaction = (session as unknown as { _checkCompaction: (message: AssistantMessage) => Promise })._checkCompaction.bind(session); + + await checkCompaction(assistant); + + expect(runAutoCompactionSpy).toHaveBeenCalledWith("threshold", true); + }); + + it("does not retry non-Responses output-budget-like errors", async () => { + const previousAssistant = previousHighUsageAssistant(); + const assistant = outputBudgetErrorAssistant(undefined, "openai-completions"); + session.agent.state.messages = [ + { role: "user", content: [{ type: "text", text: "continue the task" }], timestamp: Date.now() - 1000 }, + previousAssistant, + assistant, + ]; + compactionMocks.estimateContextTokens.mockReturnValue({ + tokens: 190_000, + usageTokens: 190_000, + trailingTokens: 0, + lastUsageIndex: 1, + }); + const runAutoCompactionSpy = vi + .spyOn(session as unknown as { _runAutoCompaction: (reason: "overflow" | "threshold", willRetry: boolean) => Promise }, "_runAutoCompaction") + .mockResolvedValue(); + const checkCompaction = (session as unknown as { _checkCompaction: (message: AssistantMessage) => Promise })._checkCompaction.bind(session); + + await checkCompaction(assistant); + + expect(runAutoCompactionSpy).toHaveBeenCalledWith("threshold", false); + }); + + it("does not retry generic invalid request errors after threshold compaction", async () => { + const previousAssistant = previousHighUsageAssistant(); + const assistant = outputBudgetErrorAssistant( + `OpenAI API error (400): {"message":"Invalid schema for function 'bash': invalid request body","code":"invalid_request_body"}`, + ); + session.agent.state.messages = [ + { role: "user", content: [{ type: "text", text: "continue the task" }], timestamp: Date.now() - 1000 }, + previousAssistant, + assistant, + ]; + compactionMocks.estimateContextTokens.mockReturnValue({ + tokens: 190_000, + usageTokens: 190_000, + trailingTokens: 0, + lastUsageIndex: 1, + }); + const runAutoCompactionSpy = vi + .spyOn(session as unknown as { _runAutoCompaction: (reason: "overflow" | "threshold", willRetry: boolean) => Promise }, "_runAutoCompaction") + .mockResolvedValue(); + const checkCompaction = (session as unknown as { _checkCompaction: (message: AssistantMessage) => Promise })._checkCompaction.bind(session); + + await checkCompaction(assistant); + + expect(runAutoCompactionSpy).toHaveBeenCalledWith("threshold", false); + }); + + it("auto-continues after threshold compaction of an output-budget error", async () => { + const userMessage: AgentMessage = { + role: "user", + content: [{ type: "text", text: "finish the task" }], + timestamp: Date.now() - 1000, + }; + const assistant = outputBudgetErrorAssistant(); + sessionManager.appendMessage(userMessage); + sessionManager.appendMessage(assistant); + session.agent.state.messages = [userMessage, assistant]; + const continueSpy = vi.spyOn(session.agent, "continue").mockImplementation(async () => { + expect(session.agent.state.messages.at(-1)?.role).toBe("user"); + }); + const waitSpy = vi.spyOn(session, "waitForRetry").mockResolvedValue(); + const drainSpy = vi + .spyOn(session as unknown as { _continueQueuedAgentMessages: () => Promise }, "_continueQueuedAgentMessages") + .mockResolvedValue(); + const runAutoCompaction = ( + session as unknown as { _runAutoCompaction: (reason: "overflow" | "threshold", willRetry: boolean) => Promise } + )._runAutoCompaction.bind(session); + + await runAutoCompaction("threshold", true); + await vi.advanceTimersByTimeAsync(100); + + expect(continueSpy).toHaveBeenCalledTimes(1); + expect(waitSpy).toHaveBeenCalledTimes(1); + expect(drainSpy).toHaveBeenCalledTimes(1); + }); + it("auto-continues after threshold compaction of a length-stopped response", async () => { const userMessage: AgentMessage = { role: "user", diff --git a/packages/coding-agent/test/first-run-onboarding.test.ts b/packages/coding-agent/test/first-run-onboarding.test.ts index de28ec520..2855c06ed 100644 --- a/packages/coding-agent/test/first-run-onboarding.test.ts +++ b/packages/coding-agent/test/first-run-onboarding.test.ts @@ -334,6 +334,7 @@ describe("first-run onboarding", () => { }, ui: { requestRender: vi.fn() }, handleFatalRuntimeError: vi.fn(), + ensureDeferredStartupComplete: vi.fn(async () => {}), }; const clear = Reflect.get(InteractiveMode.prototype, "clearFirstRunOnboardingUi") as (this: typeof host) => void; const handleClearCommand = Reflect.get(InteractiveMode.prototype, "handleClearCommand") as (this: typeof host & { clearFirstRunOnboardingUi: () => void }) => Promise; @@ -342,6 +343,7 @@ describe("first-run onboarding", () => { await handleClearCommand.call(hostWithClear); + expect(host.ensureDeferredStartupComplete).toHaveBeenCalledTimes(1); expect(host.firstRunNoticeVisible).toBe(false); expect(host.firstRunOnboardingNoticeComponents).toEqual([]); expect(host.chatContainer.children).not.toContain(staleNotice); diff --git a/packages/coding-agent/test/interactive-mode-clone-command.test.ts b/packages/coding-agent/test/interactive-mode-clone-command.test.ts index fea69c5d3..444cc3f3d 100644 --- a/packages/coding-agent/test/interactive-mode-clone-command.test.ts +++ b/packages/coding-agent/test/interactive-mode-clone-command.test.ts @@ -11,6 +11,7 @@ type CloneCommandContext = { showStatus: (message: string) => void; showError: (message: string) => void; ui: { requestRender: () => void }; + ensureDeferredStartupComplete: () => Promise; }; type InteractiveModePrototype = { @@ -27,6 +28,7 @@ describe("InteractiveMode /clone", () => { const showStatus = vi.fn(); const showError = vi.fn(); const requestRender = vi.fn(); + const ensureDeferredStartupComplete = vi.fn(async () => {}); const context: CloneCommandContext = { sessionManager: { getLeafId: () => "leaf-123" }, @@ -36,10 +38,12 @@ describe("InteractiveMode /clone", () => { showStatus, showError, ui: { requestRender }, + ensureDeferredStartupComplete, }; await interactiveModePrototype.handleCloneCommand.call(context); + expect(ensureDeferredStartupComplete).toHaveBeenCalledTimes(1); expect(fork).toHaveBeenCalledWith("leaf-123", { position: "at" }); expect(renderCurrentSessionState).toHaveBeenCalled(); expect(setText).toHaveBeenCalledWith(""); @@ -52,6 +56,7 @@ describe("InteractiveMode /clone", () => { const fork = vi.fn(async () => ({ cancelled: false })); const showStatus = vi.fn(); const showError = vi.fn(); + const ensureDeferredStartupComplete = vi.fn(async () => {}); const context: CloneCommandContext = { sessionManager: { getLeafId: () => null }, @@ -61,10 +66,12 @@ describe("InteractiveMode /clone", () => { showStatus, showError, ui: { requestRender: vi.fn() }, + ensureDeferredStartupComplete, }; await interactiveModePrototype.handleCloneCommand.call(context); + expect(ensureDeferredStartupComplete).toHaveBeenCalledTimes(1); expect(fork).not.toHaveBeenCalled(); expect(showStatus).toHaveBeenCalledWith("Nothing to clone yet"); expect(showError).not.toHaveBeenCalled(); diff --git a/packages/coding-agent/test/openai-responses-payload-sanitizer.test.ts b/packages/coding-agent/test/openai-responses-payload-sanitizer.test.ts index e0c8fc4f6..5f9fcfbc7 100644 --- a/packages/coding-agent/test/openai-responses-payload-sanitizer.test.ts +++ b/packages/coding-agent/test/openai-responses-payload-sanitizer.test.ts @@ -1,6 +1,7 @@ import assert from "node:assert/strict"; import { describe, test } from "vitest"; import { + MIN_RESPONSES_MAX_OUTPUT_TOKENS, isValidResponsesFunctionCallId, responsesFunctionCallIdForCallId, sanitizeOpenAIResponsesPayload, @@ -70,6 +71,31 @@ describe("sanitizeOpenAIResponsesPayload", () => { assert.equal("id" in input[0]!, false); }); + test("raises Responses max_output_tokens below the provider minimum", () => { + const payload = { max_output_tokens: 1, input: [{ type: "message", content: "hi" }] }; + + const sanitized = sanitizeOpenAIResponsesPayload(payload, responsesModel); + + assert.notEqual(sanitized, payload); + assert.equal((sanitized as { max_output_tokens?: number }).max_output_tokens, MIN_RESPONSES_MAX_OUTPUT_TOKENS); + }); + + test("preserves valid Responses max_output_tokens", () => { + const payload = { max_output_tokens: MIN_RESPONSES_MAX_OUTPUT_TOKENS, input: [{ type: "message", content: "hi" }] }; + + const sanitized = sanitizeOpenAIResponsesPayload(payload, responsesModel); + + assert.equal(sanitized, payload); + }); + + test("raises Responses max_output_tokens even when input is absent", () => { + const payload = { max_output_tokens: 1 }; + + const sanitized = sanitizeOpenAIResponsesPayload(payload, responsesModel); + + assert.equal((sanitized as { max_output_tokens?: number }).max_output_tokens, MIN_RESPONSES_MAX_OUTPUT_TOKENS); + }); + test("does not change non-Responses payloads", () => { const payload = { input: [{ type: "function_call", id: "raw/invalid", call_id: "call_1" }] };