diff --git a/.changeset/fix-infinite-retry-stream-invalidation.md b/.changeset/fix-infinite-retry-stream-invalidation.md new file mode 100644 index 0000000000..2cb50049cb --- /dev/null +++ b/.changeset/fix-infinite-retry-stream-invalidation.md @@ -0,0 +1,5 @@ +--- +"@moonshot-ai/kimi-code": patch +--- + +Fix a crash that killed the process when a retried LLM request had streamed a partial tool call before disconnecting. diff --git a/packages/agent-core-v2/docs/en/llm.md b/packages/agent-core-v2/docs/en/llm.md index 41f9aad47a..8613f0aff7 100644 --- a/packages/agent-core-v2/docs/en/llm.md +++ b/packages/agent-core-v2/docs/en/llm.md @@ -5,10 +5,10 @@ llm is a standalone LLM request library inside the human layer (`src/human/llm/` ## Design Principles 1. **Minimal boundary: llm = "a single request"**. llm only handles request encoding/decoding and event emission. auth, usage accounting, HistoryMessage/meta, compaction, switch, the media file system, and Tool Message assembly are all out of scope — they either move up to the turn/agent layer or plug in as contribution points. -2. **Streaming-native; events are the contract**. The only outward surface is a single, purely serializable event stream (requester level: `llm.sent / streaming.headers / streaming.part / streaming.usage / streaming.finish / streaming.message_id / failed.syntax / failed.remote / done`; the turn level adds `llm.retrying / llm.recovering`, and `llm.sent` carries the most recent recovery record). Streaming and non-streaming are isomorphic (non-streaming also accumulates over the stream, just without deltas). Events are emitted as they arrive — no caching, no fallback. +2. **Streaming-native; events are the contract**. The only outward surface is a single, purely serializable event stream (requester level: `llm.sent / streaming.headers / streaming.part / streaming.usage / streaming.finish / streaming.message_id / failed.syntax / failed.remote / done`, plus `llm.request.retrying` when the caller retries an attempt below the turn and the attempt's streamed state must be discarded; the turn level adds `llm.retrying / llm.recovering`, and `llm.sent` carries the most recent recovery record). Streaming and non-streaming are isomorphic (non-streaming also accumulates over the stream, just without deltas). Events are emitted as they arrive — no caching, no fallback. 3. **format masks inter-protocol differences; traits express provider customizations**. format lives at the protocol layer and handles encoding/decoding of requests, responses, errors, usage, and finish. Each protocol owns a typed trait interface (`OpenAITrait` / `OpenAIResponsesTrait` / `AnthropicTrait` / `GoogleGenAITrait`) exposing only the customization points that protocol actually consumes — a hook a protocol ignores is unrepresentable, never silently dead. format and trait never import each other: both speak only the neutral wire/chunk types in the protocol's `contract.ts`. The requester is the composition root — `generate` runs a fixed per-protocol pipeline (`planOpenAIRequest` and friends) that alternates pure format stages (lower → assemble → encode → stream parser) with trait hooks (cacheKey/thinking → convertMessage → mergeHistory → convertTool → buildParams → extractUsage), so customization is explicit data flow instead of a closure captured inside format. Endpoint/env resolution and default headers form the provider `connection`, error classification is a requester option, and model capability is a provider-variant field — none of them are format business. Each base's public seam is contract + trait + requester; format, lower, and patterns are internal to the requester pipeline — only bases code and tests may import them (lint-enforced). Protocol differences must not leak into the turn or into requester decorators. 4. **Two-layer error model**. Internally, code throws the SDK's native errors; local request validation throws the shared `SyntaxRequestFormatError` (`llm/syntax-errors.ts`), which the requester converts uniformly via `toLlmSyntaxErrorMessage`, with no intermediate layer. Externally there are only `llm.failed.syntax` (local message syntax errors, never retried) and `llm.failed.remote` (remote streaming errors, subdivided into connection / timeout / rate_limit / quota_exhausted / context_overflow / request_structure, etc.), converted by format at the boundary. -5. **Stateless core + turn-driven orchestration**. `generate(config, content, control)` is a stateless function; errors are delivered via onEvent, never thrown. The turn machine invokes the request actor (`createRequestActor`) directly: the actor wraps a single request (messageResolvers, abort scope, event sendBack), and the turn drives retry and recovery through the pure policy functions in retry.ts / recovery.ts: recovery is a strategy chain composed by the caller (the engine tries `credentialsRecovery` before the configured replacement-message strategies such as media degradation); each strategy's pure `propose` returns a self-describing record (`strategy`/`action`, optional replacement `messages`, optional opaque `prepare` effect) — the turn runs `prepare` and/or swaps messages and re-enters `thinking` with attempt reset to 1; retry backs off in the `retrying` state (honoring Retry-After), and the turn emits `llm.recovering / llm.retrying` for each. Empty response is judged by the turn at `llm.done` via the pure `emptyResponseError` and re-raised as `llm.failed.remote`, entering the same failure cascade. Abort is carried by an AbortController owned by the turn: the controller is passed into the request actor via `LlmInput.signal`, and the turn aborts it directly on `turn.abort`, with the request ending as `llm.failed.remote`; the request actor neither creates its own controller nor touches any signal on teardown, so a finished request can never abort a shared signal. The accumulator is held by the turn and fed by the event stream; on `llm.retrying / llm.recovering` the turn rolls it back and recreates it, so every attempt accumulates from zero while as much interrupted state as possible is preserved (the turn finishes the complete message out of the accumulator at `llm.done`). +5. **Stateless core + turn-driven orchestration**. `generate(config, content, control)` is a stateless function; errors are delivered via onEvent, never thrown. The turn machine invokes the request actor (`createRequestActor`) directly: the actor wraps a single request (messageResolvers, abort scope, event sendBack), and the turn drives retry and recovery through the pure policy functions in retry.ts / recovery.ts: recovery is a strategy chain composed by the caller (the engine tries `credentialsRecovery` before the configured replacement-message strategies such as media degradation); each strategy's pure `propose` returns a self-describing record (`strategy`/`action`, optional replacement `messages`, optional opaque `prepare` effect) — the turn runs `prepare` and/or swaps messages and re-enters `thinking` with attempt reset to 1; retry backs off in the `retrying` state (honoring Retry-After), and the turn emits `llm.recovering / llm.retrying` for each. Empty response is judged by the turn at `llm.done` via the pure `emptyResponseError` and re-raised as `llm.failed.remote`, entering the same failure cascade. Abort is carried by an AbortController owned by the turn: the controller is passed into the request actor via `LlmInput.signal`, and the turn aborts it directly on `turn.abort`, with the request ending as `llm.failed.remote`; the request actor neither creates its own controller nor touches any signal on teardown, so a finished request can never abort a shared signal. The accumulator is held by the turn and fed by the event stream; on `llm.retrying / llm.recovering / llm.request.retrying` the turn rolls it back and recreates it, so every attempt accumulates from zero while as much interrupted state as possible is preserved (the turn finishes the complete message out of the accumulator at `llm.done`). Parts already forwarded to the parent machine or UI by the interrupted attempt are not reclaimed; only the accumulator and the tool call id normalizer reset. 6. **No silent fallback**. Configuration is taken exactly as given. For beta features, thinking, empty response, and similar scenarios, define explicit error conditions first, fail at request time, and guide the user to fix the configuration — never fall back silently. 7. **Every variable capability is a contribution point**. Providers, media upload/degradation, usage, traceId, and error recovery (compaction / media degradation) all plug in through extension points; the llm core contains none of these concepts. 8. **Data is data**. A model is pure, function-free data (endpoint url + model uniquely identifies a model), serializable and directly usable as generate input. The catalog is a derived `provider -> models` cache; the dependency direction only goes from models-dev into llm internals, never the reverse. @@ -60,7 +60,7 @@ llm/ └── media/ media contribution points: cache / degrade / ref / resolver / store / upload ``` -Request lifecycle: `generate` receives (config, content, control) → the caller resolves `config.credentials` into a fully-credentialed model before each attempt (the request actor on the machine path), so requests always carry fresh credentials and a credential-refresh recovery (recoverable 401 → `credentials.invalidate()`, emitted as `llm.recovering` with strategy `credentials`) naturally re-resolves on the re-send (direct callers outside the state machines — ping, generate, full compaction, media upload — share the same single-retry recovery through `runWithCredentialRecovery` / `streamWithCredentialRecovery`) → the requester's `plan*` function composes pure format stages with trait hooks into protocol requestParams (format lowers the generic Message[] through the Pattern Rewriter; trait adjusts kwargs, converted messages, history, tools, and final params in between) → internalGenerate calls the official SDK → streaming chunks are converted by the stateless parser callbacks into `llm.streaming.part / streaming.usage / streaming.finish / streaming.message_id` events → errors are converted by format into `llm.failed.*`; on success the requester emits `llm.done`, on failure it ends with `llm.failed.syntax / llm.failed.remote` and never emits `llm.done`. At `llm.done` the turn judges empty responses via `emptyResponseError` and re-raises them as `llm.failed.remote`; the turn machine first tries recovery on `llm.failed.remote` (the engine-composed strategy chain — credential refresh on a recoverable 401 first, then replacement-message strategies — each pure `propose` returning a record whose opaque `prepare` effect the turn executes, emitting `llm.recovering`), then retries with backoff (honoring Retry-After, emitting `llm.retrying`), and only fails the turn once attempts are exhausted. The turn holds the HistoryAccumulator, fed by the event stream, rolls it back and recreates it on `llm.retrying / llm.recovering`, and finishes the complete message at `llm.done`; usage accounting, tracing, compaction, and media degradation all attach to the event stream as plugins/contribution points. +Request lifecycle: `generate` receives (config, content, control) → the caller resolves `config.credentials` into a fully-credentialed model before each attempt (the request actor on the machine path), so requests always carry fresh credentials and a credential-refresh recovery (recoverable 401 → `credentials.invalidate()`, emitted as `llm.recovering` with strategy `credentials`) naturally re-resolves on the re-send (direct callers outside the state machines — ping, generate, full compaction, media upload — share the same single-retry recovery through `runWithCredentialRecovery` / `streamWithCredentialRecovery`) → the requester's `plan*` function composes pure format stages with trait hooks into protocol requestParams (format lowers the generic Message[] through the Pattern Rewriter; trait adjusts kwargs, converted messages, history, tools, and final params in between) → internalGenerate calls the official SDK → streaming chunks are converted by the stateless parser callbacks into `llm.streaming.part / streaming.usage / streaming.finish / streaming.message_id` events → errors are converted by format into `llm.failed.*`; on success the requester emits `llm.done`, on failure it ends with `llm.failed.syntax / llm.failed.remote` and never emits `llm.done`. At `llm.done` the turn judges empty responses via `emptyResponseError` and re-raises them as `llm.failed.remote`; the turn machine first tries recovery on `llm.failed.remote` (the engine-composed strategy chain — credential refresh on a recoverable 401 first, then replacement-message strategies — each pure `propose` returning a record whose opaque `prepare` effect the turn executes, emitting `llm.recovering`), then retries with backoff (honoring Retry-After, emitting `llm.retrying`), and only fails the turn once attempts are exhausted. The turn holds the HistoryAccumulator, fed by the event stream, rolls it back and recreates it on `llm.retrying / llm.recovering / llm.request.retrying`, and finishes the complete message at `llm.done`; usage accounting, tracing, compaction, and media degradation all attach to the event stream as plugins/contribution points. ## Rejected Schemes (do not reintroduce) diff --git a/packages/agent-core-v2/docs/zh/llm.md b/packages/agent-core-v2/docs/zh/llm.md index ef5c9ff0f4..5c5e40ff32 100644 --- a/packages/agent-core-v2/docs/zh/llm.md +++ b/packages/agent-core-v2/docs/zh/llm.md @@ -5,10 +5,10 @@ llm 是 human 层内一个独立的 LLM 请求库(`src/human/llm/`),提供 ## 设计原则 1. **边界极简:llm = 「一次请求」**。llm 只负责请求编解码与事件回传。auth、usage 统计、HistoryMessage/meta、compaction、switch、媒体文件系统、Tool Message 拼装全部不属于 llm——要么上移到 turn/agent 层,要么以贡献点接入。 -2. **流式原生、事件即契约**。对外只暴露一条纯可序列化的事件流(requester 层:`llm.sent / streaming.headers / streaming.part / streaming.usage / streaming.finish / streaming.message_id / failed.syntax / failed.remote / done`;turn 层补充 `llm.retrying / llm.recovering`,`llm.sent` 携带最近一次 recovery 记录),流式与非流式同构(非流式也走流式累积,只是不发 delta);事件收到即发,不缓存、不兜底。 +2. **流式原生、事件即契约**。对外只暴露一条纯可序列化的事件流(requester 层:`llm.sent / streaming.headers / streaming.part / streaming.usage / streaming.finish / streaming.message_id / failed.syntax / failed.remote / done`,另有 `llm.request.retrying` 表示调用方在 turn 之下重试本 attempt、已流出的流式状态需作废;turn 层补充 `llm.retrying / llm.recovering`,`llm.sent` 携带最近一次 recovery 记录),流式与非流式同构(非流式也走流式累积,只是不发 delta);事件收到即发,不缓存、不兜底。 3. **format 屏蔽协议间差异,trait 表达 provider 定制**。format 位于 protocol 层,负责请求、响应、错误、usage 和 finish 的编解码。每种协议拥有自己的类型化 trait 接口(`OpenAITrait` / `OpenAIResponsesTrait` / `AnthropicTrait` / `GoogleGenAITrait`),只暴露该协议实际消费的定制点——协议不支持的 hook 在类型上无法表达,而不是配了却静默无效。format 与 trait 互不 import:双方只共享协议 `contract.ts` 里的中立 wire/chunk 类型。requester 是组合根——`generate` 执行每个协议固定的流水线(`planOpenAIRequest` 等),交替调用纯 format 阶段(lower → assemble → encode → stream parser)与 trait hooks(cacheKey/thinking → convertMessage → mergeHistory → convertTool → buildParams → extractUsage),定制逻辑是显式的数据流,而不是捕获在 format 闭包里。endpoint/环境变量解析与默认 headers 属于 provider `connection`,错误归类是 requester 选项,模型能力是 provider variant 字段——都不是 format 的职责。每个 base 的公开接缝是 contract + trait + requester;format、lower、patterns 是 requester 流水线的内部模块——只有 bases 内代码和测试可以 import(lint 强制)。协议差异不允许泄漏到 turn 或 requester 的装饰层。 4. **错误两层模型**。内部 throw SDK 原生错误;本地请求校验抛共享的 `SyntaxRequestFormatError`(`llm/syntax-errors.ts`),由 requester 经 `toLlmSyntaxErrorMessage` 统一转换,不加中间层。对外只有 `llm.failed.syntax`(本地消息语法错误,不重试)与 `llm.failed.remote`(远程流式错误,细分为 connection/timeout/rate_limit/quota_exhausted/context_overflow/request_structure 等),由 format 在边界完成转换。 -5. **无状态内核 + turn 驱动的编排**。`generate(config, content, control)` 是无状态函数,错误走 onEvent 不 throw;turn machine 直接 invoke 请求 actor(`createRequestActor`):actor 包装单次请求(messageResolvers、abort 作用域、事件 sendBack),turn 借助 retry.ts / recovery.ts 的纯策略函数驱动重试与 recovery:recovery 是一条由调用方组装的策略链(engine 先尝试 `credentialsRecovery`,再尝试配置的媒体降级等替换消息策略),每个策略的纯函数 `propose` 返回自描述记录(`strategy`/`action`、可选替换 `messages`、可选不透明 `prepare` 副作用),turn 执行 `prepare` 和/或替换消息并重进 `thinking`(attempt 重置为 1),重试走 `retrying` 状态的 backoff(尊重 Retry-After),两者分别由 turn 对外补发 `llm.recovering / llm.retrying` 事件;empty response 由 turn 在 `llm.done` 时经纯函数 `emptyResponseError` 判定并重新转为 `llm.failed.remote`,进入同一失败级联;abort 由 turn 持有的 AbortController 承载:controller 经 `LlmInput.signal` 传入 request actor,turn 在 `turn.abort` 时直接 abort 它,请求随即以 `llm.failed.remote` 收尾;request actor 不自建 controller、回收时不触碰任何 signal,正常完成的请求绝不可能误 abort 共享 signal。累积器由 turn 持有并随事件流喂入,在 `llm.retrying / llm.recovering` 时 rollback 并重建,每次 attempt 从零累积,从而尽可能保留中断现场(turn 在 `llm.done` 时从累加器 finish 出完整消息)。 +5. **无状态内核 + turn 驱动的编排**。`generate(config, content, control)` 是无状态函数,错误走 onEvent 不 throw;turn machine 直接 invoke 请求 actor(`createRequestActor`):actor 包装单次请求(messageResolvers、abort 作用域、事件 sendBack),turn 借助 retry.ts / recovery.ts 的纯策略函数驱动重试与 recovery:recovery 是一条由调用方组装的策略链(engine 先尝试 `credentialsRecovery`,再尝试配置的媒体降级等替换消息策略),每个策略的纯函数 `propose` 返回自描述记录(`strategy`/`action`、可选替换 `messages`、可选不透明 `prepare` 副作用),turn 执行 `prepare` 和/或替换消息并重进 `thinking`(attempt 重置为 1),重试走 `retrying` 状态的 backoff(尊重 Retry-After),两者分别由 turn 对外补发 `llm.recovering / llm.retrying` 事件;empty response 由 turn 在 `llm.done` 时经纯函数 `emptyResponseError` 判定并重新转为 `llm.failed.remote`,进入同一失败级联;abort 由 turn 持有的 AbortController 承载:controller 经 `LlmInput.signal` 传入 request actor,turn 在 `turn.abort` 时直接 abort 它,请求随即以 `llm.failed.remote` 收尾;request actor 不自建 controller、回收时不触碰任何 signal,正常完成的请求绝不可能误 abort 共享 signal。累积器由 turn 持有并随事件流喂入,在 `llm.retrying / llm.recovering / llm.request.retrying` 时 rollback 并重建,每次 attempt 从零累积,从而尽可能保留中断现场(turn 在 `llm.done` 时从累加器 finish 出完整消息);被中断 attempt 已实时转发给父机/UI 的 part 不回收,只重置累加器与 tool call id normalizer。 6. **不兜底**。配置是什么就是什么;beta 特性、thinking、empty response 等场景先定义明确报错条件,在请求阶段报错并引导用户修正,而不是静默兜底。 7. **一切可变能力都是贡献点**。provider、媒体上传/降级、usage、traceId、错误恢复(compaction/媒体降级)都通过扩展点接入,llm 内核不含这些概念。 8. **数据即数据**。model 是无函数的纯数据(endpoint url + model 唯一标识一个模型),可序列化、可直接作为 generate 输入;catalog 是 `provider -> models` 的派生缓存,依赖方向只能从 models-dev 指向 llm 内部,不能反向依赖。 @@ -60,7 +60,7 @@ llm/ └── media/ 媒体贡献点:cache / degrade / ref / resolver / store / upload ``` -请求生命周期:`generate` 收到 (config, content, control) → 调用方在每次 attempt 前把 `config.credentials` 解析成带完整凭证的 model(machine 路径由 request actor 完成),请求因此始终携带新鲜凭证,而凭证刷新恢复(可恢复的 401 → `credentials.invalidate()`,以 `llm.recovering`(strategy 为 `credentials`)发出)在重发时自然重新解析(不经状态机的 direct 调用方——ping、generate、full compaction、媒体上传——通过 `runWithCredentialRecovery` / `streamWithCredentialRecovery` 共享同一套单次重试恢复) → requester 的 `plan*` 函数将纯 format 阶段与 trait hooks 组合为协议 requestParams(format 将通用 Message[] 经 Pattern Rewriter 降低,trait 在其间调整 kwargs、转换消息、合并历史、转换 tools 并收尾 params) → internalGenerate 调用官方 SDK → 流式 chunk 经无状态 parser 回调转换为 `llm.streaming.part / streaming.usage / streaming.finish / streaming.message_id` 事件 → 错误由 format 转换为 `llm.failed.*`;成功时 requester 发出 `llm.done`,失败时以 `llm.failed.syntax / llm.failed.remote` 收尾、不再发 `llm.done`。turn 在 `llm.done` 时经 `emptyResponseError` 判定空响应并重新转为 `llm.failed.remote`;turn machine 对 `llm.failed.remote` 先尝试恢复(由 engine 组装的策略链——可恢复 401 的凭证刷新在前、替换消息策略在后——经纯函数 `propose` 产出带不透明 `prepare` 副作用的记录,发 `llm.recovering`),再按策略 backoff 重试(尊重 Retry-After,发 `llm.retrying`),耗尽后才将 turn 置为失败。turn 持有 HistoryAccumulator 随事件流累积,在 `llm.retrying / llm.recovering` 时 rollback 并重建累加器,`llm.done` 时 finish 出完整消息;usage 统计、trace、compaction、媒体降级均以插件/贡献点身份挂接在事件流上。 +请求生命周期:`generate` 收到 (config, content, control) → 调用方在每次 attempt 前把 `config.credentials` 解析成带完整凭证的 model(machine 路径由 request actor 完成),请求因此始终携带新鲜凭证,而凭证刷新恢复(可恢复的 401 → `credentials.invalidate()`,以 `llm.recovering`(strategy 为 `credentials`)发出)在重发时自然重新解析(不经状态机的 direct 调用方——ping、generate、full compaction、媒体上传——通过 `runWithCredentialRecovery` / `streamWithCredentialRecovery` 共享同一套单次重试恢复) → requester 的 `plan*` 函数将纯 format 阶段与 trait hooks 组合为协议 requestParams(format 将通用 Message[] 经 Pattern Rewriter 降低,trait 在其间调整 kwargs、转换消息、合并历史、转换 tools 并收尾 params) → internalGenerate 调用官方 SDK → 流式 chunk 经无状态 parser 回调转换为 `llm.streaming.part / streaming.usage / streaming.finish / streaming.message_id` 事件 → 错误由 format 转换为 `llm.failed.*`;成功时 requester 发出 `llm.done`,失败时以 `llm.failed.syntax / llm.failed.remote` 收尾、不再发 `llm.done`。turn 在 `llm.done` 时经 `emptyResponseError` 判定空响应并重新转为 `llm.failed.remote`;turn machine 对 `llm.failed.remote` 先尝试恢复(由 engine 组装的策略链——可恢复 401 的凭证刷新在前、替换消息策略在后——经纯函数 `propose` 产出带不透明 `prepare` 副作用的记录,发 `llm.recovering`),再按策略 backoff 重试(尊重 Retry-After,发 `llm.retrying`),耗尽后才将 turn 置为失败。turn 持有 HistoryAccumulator 随事件流累积,在 `llm.retrying / llm.recovering / llm.request.retrying` 时 rollback 并重建累加器,`llm.done` 时 finish 出完整消息;usage 统计、trace、compaction、媒体降级均以插件/贡献点身份挂接在事件流上。 ## 已被否决的方案(不要再引入) diff --git a/packages/agent-core-v2/src/agent/llmRequester/llmRequester.ts b/packages/agent-core-v2/src/agent/llmRequester/llmRequester.ts index d7897f363e..9dac4abf04 100644 --- a/packages/agent-core-v2/src/agent/llmRequester/llmRequester.ts +++ b/packages/agent-core-v2/src/agent/llmRequester/llmRequester.ts @@ -45,6 +45,7 @@ export interface AgentLLMRequestOverrides { source?: AgentLLMRequestSource; maxOutputSize?: number; model?: string; + onAttemptRetry?: () => void; } export interface AgentLLMRequestTask { diff --git a/packages/agent-core-v2/src/agent/llmRequester/llmRequesterService.ts b/packages/agent-core-v2/src/agent/llmRequester/llmRequesterService.ts index 7419d5cde1..2c347d515b 100644 --- a/packages/agent-core-v2/src/agent/llmRequester/llmRequesterService.ts +++ b/packages/agent-core-v2/src/agent/llmRequester/llmRequesterService.ts @@ -265,7 +265,7 @@ export class AgentLLMRequesterService implements IAgentLLMRequesterService { }; setTrace(undefined); try { - return await this.runRequest(overrides, onPart, signal, setTrace); + return await this.runRequest(overrides, onPart, signal, setTrace, overrides.onAttemptRetry); } catch (error) { this.logRequestFailure(error, overrides, signal); setTrace(this.trackApiError(error, startedAt, signal, overrides.source, trace.traceId)); @@ -337,6 +337,7 @@ export class AgentLLMRequesterService implements IAgentLLMRequesterService { onPart: AgentLLMRequestPartHandler, signal: AbortSignal | undefined, onRequestTrace: (traceId: string | undefined) => void, + onAttemptRetry: (() => void) | undefined, ): Promise { let request = this.resolveRequest(overrides); this.toolCallIdNormalizer.seedFrom(this.context.get()); @@ -480,6 +481,7 @@ export class AgentLLMRequesterService implements IAgentLLMRequesterService { captureMediaStripPolicy, ); if (nextPolicy !== undefined) { + onAttemptRetry?.(); policy = nextPolicy; continue; } @@ -491,6 +493,7 @@ export class AgentLLMRequesterService implements IAgentLLMRequesterService { this.activateFallback('terminal-error') ) { request = this.resolveRequest(overrides); + onAttemptRetry?.(); policy = undefined; infiniteRetryAttempt = 0; continue; @@ -514,6 +517,7 @@ export class AgentLLMRequesterService implements IAgentLLMRequesterService { delayMs, ...retryErrorFields(error), }); + onAttemptRetry?.(); await sleepForRetry(delayMs, signal); } } diff --git a/packages/agent-core-v2/src/agent/loop/machine/requester.ts b/packages/agent-core-v2/src/agent/loop/machine/requester.ts index a5f9848b82..da374a4197 100644 --- a/packages/agent-core-v2/src/agent/loop/machine/requester.ts +++ b/packages/agent-core-v2/src/agent/loop/machine/requester.ts @@ -81,7 +81,10 @@ export function createMachineRequester( ? { ...baseSource, step: decision.step } : baseSource; const task = service.start( - { source }, + { + source, + onAttemptRetry: () => control.onEvent?.({ type: 'llm.request.retrying' }), + }, (part) => control.onEvent?.({ type: 'llm.streaming.part', part }), signal, ); diff --git a/packages/agent-core-v2/src/agent/loop/machine/tools.ts b/packages/agent-core-v2/src/agent/loop/machine/tools.ts index aafebbc29c..2e67fc91ef 100644 --- a/packages/agent-core-v2/src/agent/loop/machine/tools.ts +++ b/packages/agent-core-v2/src/agent/loop/machine/tools.ts @@ -82,9 +82,6 @@ export function createMachineTools(options: CreateMachineToolsOptions): MachineT const runBatch = async (entries: readonly PendingEntry[]): Promise => { batchInFlight = true; const inFlight = new Map(); - for (const entry of entries) inFlight.set(entry.input.toolCall.id, entry); - const signal = AbortSignal.any(entries.map((entry) => entry.input.signal)); - const calls = entries.map((entry) => entry.input.toolCall); const settleRemaining = (error?: unknown): void => { for (const entry of inFlight.values()) { settleEntry(entry, { @@ -103,6 +100,9 @@ export function createMachineTools(options: CreateMachineToolsOptions): MachineT inFlight.clear(); }; try { + for (const entry of entries) inFlight.set(entry.input.toolCall.id, entry); + const signal = AbortSignal.any(entries.map((entry) => entry.input.signal)); + const calls = entries.map((entry) => entry.input.toolCall); const stream = options.toolExecutor.execute(calls, { signal, steerSignal: options.steerSignal?.(), @@ -140,6 +140,12 @@ export function createMachineTools(options: CreateMachineToolsOptions): MachineT } }; + const startBatch = (entries: readonly PendingEntry[]): void => { + void runBatch(entries).catch((error: unknown) => { + options.onBatchError?.(error); + }); + }; + const applyResult = (entry: PendingEntry, matched: ToolExecutionResult): void => { const id = entry.input.toolCall.id; const { result } = matched; @@ -164,23 +170,38 @@ export function createMachineTools(options: CreateMachineToolsOptions): MachineT if (!expectedIds.every((id) => pending.has(id))) return; const entries: PendingEntry[] = []; for (const id of expectedIds) { - const entry = pending.get(id)!; + const entry = pending.get(id); + if (entry === undefined) continue; pending.delete(id); entries.push(entry); } if (entries.length === 0) return; - void runBatch(entries); + startBatch(entries); }; const execute = (input: ToolExecuteInput): Promise => { - progressHandlers.set(input.toolCall.id, input.onUpdate); if (expectedIds === undefined || batchInFlight) { + progressHandlers.set(input.toolCall.id, input.onUpdate); return new Promise((resolve) => { const entry: PendingEntry = { input, resolve, removeAbortListener: () => {} }; - void runBatch([entry]); + startBatch([entry]); }); } return new Promise((resolve) => { + const previous = pending.get(input.toolCall.id); + if (previous !== undefined) { + pending.delete(input.toolCall.id); + settleEntry(previous, { + content: [ + { + type: 'text', + text: `Tool "${previous.input.toolCall.name}" superseded by a duplicate tool call id.`, + }, + ], + isError: true, + }); + } + progressHandlers.set(input.toolCall.id, input.onUpdate); const onAbort = (): void => { if (!pending.delete(input.toolCall.id)) return; const stale = [...pending.values()]; @@ -217,7 +238,9 @@ export function createMachineTools(options: CreateMachineToolsOptions): MachineT for (const entry of stale) settleAborted(entry); return; } - expectedIds = expectedCalls.filter((call) => knownNames.has(call.name)).map((call) => call.id); + expectedIds = [ + ...new Set(expectedCalls.filter((call) => knownNames.has(call.name)).map((call) => call.id)), + ]; flushIfReady(); }, handleProgress: (toolCallId, update) => { diff --git a/packages/agent-core-v2/src/human/agent/turn.ts b/packages/agent-core-v2/src/human/agent/turn.ts index 43b6fc684c..5c710a697a 100644 --- a/packages/agent-core-v2/src/human/agent/turn.ts +++ b/packages/agent-core-v2/src/human/agent/turn.ts @@ -377,6 +377,13 @@ export function createTurnMachine( sendToParent: ({ self }, params: TurnLlmEvent) => { self._parent?.send(params); }, + discardAttemptStream: ({ context }) => { + context.accumulator.rollback(); + context.accumulator = createHistoryAccumulator( + modelMeta(context.input.request.model), + context.toolCallIds, + ); + }, salvageAborted: assign(({ context }) => { const partial = context.accumulator.finish({ source: 'salvaged' }); const salvaged = salvageInterruptedMessage(partial.message); @@ -468,15 +475,12 @@ export function createTurnMachine( recovery: context.appliedRecoveries.at(-1), }), }, - ({ context }) => { - context.accumulator.rollback(); - context.accumulator = createHistoryAccumulator( - modelMeta(context.input.request.model), - context.toolCallIds, - ); - }, + 'discardAttemptStream', ], }, + 'llm.request.retrying': { + actions: ['discardAttemptStream'], + }, 'llm.streaming.headers': { actions: [ 'forwardToParent', diff --git a/packages/agent-core-v2/src/human/llm/requester/requester.ts b/packages/agent-core-v2/src/human/llm/requester/requester.ts index 185e4abf9b..6b60928486 100644 --- a/packages/agent-core-v2/src/human/llm/requester/requester.ts +++ b/packages/agent-core-v2/src/human/llm/requester/requester.ts @@ -31,6 +31,7 @@ export type LlmRequestEvent = | { type: 'llm.streaming.message_id'; messageId: string } | { type: 'llm.failed.syntax'; error: LlmErrorMessage<'syntax'> } | { type: 'llm.failed.remote'; error: LlmRemoteErrorMessage; rawError?: unknown } + | { type: 'llm.request.retrying' } | { type: 'llm.done' }; export interface ExtraParams { diff --git a/packages/agent-core-v2/src/llm-adapter/model/model-requester-impl.ts b/packages/agent-core-v2/src/llm-adapter/model/model-requester-impl.ts index 2cbb94a240..a1b4978443 100644 --- a/packages/agent-core-v2/src/llm-adapter/model/model-requester-impl.ts +++ b/packages/agent-core-v2/src/llm-adapter/model/model-requester-impl.ts @@ -213,6 +213,13 @@ export class ModelRequesterImpl implements ModelRequester { failed = event.error; return; } + case 'llm.request.retrying': { + accumulator = createMessageAccumulator(); + usage = undefined; + finish = undefined; + messageId = undefined; + return; + } case 'llm.done': { streamEndedAt = Date.now(); if (firstChunkAt !== undefined) { diff --git a/packages/agent-core-v2/test/agent/llmRequester/llmRequesterService.test.ts b/packages/agent-core-v2/test/agent/llmRequester/llmRequesterService.test.ts index 21bbe5fe7f..e254950c90 100644 --- a/packages/agent-core-v2/test/agent/llmRequester/llmRequesterService.test.ts +++ b/packages/agent-core-v2/test/agent/llmRequester/llmRequesterService.test.ts @@ -15,10 +15,22 @@ import { import { AgentContextProjectorService } from '#/agent/contextProjector/contextProjectorService'; import { AgentLLMRequesterService, KIMI_CODE_INFINITE_RETRY_ENV } from '#/agent/llmRequester/llmRequesterService'; import { IAgentLLMRequesterService } from '#/agent/llmRequester/llmRequester'; +import { createMachineRequester } from '#/agent/loop/machine/requester'; import { IBootstrapService } from '#/app/bootstrap/bootstrap'; import { FALLBACK_MODEL_SECTION } from '#/app/kosongConfig/configSection'; import { IFlagService } from '#/app/flag/flag'; import { FALLBACK_MODEL_FLAG_ID } from '#/session/fallback/flag'; +import { + createTurnMachine, + type AssistantEntry, + type TurnEvent, + type TurnInput, + type TurnLlmEvent, +} from '#human/agent/turn'; +import { UNKNOWN_CAPABILITY } from '#human/llm/capability'; +import type { LlmModel } from '#human/llm/model'; +import type { LlmRequester } from '#human/llm/requester/requester'; +import { createActor, emit, setup } from '#human/xstate2'; import { ISessionTokenCountingService } from '#/session/tokenCounting/sessionTokenCounting'; import { IAgentProfileService } from '#/agent/profile/profile'; import { IAgentStateService } from '#/agent/state/agentState'; @@ -65,6 +77,43 @@ import { registerTestEventDispatcher, } from '../../wire/stubs'; +const turnHarnessModel: LlmModel = { + provider: 'test', + model: 'test-model', + capability: UNKNOWN_CAPABILITY, +}; + +function createTurnHarness(requester: LlmRequester) { + return setup({ + types: { + input: {} as TurnInput, + context: {} as { turnInput: TurnInput }, + events: {} as TurnEvent, + emitted: {} as TurnLlmEvent, + }, + actors: { turn: createTurnMachine(requester) }, + }).createMachine({ + id: 'turn-harness', + initial: 'running', + context: ({ input }) => ({ turnInput: input }), + states: { + running: { + invoke: { + src: 'turn', + input: ({ context }) => context.turnInput, + onDone: { target: 'completed' }, + }, + on: { + '*': { + actions: emit(({ event }) => event as TurnLlmEvent), + }, + }, + }, + completed: { type: 'final' }, + }, + }); +} + const capabilities: ModelCapability = { image_in: false, video_in: false, @@ -1001,54 +1050,54 @@ describe('AgentLLMRequesterService media resolver wiring', () => { }); }); -describe('AgentLLMRequesterService tool call id normalization', () => { - function createScriptedRequester( - script: { ids: string[]; error?: Error }[], - ): ModelRequester { - const base = createRequester({ value: 0 }); - let callIndex = 0; - return { - model: base.model, - request: async function* () { - const step = script[Math.min(callIndex++, script.length - 1)]!; - if (step.error !== undefined) { - if (step.ids.length > 0) { - yield { - type: 'part', - part: { - type: 'function', - id: step.ids[0]!, - name: 'Bash', - arguments: null, - _streamIndex: 0, - }, - } satisfies ModelRequestEvent; - } - throw step.error; - } - const toolCalls: ToolCall[] = []; - for (const [index, id] of step.ids.entries()) { - yield { - type: 'part', - part: { type: 'function', id, name: 'Bash', arguments: null, _streamIndex: index }, - } satisfies ModelRequestEvent; +function createScriptedRequester( + script: { ids: string[]; error?: Error }[], +): ModelRequester { + const base = createRequester({ value: 0 }); + let callIndex = 0; + return { + model: base.model, + request: async function* () { + const step = script[Math.min(callIndex++, script.length - 1)]!; + if (step.error !== undefined) { + if (step.ids.length > 0) { yield { type: 'part', - part: { type: 'tool_call_part', argumentsPart: '{"command":"ls"}', index }, + part: { + type: 'function', + id: step.ids[0]!, + name: 'Bash', + arguments: null, + _streamIndex: 0, + }, } satisfies ModelRequestEvent; - toolCalls.push({ type: 'function', id, name: 'Bash', arguments: '{"command":"ls"}' }); } + throw step.error; + } + const toolCalls: ToolCall[] = []; + for (const [index, id] of step.ids.entries()) { yield { - type: 'finish', - message: { role: 'assistant', content: [], toolCalls }, - providerFinishReason: 'completed', - rawFinishReason: 'stop', - id: 'resp-1', + type: 'part', + part: { type: 'function', id, name: 'Bash', arguments: null, _streamIndex: index }, } satisfies ModelRequestEvent; - }, - }; - } + yield { + type: 'part', + part: { type: 'tool_call_part', argumentsPart: '{"command":"ls"}', index }, + } satisfies ModelRequestEvent; + toolCalls.push({ type: 'function', id, name: 'Bash', arguments: '{"command":"ls"}' }); + } + yield { + type: 'finish', + message: { role: 'assistant', content: [], toolCalls }, + providerFinishReason: 'completed', + rawFinishReason: 'stop', + id: 'resp-1', + } satisfies ModelRequestEvent; + }, + }; +} +describe('AgentLLMRequesterService tool call id normalization', () => { it('passes provider-unique ids through unchanged', async () => { const parts: StreamedMessagePart[] = []; const { service } = createService( @@ -1142,12 +1191,14 @@ describe('AgentLLMRequesterService terminal-error fallback', () => { fallbackFlag: true, fallbackConfig: { model: 'fallback-alias' }, }); + const onAttemptRetry = vi.fn(); - const result = await service.request(); + const result = await service.request({ onAttemptRetry }); expect(result.message.content).toEqual([{ type: 'text', text: 'ok' }]); expect(calls.value).toBe(2); expect(result.model).toBe('fallback-alias'); + expect(onAttemptRetry).toHaveBeenCalledTimes(1); expect(events.filter((event) => event.type === 'warning')).toEqual([ expect.objectContaining({ type: 'warning', code: 'fallback-model' }), ]); @@ -1165,3 +1216,85 @@ describe('AgentLLMRequesterService terminal-error fallback', () => { expect(events.filter((event) => event.type === 'warning')).toEqual([]); }); }); + +describe('AgentLLMRequesterService attempt retry notification', () => { + afterEach(() => { + vi.useRealTimers(); + }); + + it('notifies before resending with a repaired projection', async () => { + const calls = { value: 0 }; + const { service } = createService(createRequester(calls), undefined); + const onAttemptRetry = vi.fn(); + + const result = await service.request({ onAttemptRetry }); + + expect(result.message.content).toEqual([{ type: 'text', text: 'ok' }]); + expect(calls.value).toBe(2); + expect(onAttemptRetry).toHaveBeenCalledTimes(1); + }); + + it('notifies before each indefinite-retry backoff', async () => { + vi.useFakeTimers(); + const calls = { value: 0 }; + const requester = createRequester(calls, new APIConnectionError('socket hang up'), [ + new APIConnectionError('socket hang up again'), + ]); + const { service } = createService(requester, undefined, { + env: { [KIMI_CODE_INFINITE_RETRY_ENV]: '1' }, + }); + const onAttemptRetry = vi.fn(); + + const promise = service.request({ onAttemptRetry }); + await vi.runAllTimersAsync(); + await promise; + + expect(calls.value).toBe(3); + expect(onAttemptRetry).toHaveBeenCalledTimes(2); + }); + + it('does not notify when the error is final', async () => { + const calls = { value: 0 }; + const { service } = createService( + createRequester(calls, new APIStatusError(400, 'max_tokens must be positive')), + undefined, + ); + const onAttemptRetry = vi.fn(); + + await expect(service.request({ onAttemptRetry })).rejects.toMatchObject({ statusCode: 400 }); + expect(onAttemptRetry).not.toHaveBeenCalled(); + }); +}); + +describe('turn machine stream state across service-internal retries', () => { + afterEach(() => { + vi.useRealTimers(); + }); + + it('discards the interrupted attempt stream when the service retries below the turn', async () => { + vi.useFakeTimers(); + const { service } = createService( + createScriptedRequester([ + { ids: ['call_a'], error: new APIConnectionError('terminated') }, + { ids: ['call_b'] }, + ]), + undefined, + { env: { [KIMI_CODE_INFINITE_RETRY_ENV]: '1' } }, + ); + const machineRequester = createMachineRequester(service); + const doneEntries: AssistantEntry[] = []; + const actor = createActor(createTurnHarness(machineRequester.requester), { + input: { request: { model: turnHarnessModel }, history: [] }, + }); + actor.on('llm.done', (event) => doneEntries.push(event.entry)); + actor.start(); + + await vi.runAllTimersAsync(); + for (let index = 0; index < 10; index += 1) { + await vi.advanceTimersByTimeAsync(0); + } + + expect(doneEntries).toHaveLength(1); + expect(doneEntries[0]?.message.toolCalls.map((toolCall) => toolCall.id)).toEqual(['call_b']); + }); +}); diff --git a/packages/agent-core-v2/test/agent/loop/machineTools.test.ts b/packages/agent-core-v2/test/agent/loop/machineTools.test.ts new file mode 100644 index 0000000000..7022a3ad43 --- /dev/null +++ b/packages/agent-core-v2/test/agent/loop/machineTools.test.ts @@ -0,0 +1,105 @@ +import { describe, expect, it, vi } from 'vitest'; + +import type { + IAgentToolExecutorService, + ToolExecutionResult, +} from '#/agent/toolExecutor/toolExecutor'; +import { createMachineTools } from '#/agent/loop/machine/tools'; +import type { ToolCall } from '#human/llm/message'; +import type { ToolExecuteInput } from '#human/tool/executor'; +import type { ToolInfo } from '#/tool/toolContract'; + +function call(id: string, name: string): ToolCall { + return { type: 'function', id, name, arguments: '{}' }; +} + +function input(toolCall: ToolCall): ToolExecuteInput { + return { toolCall, signal: new AbortController().signal }; +} + +function createRecordingExecutor(): { + toolExecutor: IAgentToolExecutorService; + batches: string[][]; +} { + const batches: string[][] = []; + const toolExecutor = { + execute: async function* (calls: ToolCall[]) { + batches.push(calls.map((toolCall) => toolCall.id)); + for (const toolCall of calls) { + yield { + toolCallId: toolCall.id, + toolName: toolCall.name, + result: { output: `ok:${toolCall.id}` }, + } satisfies ToolExecutionResult; + } + }, + } as unknown as IAgentToolExecutorService; + return { toolExecutor, batches }; +} + +const toolInfos: ToolInfo[] = [ + { name: 'Bash', description: 'run a command', source: 'builtin' }, + { name: 'Read', description: 'read a file', source: 'builtin' }, +]; + +describe('createMachineTools duplicate tool call ids', () => { + it('runs one batch per unique id and settles the superseded pending entry', async () => { + const { toolExecutor, batches } = createRecordingExecutor(); + const onBatchError = vi.fn(); + const tools = createMachineTools({ + toolExecutor, + toolInfos, + turnId: () => 1, + onBatchError, + }); + const bash = tools.tools.find((tool) => tool.name === 'Bash'); + const read = tools.tools.find((tool) => tool.name === 'Read'); + if (bash === undefined || read === undefined) throw new Error('missing tool definitions'); + + tools.beginBatch([call('t1', 'Bash'), call('t1', 'Bash'), call('t2', 'Read')]); + const first = bash.execute(input(call('t1', 'Bash'))); + const second = bash.execute(input(call('t1', 'Bash'))); + const third = read.execute(input(call('t2', 'Read'))); + + const [firstResult, secondResult, thirdResult] = await Promise.all([first, second, third]); + + expect(batches).toEqual([['t1', 't2']]); + expect(onBatchError).not.toHaveBeenCalled(); + expect(firstResult.isError).toBe(true); + expect(secondResult.content).toEqual([{ type: 'text', text: 'ok:t1' }]); + expect(secondResult.isError).toBeUndefined(); + expect(thirdResult.content).toEqual([{ type: 'text', text: 'ok:t2' }]); + expect(tools.extras.has('t1')).toBe(true); + expect(tools.extras.has('t2')).toBe(true); + }); + + it('reports executor failures through onBatchError and settles every pending call', async () => { + const toolExecutor = { + execute: (): AsyncIterable => ({ + [Symbol.asyncIterator]() { + return { next: () => Promise.reject(new Error('executor exploded')) }; + }, + }), + } as unknown as IAgentToolExecutorService; + const onBatchError = vi.fn(); + const tools = createMachineTools({ + toolExecutor, + toolInfos, + turnId: () => 1, + onBatchError, + }); + const bash = tools.tools.find((tool) => tool.name === 'Bash'); + const read = tools.tools.find((tool) => tool.name === 'Read'); + if (bash === undefined || read === undefined) throw new Error('missing tool definitions'); + + tools.beginBatch([call('t1', 'Bash'), call('t2', 'Read')]); + const [firstResult, secondResult] = await Promise.all([ + bash.execute(input(call('t1', 'Bash'))), + read.execute(input(call('t2', 'Read'))), + ]); + + expect(onBatchError).toHaveBeenCalledTimes(1); + expect(firstResult.isError).toBe(true); + expect(secondResult.isError).toBe(true); + }); +});