Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
8c959b4
fix(agent-core-v2): honor request-level toolMessageConversion in open…
7Sageer Sep 8, 2026
1874f4c
refactor(agent-core-v2): split the protocol trait into per-protocol d…
7Sageer Sep 8, 2026
1833731
refactor(agent-core-v2): compose format stages and dialect hooks in t…
7Sageer Sep 8, 2026
54ce16e
refactor(agent-core-v2): rename protocol dialects to traits
7Sageer Sep 9, 2026
4844b9a
fix(agent-core-v2): include the system message in openai mergeHistory…
7Sageer Sep 9, 2026
fef7f8d
refactor(agent-core-v2): enforce the format/trait contract boundary
7Sageer Sep 9, 2026
38598cf
refactor(agent-core-v2): thread the request config by spread and drop…
7Sageer Sep 9, 2026
e330f50
refactor(agent-core-v2): keep protocol format modules internal to the…
7Sageer Sep 9, 2026
3effd9b
refactor(agent-core-v2): plug llm credentials in through request config
7Sageer Sep 9, 2026
fe5b4b9
fix(agent-core-v2): surface request-actor failures as llm.failed.remote
7Sageer Sep 9, 2026
0f899d2
docs(agent-core-v2): attribute retry/recovery to the turn machine in …
7Sageer Sep 9, 2026
8bb2f04
refactor(agent-core-v2): unify credentials in human/credentials, turn…
7Sageer Sep 9, 2026
d7ffce0
fix(agent-core-v2): credential recovery for direct paths, abort guard…
7Sageer Sep 9, 2026
7bb6114
feat(agent-core-v2): IModelCatalog.generate with stream credential re…
7Sageer Sep 9, 2026
2e870d1
refactor(agent-core-v2): inline credential recovery at call sites
7Sageer Sep 9, 2026
fc59aec
fix(agent-core-v2): settle the queued turn when a machine turn settle…
7Sageer Sep 9, 2026
426e0ba
fix(agent-core-v2): bind and end pre-gate failures through the normal…
7Sageer Sep 9, 2026
b9be6d3
fix(agent-core-v2): settle seeded turns on pre-gate failure, preserve…
7Sageer Sep 9, 2026
bc38daa
fix(kap-server): add generate to IModelCatalog test fakes
7Sageer Sep 9, 2026
bb569c8
fix(klient): migrate examples to the credentials API
7Sageer Sep 9, 2026
0be003f
test(agent-core-v2): drop credential recovery coverage
7Sageer Sep 9, 2026
eba30ea
refactor(agent-core-v2): fold kimi-oauth credential adapter into cred…
7Sageer Sep 10, 2026
17b256b
refactor(agent-core-v2): pair credential refresh with invalidate, opt…
7Sageer Sep 10, 2026
d78c485
refactor(agent-core-v2): credentials recovery strategy chain, shared …
7Sageer Sep 10, 2026
7086d06
fix(agent-core-v2): settle message-less notifications on pre-gate fai…
7Sageer Sep 10, 2026
812ab19
Merge branch 'main' into refactor/llm-protocol-dialect
7Sageer Sep 10, 2026
4227356
Merge branch 'main' into refactor/human-connection-credentials
7Sageer Sep 10, 2026
22cc391
Merge branch 'main' into refactor/human-connection-credentials
7Sageer Sep 10, 2026
124c383
test(kap-server): add the IModelCatalog.generate stub to the history …
7Sageer Sep 10, 2026
a0ec899
Merge branch 'main' into refactor/human-connection-credentials
7Sageer Sep 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions packages/agent-core-v2/docs/en/llm.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ llm is a standalone LLM request library inside the human layer (`src/human/llm/`
2. **Streaming-native; events are the contract**. The only outward surface is a single, purely serializable event stream (requester level: `llm.sent / streaming.headers / streaming.part / streaming.usage / streaming.finish / streaming.message_id / failed.syntax / failed.remote / done`; the turn level adds `llm.retrying / llm.recovering`, and `llm.sent` carries the most recent recovery record). Streaming and non-streaming are isomorphic (non-streaming also accumulates over the stream, just without deltas). Events are emitted as they arrive — no caching, no fallback.
3. **format masks inter-protocol differences; traits express provider customizations**. format lives at the protocol layer and handles encoding/decoding of requests, responses, errors, usage, and finish. Each protocol owns a typed trait interface (`OpenAITrait` / `OpenAIResponsesTrait` / `AnthropicTrait` / `GoogleGenAITrait`) exposing only the customization points that protocol actually consumes — a hook a protocol ignores is unrepresentable, never silently dead. format and trait never import each other: both speak only the neutral wire/chunk types in the protocol's `contract.ts`. The requester is the composition root — `generate` runs a fixed per-protocol pipeline (`planOpenAIRequest` and friends) that alternates pure format stages (lower → assemble → encode → stream parser) with trait hooks (cacheKey/thinking → convertMessage → mergeHistory → convertTool → buildParams → extractUsage), so customization is explicit data flow instead of a closure captured inside format. Endpoint/env resolution and default headers form the provider `connection`, error classification is a requester option, and model capability is a provider-variant field — none of them are format business. Each base's public seam is contract + trait + requester; format, lower, and patterns are internal to the requester pipeline — only bases code and tests may import them (lint-enforced). Protocol differences must not leak into the turn or into requester decorators.
4. **Two-layer error model**. Internally, code throws the SDK's native errors; local request validation throws the shared `SyntaxRequestFormatError` (`llm/syntax-errors.ts`), which the requester converts uniformly via `toLlmSyntaxErrorMessage`, with no intermediate layer. Externally there are only `llm.failed.syntax` (local message syntax errors, never retried) and `llm.failed.remote` (remote streaming errors, subdivided into connection / timeout / rate_limit / quota_exhausted / context_overflow / request_structure, etc.), converted by format at the boundary.
5. **Stateless core + turn-driven orchestration**. `generate(config, content, control)` is a stateless function; errors are delivered via onEvent, never thrown. The turn machine invokes the request actor (`createRequestActor`) directly: the actor wraps a single request (messageResolvers, abort scope, event sendBack), and the turn drives retry and recovery through the pure policy functions in retry.ts / recovery.ts: recovery re-sends with replacement messages produced by the pure `propose` function (attempt resets to 1), retry backs off in the `retrying` state (honoring Retry-After), and the turn emits `llm.recovering / llm.retrying` for each. Empty response is judged by `withEmptyResponseGuard` at the requester boundary and raised as `llm.failed.remote`, entering the same retry path. Abort is carried by an AbortController owned by the turn: the controller is passed into the request actor via `LlmInput.signal`, and the turn aborts it directly on `turn.abort`, with the request ending as `llm.failed.remote`; the request actor neither creates its own controller nor touches any signal on teardown, so a finished request can never abort a shared signal. The accumulator is held by the turn and fed by the event stream; on `llm.retrying / llm.recovering` the turn rolls it back and recreates it, so every attempt accumulates from zero while as much interrupted state as possible is preserved (the turn finishes the complete message out of the accumulator at `llm.done`).
5. **Stateless core + turn-driven orchestration**. `generate(config, content, control)` is a stateless function; errors are delivered via onEvent, never thrown. The turn machine invokes the request actor (`createRequestActor`) directly: the actor wraps a single request (messageResolvers, abort scope, event sendBack), and the turn drives retry and recovery through the pure policy functions in retry.ts / recovery.ts: recovery is a strategy chain (credential refresh via `credentialsRecovery`, then replacement-message strategies such as media degradation) whose pure `propose` yields a recovery action — the turn executes the action (invalidating credentials and/or swapping messages) and re-enters `thinking` with attempt reset to 1; retry backs off in the `retrying` state (honoring Retry-After), and the turn emits `llm.recovering / llm.retrying` for each. Empty response is judged by the turn at `llm.done` via the pure `emptyResponseError` and re-raised as `llm.failed.remote`, entering the same failure cascade. Abort is carried by an AbortController owned by the turn: the controller is passed into the request actor via `LlmInput.signal`, and the turn aborts it directly on `turn.abort`, with the request ending as `llm.failed.remote`; the request actor neither creates its own controller nor touches any signal on teardown, so a finished request can never abort a shared signal. The accumulator is held by the turn and fed by the event stream; on `llm.retrying / llm.recovering` the turn rolls it back and recreates it, so every attempt accumulates from zero while as much interrupted state as possible is preserved (the turn finishes the complete message out of the accumulator at `llm.done`).
6. **No silent fallback**. Configuration is taken exactly as given. For beta features, thinking, empty response, and similar scenarios, define explicit error conditions first, fail at request time, and guide the user to fix the configuration — never fall back silently.
7. **Every variable capability is a contribution point**. Providers, media upload/degradation, usage, traceId, and error recovery (compaction / media degradation) all plug in through extension points; the llm core contains none of these concepts.
8. **Data is data**. A model is pure, function-free data (endpoint url + model uniquely identifies a model), serializable and directly usable as generate input. The catalog is a derived `provider -> models` cache; the dependency direction only goes from models-dev into llm internals, never the reverse.
Expand All @@ -33,11 +33,18 @@ llm/
├── requester/
│ ├── requester.ts LlmRequester.generate(config, content, control);
│ │ ExtraParams typed per protocol {openai?, responses?, anthropic?, googleGenai?}
│ │ ExtraParams typed per protocol {openai?, responses?, anthropic?, googleGenai?};
│ │ LlmRequestConfig.credentials: credential contribution point
│ │ (resolve/canRecover/invalidate), resolved per attempt by the caller;
│ │ factories and the credentialsRecovery strategy live in human/credentials
│ │ (staticCredentials / oauthCredentials; kimiOAuthCredentialProvider adapts
│ │ Kimi OAuth tokens); the runWithCredentialRecovery /
│ │ streamWithCredentialRecovery executors for direct callers live in
│ │ llm-adapter/model/credential-recovery
│ ├── actor.ts request actor: a fromCallback wrapping a single request
│ │ (messageResolvers, abort scope, event sendBack); invoked by the turn
│ ├── retry.ts / recovery.ts pure retry/recovery policy functions (driven by the turn machine; propose is pure)
│ ├── empty-response.ts withEmptyResponseGuard: judges empty responses at finish and raises llm.failed.remote
│ ├── empty-response.ts emptyResponseError: pure empty-response judgment; the turn raises it as llm.failed.remote at llm.done
│ └── bases/ four protocol bases: openai / openai-responses / anthropic / google-genai
│ each with contract / format / lower / patterns / capability / extra-params / trait / requester
│ (public seam: contract / trait / requester; format / lower / patterns stay internal)
Expand All @@ -53,7 +60,7 @@ llm/
└── media/ media contribution points: cache / degrade / ref / resolver / store / upload
```

Request lifecycle: `generate` receives (config, content, control) → the requester's `plan*` function composes pure format stages with trait hooks into protocol requestParams (format lowers the generic Message[] through the Pattern Rewriter; trait adjusts kwargs, converted messages, history, tools, and final params in between) → internalGenerate calls the official SDK → streaming chunks are converted by the stateless parser callbacks into `llm.streaming.part / streaming.usage / streaming.finish / streaming.message_id` events → errors are converted by format into `llm.failed.*`; on success the requester emits `llm.done`, on failure it ends with `llm.failed.syntax / llm.failed.remote` and never emits `llm.done`. At `llm.done` the turn judges empty responses via `emptyResponseError` and re-raises them as `llm.failed.remote`; the turn machine first tries recovery on `llm.failed.remote` (replacement messages from the pure `propose` function, emitting `llm.recovering`), then retries with backoff (honoring Retry-After, emitting `llm.retrying`), and only fails the turn once attempts are exhausted. The turn holds the HistoryAccumulator, fed by the event stream, rolls it back and recreates it on `llm.retrying / llm.recovering`, and finishes the complete message at `llm.done`; usage accounting, tracing, compaction, and media degradation all attach to the event stream as plugins/contribution points.
Request lifecycle: `generate` receives (config, content, control) → the caller resolves `config.credentials` into a fully-credentialed model before each attempt (the request actor on the machine path), so requests always carry fresh credentials and a credential-refresh recovery (recoverable 401 → `credentials.invalidate()`, emitted as `llm.recovering` with strategy `credentials`) naturally re-resolves on the re-send (direct callers outside the state machines — ping, generate, full compaction, media upload — share the same single-retry recovery through `runWithCredentialRecovery` / `streamWithCredentialRecovery`) → the requester's `plan*` function composes pure format stages with trait hooks into protocol requestParams (format lowers the generic Message[] through the Pattern Rewriter; trait adjusts kwargs, converted messages, history, tools, and final params in between) → internalGenerate calls the official SDK → streaming chunks are converted by the stateless parser callbacks into `llm.streaming.part / streaming.usage / streaming.finish / streaming.message_id` events → errors are converted by format into `llm.failed.*`; on success the requester emits `llm.done`, on failure it ends with `llm.failed.syntax / llm.failed.remote` and never emits `llm.done`. At `llm.done` the turn judges empty responses via `emptyResponseError` and re-raises them as `llm.failed.remote`; the turn machine first tries recovery on `llm.failed.remote` (the strategy chain — credential refresh on a recoverable 401, then replacement-message strategies — via the pure `propose`, emitting `llm.recovering`), then retries with backoff (honoring Retry-After, emitting `llm.retrying`), and only fails the turn once attempts are exhausted. The turn holds the HistoryAccumulator, fed by the event stream, rolls it back and recreates it on `llm.retrying / llm.recovering`, and finishes the complete message at `llm.done`; usage accounting, tracing, compaction, and media degradation all attach to the event stream as plugins/contribution points.

## Rejected Schemes (do not reintroduce)

Expand Down
Loading
Loading