feat(protocols): protocol paths as a first-class concern — PF-01..PF-12

devlog/_plan/260924_protocol_first_class/000_plan.md

@@ -0,0 +1,88 @@
+# 260924 Protocol first class — plan
+
+Responses stays the first-class feature surface. What changes is that Chat Completions and
+Anthropic Messages stop needing the public Responses JSON/SSE as a mandatory intermediate to
+reach the shared execution policy. Same-wire requests keep their source representation;
+cross-wire requests convert through the adapter-neutral IR; one execution owner keeps account
+selection, affinity, send budget, retry, cancellation and logging.
+
+## Outcome
+
+```text
+Responses / Chat / Messages request
+  -> source body kept + lazily parsed intent
+  -> shared admission, routing, execution policy
+  -> final provider / model / credential settled
+  -> per-attempt protocol plan
+       same wire        : native builder from the source body
+       different wire   : codec -> IR -> target builder
+       not migrated yet : legacy bridge (internal Responses), labelled as such
+       not expressible  : refused before any send (when policy = reject)
+  -> upstream
+
+same wire     : upstream -> safe relay + observation -> client wire
+different wire: upstream -> AdapterEvent -> client encoder
+```
+
+## Non-goals
+
+- Files, Batches and Responses CRUD APIs; every beta feature; lossless behavior for arbitrary
+  custom providers; emulating every Responses-only feature on Chat or Messages.
+- A second or third execution engine. Native lanes reuse the shared attempt, budget, cancel
+  and log owners; they do not copy them.
+- Renaming the Responses schema and calling it a neutral IR.
+- Forwarding arbitrary headers or body fields to every provider unchecked.
+- Guessing native support from a provider name in the GUI.
+- A shadow mode that sends two inferences. Shadow compares plans only.
+- Presenting one successful connection as protocol verification.
+
+## Invariants every work packet keeps
+
+1. The planner never selects a provider. It consumes the route the router settled
+   (`ResolvedModelPolicy` precedence: hard-pin, explicit override, ingress-scoped registry
+   default, provider default) and decides only the wire within that route.
+2. No native lane may bypass admission scope, send budget, affinity, key failover, cancellation,
+   request logging or spend accounting. A native lane that lands before its safety wiring is
+   not acceptable in any order.
+3. Every fallback candidate builds its request from the source envelope. An earlier candidate's
+   deleted fields or injected headers are never the next candidate's input.
+4. Plan and trace records carry only fixed vocabulary (`src/protocols/contract.ts`) and
+   identifiers the server already exposes. No prompt, tool argument, token, signature or key.
+5. `native` (a delivery mode) and `VERIFIED` (a Lab evidence verdict) are different axes and
+   are never merged into one badge.
+6. Every rollout switch defaults off and changes nothing while off
+   (`resolveProtocolSettings`, `src/protocols/settings.ts`).
+7. The core request path stays free of Lab imports
+   (`tests/lab/core-lab-boundary.test.ts`).
+8. Files at their file-size cap (`tests/fixtures/file-size-baseline.json`) do not grow; code
+   moves out first. `src/server/request-log.ts` sits at 1996 of a 2000-line threshold.
+
+## Work packets and stack order
+
+Each packet is one pull request, stacked on the previous one. PF numbers are work ids, not
+GitHub numbers.
+
+| Packet | Branch | Scope | Doc |
+|---|---|---|---|
+| PF-01 | `feat/pf01-protocol-contract` | vocabulary, feature dispositions, 18-cell baseline, plan/trace DTOs, settings keys | [010](010_contract_and_baseline.md) |
+| PF-02 | `feat/pf02-protocol-trace` | observed path trace on request/attempt rows, persisted; Logs badge, detail, filter | [030](030_gui_and_management_api.md#pf-02-observed-path-trace) |
+| PF-03 | `feat/pf03-protocol-plan` | pure planner, `GET /api/protocols`, `POST /api/protocols/plan`, API page preview | [030](030_gui_and_management_api.md#pf-03-planner-and-preview) |
+| PF-05 | `feat/pf05-inference-primitives` | shared execution context, attempt and delivery primitives, client-wire marker | [020](020_engine_and_codecs.md#pf-05-shared-inference-primitives) |
+| PF-04 | `feat/pf04-api-surfaces` | Messages exposure split from Claude integration, settings PATCH, API cards | [030](030_gui_and_management_api.md#pf-04-api-surface-settings) |
+| PF-06 | `feat/pf06-source-envelope` | source envelope, codecs, unrepresentable guard | [020](020_engine_and_codecs.md#pf-06-source-envelope-and-guard) |
+| PF-09 | `feat/pf09-direct-encoders` | AdapterEvent to Chat/Messages encoders behind `directEncoders` | [020](020_engine_and_codecs.md#pf-09-direct-client-encoders) |
+| PF-07 | `feat/pf07-native-chat-combos` | eligible Chat candidates in combos/policy send natively | [020](020_engine_and_codecs.md#pf-07-native-chat-candidates-in-combos) |
+| PF-08 | `feat/pf08-managed-messages-native` | key-auth Anthropic targets receive `/v1/messages` natively | [020](020_engine_and_codecs.md#pf-08-managed-native-messages) |
+| PF-11 | `feat/pf11-protocol-evidence-gui` | provider protocol panel, compatibility pair filters, combo guarantees, deep links | [030](030_gui_and_management_api.md#pf-11-evidence-combo-and-provider-views) |
+| PF-10 | `feat/pf10-auth-opaque-state` | beta allowlist, OAuth native Messages, cross-domain opaque state guard | [020](020_engine_and_codecs.md#pf-10-auth-and-opaque-state) |
+| PF-12 | `feat/pf12-protocol-rollout` | shadow plan comparison, docs, not-migrated inventory | [040](040_acceptance_and_rollout.md) |
+
+PF-05 lands before PF-04 because PF-06 through PF-09 build on its primitives and PF-04 does
+not; the dependency order, not the id order, decides the stack.
+
+## Verification policy for this unit
+
+Each pull request records exactly what ran. Unit tests are written beside each change and
+registered in the test layout; whether they were executed is stated per PR, never implied.
+Default flips for rollout switches are out of scope until the acceptance scenarios in
+[040](040_acceptance_and_rollout.md) have recorded evidence.

devlog/_plan/260924_protocol_first_class/010_contract_and_baseline.md

@@ -0,0 +1,104 @@
+# 010 — PF-01 contract and baseline
+
+## Modules
+
+| File | Role | Import rule |
+|---|---|---|
+| `src/protocols/contract.ts` | protocols, upstream wires, hops, delivery modes, reason codes, name mappings | leaf |
+| `src/protocols/features.ts` | feature keys, sources, cross-wire hop dispositions, body extraction, path effects | leaf (+ type from `src/compatibility/manifest.ts`) |
+| `src/protocols/baseline.ts` | 18-cell current/target matrix | leaf |
+| `src/protocols/dto.ts` | `ProtocolPlanV1`, `ProtocolTraceV1`, validators, limits | leaf |
+| `src/protocols/settings.ts` | only reader of `apiSurfaces` and `protocols` config | type-only config import |
+
+"Leaf" means importable from `gui/src/*`: no server, router, provider or Lab imports, not even
+as types. `tests/responses/protocol-contract.test.ts` enforces it by reading import specifiers.
+
+## Vocabulary
+
+- **Protocol**: `responses | chat | messages` — the public API a client speaks.
+- **Upstream wire**: a protocol or `other` (Gemini, Kiro, Cursor, ...). Nothing is claimed about
+  `other`; absent feature dispositions there mean unknown.
+- **Hop**: a wire name, `ir` (`OcxParsedRequest` / `AdapterEvent`), or `responses-internal`
+  (Responses JSON/SSE produced only as an internal bridge).
+- **Delivery mode**: `native` (same wire, source body is the wire source), `translated`
+  (cross-wire through the IR or the target wire only), `legacy-bridge` (path contains
+  `responses-internal`), `blocked` (refused before any send).
+- **Fidelity**: `preserved`, `degraded`, `unknown`.
+
+Internal spellings keep their names and map explicitly: routing `InboundWire` "anthropic" is
+`messages`; adapters `openai-responses`/`openai-chat`/`anthropic` are the three protocol wires;
+Lab identities `openai-responses`/`openai-chat`/`anthropic-messages` map one to one. Persisted
+rows are not rewritten.
+
+## Feature dispositions and their evidence
+
+Same-wire hops are passthrough by definition. Cross-wire claims about current code:
+
+| Claim | Evidence |
+|---|---|
+| Chat `n`, `logprobs`/`top_logprobs`, `logit_bias`, `seed`, `audio`/`modalities`, `prediction` are unsupported on `chat>responses` | `chatCompletionsToResponsesBody` in `src/chat/inbound.ts` builds the body from an explicit field list without them |
+| Chat tools, sampling, stop, user, parallel tool calls, service tier, prompt cache key, metadata, reasoning effort, response format are translated on `chat>responses` | same function, explicit field copies |
+| Messages `top_k` is unsupported on `messages>responses` | `src/claude/inbound.ts` header: accepted and dropped |
+| Messages `thinking.budget_tokens` is degraded on `messages>responses` | maps to an effort tier, never forwarded raw |
+| Messages `cache_control` is degraded on `messages>responses` | block-level cache hints are not carried into the Responses body; caching is re-derived downstream |
+| Responses hosted tools are degraded on `responses>chat` and `responses>messages` | only web search and image generation have sidecar bridges |
+| Responses `previous_response_id` and compaction are translated off-wire | expanded from proxy-side state before the adapter runs |
+| Responses `store` is degraded and `background` unsupported off-wire | only proxy-side state exists for non-Responses upstreams |
+
+The `chat>messages` and `messages>chat` rows describe the target direct codecs (PF-06). Today
+those pairs travel through `responses-internal`, so their effective disposition is the
+composition of the two Responses hops, which `featureEffectsForPath` computes from the path.
+
+## Baseline (eligible single-provider route)
+
+| inbound → upstream | current | target |
+|---|---|---|
+| responses → responses | native `responses,responses` | native |
+| responses → chat / messages | translated `responses,ir,X` | translated |
+| chat → chat | native `chat,chat` (JSON when `stream:false`) | native |
+| chat → responses | translated `chat,responses` | translated |
+| chat → messages | legacy-bridge `chat,responses-internal,ir,messages` | translated `chat,ir,messages` |
+| messages → messages | legacy-bridge for managed keys (native only for caller-forwarded Anthropic credentials) | native |
+| messages → responses | translated `messages,responses` | translated |
+| messages → chat | legacy-bridge | translated `messages,ir,chat` |
+
+Every routed (non-native) Chat and Messages path streams internally and folds for a
+non-streaming client (`sse-folded`). The stream axis doubles the table to 18 cells in
+`src/protocols/baseline.ts`.
+
+Out of scope for the baseline, and described per request by the planner instead: combos and
+policy routes (legacy bridge today), OAuth/forward credentials, synthetic effort rows,
+vision-preprocessed images, tool-result images, and Responses-only features on Chat.
+
+## DTOs
+
+`ProtocolTraceV1` is the observed record for one request (`v: 1`), persisted with the request
+log row; `attempts[]` records each physical attempt's upstream, mode and request path. A blocked
+request has empty paths. `ProtocolPlanV1` is the predicted record (`schemaVersion: 1`) with one
+candidate per route target, `guaranteedFeatures` (preserved by every eligible candidate) and
+`partialFeatures`. Both validators reject anything that is not exactly v1, oversize, or outside
+the vocabulary. Limits: 6 hops, 8 reason codes, 24 feature effects, 16 candidates, 16 attempts,
+200-character identifiers without control characters.
+
+## Settings
+
+```jsonc
+{
+  "apiSurfaces": { "messages": { "enabled": false } },   // absent => inherit claudeCode.enabled
+  "protocols": {
+    "unrepresentable": "legacy",                        // or "reject"
+    "rollout": {
+      "nativeChatCombos": false,
+      "managedMessagesNative": false,
+      "managedMessagesNativeOAuth": false,              // effective only with the key-auth switch
+      "directEncoders": false,
+      "shadowPlan": false
+    }
+  }
+}
+```
+
+`apiSurfaces` is kept raw by the config schema and parsed fail-closed: a present malformed value
+closes the surface rather than inheriting. `protocols` is schema-validated and drops to defaults
+when malformed, because every default is the conservative one. PF-01 adds the keys and the
+resolver only; no request path reads them until PF-04.