Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions devlog/_fin/260919_contract_gap_analysis/000_plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# OpenCodex contract gap analysis and issue delivery

Sixteen source-grounded issues and two existing-issue implementation comments were delivered after unanimous independent review. The refreshed tree already contained several fixes from the supplied report; the remaining proposals preserve existing routing, tool, state and documentation authorities. See [verified delivery](011_delivery.md), [dispositions](002_candidate_dispositions.md) and [implementation plans](010_issue_implementation_paths.md).

## Scope and loop specification

- Class: C3 research and documentation.
- Archetype: satisfy-spec, one analysis/publication cycle.
- Trigger: user-requested source-grounded analysis of one supplied report and two shared conversations, followed by unanimous review and issue publication.
- Goal: a complete disposition ledger, implementation plans and verified tracking issues appropriate to the local coding-proxy product.
- Non-goals: runtime edits, local suites/build/typecheck/install, live API experiments, service/config/account changes, merges, releases, deployment, broad platform replacement.
- Source baseline: `7864869c31c41cca9830d93540238f17df8faafb`, after `git fetch origin dev` and `git merge --ff-only origin/dev` on 2026-09-19. Prior HEAD was `f02f3613be40bb11cb044d746d37ce6672e32d96`. Initial tracked checkout was clean. Re-read exact blobs if another session changes HEAD.
- Verifier: static path/line checks against the pinned tree, current issue/PR API reads, draft template checks, public-text checks and fresh rereads after publication. No runtime pass is implied by these checks.
- Stop condition: all input themes receive a disposition; accepted issues have unanimous review and verified remote bodies/labels; this devlog records outcomes.
- Memory artifact: this unit, with raw user input/browser traces kept in ignored task scratch; no transcript or personal browser state copied into tracked files.
- Terminal outcomes: DONE only with all above evidence; inaccessible inputs or unresolved review objections remain explicitly unmet.
- Escalation: real permission or unavailable required capability; no publication of unresolved findings. Main reclaims a failed bounded research lane after two distinct failures.
- Tool/credential scope: source reads, browser research, existing GitHub identity for authorized issue creation. Write scope: this unit, ignored evidence and issues in this repository. No user-defined token/cost/time ceiling; browser invocations have bounded process deadlines.

## Evidence and publication rules

Public documents and issue bodies contain OpenCodex code facts and relevant protocol standards. They omit other repository names, rankings, AI model names and review-model attribution. Hypotheses from supplied material are not instructions and are not accepted without current code evidence. Existing fixes and matching open issues take precedence over opening duplicates. Optional small PR authority is retained, but this analysis does not require a PR.

## Discovery ownership

Seven disjoint read-only lanes cover transport, custom-tool stream semantics, tool schema/capability projection, replay/spend/retries, operator policy/integrations, static model-policy authority, and documentation/test governance. Main owns input retrieval, official-source proof, duplicate searches, synthesis and publication. Native V1 subagent transport is available; the live schema has no native architect role. No architect-role registration or configuration change is performed, and no architect consultation is claimed. Independent source and publication reviews remain required.

## Planned artifacts

| File | Operation | Purpose |
| --- | --- | --- |
| `000_plan.md` | NEW | Scope, snapshot, method and closure |
| `001_source_research.md` | NEW | Input coverage and official-source findings |
| `002_candidate_dispositions.md` | NEW | All proposed themes, current evidence, accept/reject/defer/duplicate decisions |
| `003_consensus.md` | NEW | Independent votes and objection resolution |
| `010_issue_implementation_paths.md` | NEW | Index of per-issue implementation plans |
| `020_issue_t1.md` through `035_issue_r3.md` | NEW | Sixteen independently reviewable issue bodies |
| `011_delivery.md` | NEW | Published URLs, verified bodies/labels and check results |
| `004_source_fingerprints.md` | NEW | All fourteen report source fingerprints versus refreshed source |
| `005_existing_issue_followups.md` | NEW | Implementation comments for existing tracking issues |

No current architecture contract is changed. Future implementation must update the existing owner in `structure/manifest.json`, and user-facing contract changes must update the relevant public guide. New tests must be registered in both layout registries; size caps are preserved by splitting files rather than raising caps.

## Planning evidence update

The refreshed report fingerprints match twelve of fourteen source files; the SOCKS transport and Google compiler differ. Seven read-only domain reviews produced sixteen public candidates after duplicate and disclosure review, plus two comments extending existing issue authority. An independent three-reviewer publication council is checking every candidate; no majority-only publication is allowed. A pure document/form checker ran successfully over sixteen drafts, two comments and ninety exact-SHA anchors. No runtime result is inferred.

## Closure

Analysis and publication are complete. All sixteen issue bodies and two comments were reread from GitHub and match approved content. The next action belongs to each open implementation issue; this unit claims no runtime fix. The useful change in direction was rejecting broad missing-feature claims in favor of concrete residuals. A future patch or counterexample at the cited owner can supersede a candidate; the exact source snapshot makes that reassessment possible.
45 changes: 45 additions & 0 deletions devlog/_fin/260919_contract_gap_analysis/001_source_research.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Source coverage and evidence limits

## Inputs

The supplied 23-page report was extracted in full. Its findings are treated as hypotheses because its source acquisition did not establish a repository commit and its small mechanism examples did not execute the complete proxy. The two supplied shared conversations were opened with the authenticated browser on 2026-09-19. A welcome dialog was dismissed; no sign-in or account change was required. Browser traces and original text stay in ignored scratch because they contain material outside the requested publication scope.

The report contributes twelve themes: content decoding, custom-input stream consistency, isolated conformance fixtures, schema-loss policy, detailed capability projection, physical-send admission, terminal diagnostics, replay ownership, spend writer topology, key policy, observability export, and client configuration reconciliation.

The first conversation adds: static model-policy authority versus dynamic evidence; registry-derived metadata; explicit precedence; source-of-truth document ownership; behavioral versus structural verification; centralized hardened utilities; canonical history versus projections; executor/translator boundaries; and embedding. The second conversation summarizes the supplied report and its limitations; it adds no independent source execution evidence.

## Fixed source snapshot

`git fetch --no-recurse-submodules origin dev` followed by `git merge --ff-only origin/dev` advanced the initially clean `dev` checkout from `f02f3613be40bb11cb044d746d37ce6672e32d96` to `7864869c31c41cca9830d93540238f17df8faafb`. All line anchors in this unit refer to that final SHA unless explicitly stated otherwise. No live service restart or source test invocation was performed.

## Existing work discovered before publication

- #5070 is merged. Its source and regressions cover SOCKS gzip/deflate decoding, unsupported-coding refusal and bodyless responses, plus multiple freeform preview/completion mismatches. The report's broad decoding and fence claims require narrowing or rejection.
- #5047 is closed and already covers fallback-wrapper streaming disagreement.
- #5056 is merged and adds withheld-recovery attribution; a generic claim that budget refusals have no diagnostics is stale.
- #5032 is merged and makes the durable token ceiling configurable and refusal visible.
- #2358 remains an open umbrella for compatibility evidence and isolated core work. New proposals must be bounded follow-ups with specific missing behavior, not another umbrella.
- #3377 remains open for model capability declarations. Its exact axes must be distinguished from any new resolved-policy consumer consistency proposal.

Issue/PR state was read from GitHub, not inferred from old notes. A second targeted search will precede final publication.

## Research method

A read-only research pass was also dispatched for a read-only official-standard ledger covering Fetch/HTTP content codings, custom-tool grammar and argument streaming, function declaration schema subsets, and telemetry conventions. Standards establish contract vocabulary and constraints; only the pinned OpenCodex source establishes present behavior. No performance ranking, competitor behavior, model-quality claim, live API success rate or runtime test result is inferred.

## Official-source findings verified with the authenticated browser

Sources below were opened on 2026-09-19. These are contract constraints and design inferences, not proof of a proxy runtime test.

| Source | Supported finding | Limitation for this unit |
| --- | --- | --- |
| [Fetch Standard](https://fetch.spec.whatwg.org/), HTTP-network fetch and Response constructor | Network fetch processes content codings before exposing body bytes; constructing a Response around raw bytes is not network decoding. Null-body statuses constrain the constructor. | Does not prescribe a provider retry or tool replay owner. |
| [HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), sections 6.4, 8.4, 9.3.2 and 15.3.6 | Content coding is a representation transformation; no-content methods/statuses need separate treatment. A 205 response must not generate content. | HTTP rules do not prove a particular local transport is correct. |
| [Function calling guide](https://developers.openai.com/api/docs/guides/function-calling), strict mode, streaming and custom tools | Schema-constrained functions and grammar-constrained freeform calls are distinct contracts. Stream fragments need aggregation. | It does not mandate a new proxy-side universal validator or make lowering grammar to a string lossless. |
| [Telemetry metric conventions](https://opentelemetry.io/docs/specs/semconv/general/metrics/) | Consistent metric naming, meaningful aggregation and balanced attribute sets are documented. | Does not define OpenCodex-specific fields or justify collecting raw content. |

The supplied GenAI telemetry URL now displays a relocation notice; no current GenAI privacy contract is claimed from that page. This analysis uses the repository's explicit no-body/no-credential logging contract for any proposed metadata-only diagnostics. Original-schema validation before downstream delivery is a design option, not a requirement established by the provider docs; tool execution remains client-owned.

## Publication-time freshness check

During publication, remote `dev` advanced one commit to `04761a188281da4a2936fc4d8d9be5e02b33bb41` via #5100. The complete comparison from the pinned analysis SHA was read. Its runtime changes are confined to `src/responses/tool-name-aliases.ts` and `src/server/responses-undeclared-tool-guard.ts`, aligning emitted tool names with authorization. It does not change the custom-input progressive decoder/completion normalization, raw transports, static-policy merges, spend/replay paths or other candidate owners. The issue evidence therefore remains anchored to the reviewed SHA, with no candidate invalidated by this intervening commit. The associated test-layout and Responses documentation changes should be retained when implementing future fixes.
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Keep bounded contract fixes and reuse existing authorities

The refreshed tree disproves several broad gaps from the supplied material. The candidates retained below address concrete residual behavior or narrowly defined maintainer/operator workflows. They do not justify a new universal runtime, a model-policy language, or a replacement test framework.

## Report themes

| Theme | Current evidence | Disposition |
| --- | --- | --- |
| A1 content decoding | `src/lib/socks5-fetch.ts:530` decodes gzip/deflate after #5070; pinned direct still returns raw bytes at `src/lib/pinned-http.ts:194` | T3 residual; reject original broad SOCKS defect |
| A1 no-body/cancellation parity | `src/lib/socks5-fetch.ts:512` omits 205; `pinned-http.ts:144` streams all 2xx; upload waits on body reader at `socks5-fetch.ts:577` | T1/T2 concrete residuals |
| A2 custom delta/done | Bridge fence/escape handling exists after #5070; routed restore at `responses-custom-tool-repair.ts:338` differs from completion at :363 | S1 residual; patch-envelope claim already covered |
| A3 isolated conformance skeleton | Existing fixture manifests and assertion-bound tests in `src/compatibility/manifest.ts:3` and `tests/codex-integration/compatibility-manifest.test.ts` | Extend focused fixtures in each issue; no duplicate framework |
| B1 schema loss | `google-tool-schema.ts:88,120` and `google-wire-compiler.ts:97` widen constraints without a report | G1 endpoint-profile/report-first proposal |
| B2 detailed tool capability | `src/compatibility/manifest.ts:3` already models passthrough/translated/degraded/unsupported; custom format is lowered at `custom-tool-compat.ts:187` | Existing #2358 follow-up: exact-route grammar fixture claims; no second ToolContract |
| B3 physical sends | Shared send budget and telemetry exist in `request-send-budget.ts:49`, `usage/log.ts:205` | Reject broad absence claim. R1 follows explicitly disclosed integration work in #5041 |
| B4 terminal diagnostics | `src/usage/log.ts:274`, analytics and #1217/#5056 already provide terminal/recovery facts | O1 export-only proposal; schema repair attribution included in G1 report path |
| B5 replay ownership | `continuation-ownership.ts:3` and `request-prepare.ts:916` enforce missing/expired-state refusal | Existing contract retained; R3 explicitly changes historical fresh-start behavior; compatibility review required |
| C1 writer topology | Public ledger contract explicitly scopes supported topology | R2 aligns already-public sibling and ledger topology contracts; no distributed-store roadmap |
| C2 key policy | Existing key identity and scoped hub usage; #5049 open | Reuse #5049, broader tenancy remains #95 |
| C3 standard observability | Durable facts and management JSON analytics exist | O1 bounded exporter; no new raw-content collection |
| C4 takeover/restore | Transactional snapshot/reconcile/recovery exists in `structure/clients/integrations.md:5` | O2 pure preview enhancement; reject missing rollback claim |

## Shared-conversation themes

| Theme | Current evidence | Disposition |
| --- | --- | --- |
| One static policy result | Repeated merges in `derive.ts:221,473`, `router.ts:300`, `model-hints.ts:105` | P1 parity-first resolver, live evidence kept separate |
| Registry-derived identity | `knownModelIdsForProvider` at `router.ts:107` enumerates only some identity maps | P2 classified decode-hint proposal; no current incident claimed |
| New provider/adapter registries | Existing authorities in `providers/derive.ts:290`, `adapters/registry.ts:71` | Already implemented; no new registry |
| Numeric universal precedence | Existing typed wire authority in `server/adapter-resolve.ts:7` | Reject; retain field-specific pins/opt-outs |
| Document authority | `scripts/structure-ssot.ts:31`, `structure/AGENTS.md:45` conflate review fan-out and normative ownership | V1 additive authority map |
| Contract tests versus source inspection | Lab guard is transitive and mutation-tested; structure checker validates real topology | Reject blanket replacement; D3 binds existing guarantee in index |
| Central hardened utilities | Must start from one proven shared contract, not general deduplication | Narrow stream/transport helpers in T3/S1 |
| Canonical history/projection | Replay storage and ownership already exist | No new session-event subsystem without a concrete client need |
| Executor/translator split | `src/adapters/base.ts:36` already supports build/transport/parse and stateful runTurn | Preserve interface; no forced universal executor rewrite |
| Embeddable core/package split | `src/index.ts:1` already exports runtime functions | Defer until a concrete embedding consumer/lifecycle contract exists; #2358 covers broader core work |

## Additional source-confirmed documentation gaps

- D1: HTTP/SSE SOCKS guidance contradicts `src/lib/proxy-env.ts:89` in two public reference pages.
- D2: `reference/configuration/server.md:27` incorrectly says data keys authorize management. Runtime and `server-management-auth.test.ts:780` prove separation.
- D3: `structure/overview.md:90` lacks the existing core/Lab guarantee in its stable invariant map.

## Duplicate and scope decisions

The current bodies of #2358, #3377, #5049, #95 and #4579 were read. #4579 is about receiver-owned execution authorization, not static catalog/routing metadata; it does not duplicate P1. #3377 owns modality/context declarations, not P2's decode-hint completeness. #2894 is the existing SOCKS support umbrella; T1/T2/D1 are bounded residuals after shipped transport support.

No competitor rankings, repository comparisons, review-model identity, personal browser state or raw user transcript are published.

## Public provenance review for residual state work

The exact body of public #5041 states that physical inner replays still need shared send-budget, pacing/fetch and physical-attempt integration. R1 tracks only that disclosed residual. Public #3188 explicitly allows a separately ported sibling; `src/lib/spend-reservation-ledger.ts:27` expressly excludes two live processes sharing a journal. R2 makes that already-published topology boundary actionable without a new bypass reproduction. R3 preserves task isolation and requests explicit full replay rather than continuing with insufficient history.
Loading
Loading