GPT-6 lane compression A/B: preregistration draft (not frozen, not run)

[seathatflowsinourveins]

Scope

• What this PR changes: adds a draft preregistration, not frozen and not run, for an A/B of the GPT-6 Codex lane.
  • The treatment is the framework OmniRoute instance on 127.0.0.1:20129: all engines, the allow-lossy header and output styles.
  • The control is the uncompressed shared gateway on 127.0.0.1:20128.
  • The A/B decides, per GPT-6 role, which token-save features stay on without harming quality.
• Base commit: e82e6be7 (rebased; head 73fc873e)
• Lane: lane:foundation
• Owned paths touched: blueprints/gpt6-lane-compression-ab/, tests/test_gpt6_lane_compression_ab_preregistration.py

SOTA sources

• Harbor v0.23.0 (harbor-framework/harbor @ 1e5c5c6d), the primary agent E2E harness.
  • Codex agent configuration: src/harbor/agents/installed/codex.py L1207-1296.
  • Native launch: L1432-1448.
  • Forwarding of the -p profile was not found, which is recorded as a sealing blocker.
• promptfoo 0.123.1 (promptfoo/promptfoo @ 34f74d34), for gateway-level cells.
  • OpenAI base URL: src/providers/openai/index.ts L111-128.
  • Responses headers: src/providers/openai/responses.ts L1195-1210.
• inspect_swe (meridianlabs-ai/inspect_swe @ 7eb8dd64), the runner-up. Its Codex bridge is at src/inspect_swe/_codex_cli/codex_cli.py L619-637.
• Repair-round sources:
  • Harbor main 3c823808, codex.py L1502-1605: it strips the model prefix, as the pin does (L1339-1449).
  • promptfoo src/providers/openai/codex-sdk.ts L1034-1134, and the Codex SDK sdk/typescript/src/exec.ts L91-178 (openai/codex @ 36650394): the model passes intact to --model.
  • The Codex profile loader, codex-rs/config/src/loader/mod.rs L286-340.
  • inspect_swe codex_cli.py L483-637.
• scipy stats.bootstrap and permutation_test, plus statsmodels multipletests(method="holm"), for the statistics.
• OmniRoute 3.8.51 (build dd6e9607e), the installed source for the engine behaviour the canaries target. The 17 files cited matched upstream byte for byte.
• Codex rust-v0.157.1.
  • models-manager/src/manager.rs L763-780 and model_info.rs L99-150 explain why the engines-on model must have one slash.

Evidence-class table

Claim	Evidence class	Command / receipt
The protocol file is structurally complete: required keys, null results, hashed packets, and a structural check for every review blocker	synthetic (structural test)	python3 -m unittest tests.test_gpt6_lane_compression_ab_preregistration: 10 tests. Before the repair edits: FAILED (failures=12). Now: OK
Harness capabilities at their pins	source_review	the gh api reads cited above
Any quality or cost outcome	not claimed	launch is prohibited until the review round is done and the draft is sealed

Local commands run (head 73fc873e, with TMPDIR outside every repository)

$ python3 -m unittest tests.test_gpt6_lane_compression_ab_preregistration
Ran 10 tests ... OK
$ python3 -m unittest tests.test_osv_lockfile_coverage tests.test_blind_checkout tests.test_workflow_security_coverage
Ran 57 tests ... OK
$ python3 scripts/validate.py
passed, after re-registration per docs/lanes.md

Status

A draft, unfrozen and unrun. The review round is done. All eleven sealing gates remain open, and every gate result is null.

• Review and repair. The independent Claude review raised 23 findings. One GPT-6 repair round (gpt-6-astra, effort max) resolved them: 11 fixed and 12 fixed by gating. Fixed by gating means the protocol is corrected and native qualification is still required.
  • No finding was declined whole.
  • M9 declines only one thing: using request size as a bound on total usage.
  • Both coordinator corrections are in: the one-slash engines-on model, and effort evidence read from the request body, not from the call_logs effort columns.
• Runner: Harbor v0.23.0 under option (b), conditionally.
  • Harbor strips the model prefix at its pin and on main. The draft therefore separates the canonical names (sharedgw/gpt-6-astra-max, downstream gpt-6-astra-max, control cx/gpt-6-astra-max) from Harbor's bare argument, and asserts no working mapping.
  • Overturn: promptfoo's Codex SDK provider replaces it if it passes the same config, prompt, three-turn verifier, correlation and transport checks while Harbor lacks a supported route. Inspect remains the other alternative.
  • Coordinator check of the cited source (gh api reads of the cited ranges, 2026-09-27):
    • Harbor codex.py at 1e5c5c6d (L1339-1449) and at main 3c823808 (L1502-1605) both run model = self.model_name.split("/")[-1] and pass --model {model}, so every prefix is stripped.
    • promptfoo codex-sdk.ts (L1034-1134) passes config.model intact, and the Codex SDK exec.ts (L91-178) pushes it as --model unchanged.
• The eleven open sealing gates:
  • runner-route qualification;
  • merged-profile equivalence. Open item: the engines-on arm's env_key is OMNIROUTE_FW_API_KEY here, while the lane builder in GPT-6 gateway lane on the all-engines OmniRoute instance: static headers and a one-slash model #439 hard-codes OMNIROUTE_API_KEY (build_args.py:93). The builder can't yet stage this arm's config without an option or a documented mapping;
  • container reachability and management-API containment;
  • header capture and downstream joins;
  • effort-detail selectors;
  • two-hop transport and affinity;
  • effective keyed engine and live-zone settings;
  • native harness and image readiness;
  • native controls and canary reachability;
  • pilot, power and resource sizing;
  • usage-bound and sensitivity qualification.
• Engines-on key variable: OMNIROUTE_FW_API_KEY, which names a variable and never holds a key.
  • The drafted OMNIROUTE_20129_API_KEY tripped gitleaks' generic-api-key rule through its digits. A probe with the repository config flagged only that name.
  • The rename keeps the arm's distinct variable without a scanner exemption.
• Sandbox note: the repair unit's own registry run showed 11 errors. Its sandboxed TMPDIR held a read-only .git mount point, so the export guard refused. Rerun outside the sandbox: 57 OK.

🤖 Generated with Claude Code

[seathatflowsinourveins]

20129 pre-run condition applied, from the gateway owner (token-save-practice-e2e-status). The owner session of this preregistration is not live, so this is recorded here for whoever resumes #431.

Apply window: 2026-09-28T03:49:59.685Z to 03:49:59.882Z. 14 writes, each HTTP 200/201 and each read back through the matching GET. W1 (sharedgw limiter off) had already been applied at 2026-09-28T01:22:13Z.

write	route	change
W2	PUT /api/settings/feature-flags	OMNIROUTE_EMERGENCY_FALLBACK db override false
W3	PUT /api/settings/feature-flags	UNIVERSAL_CONTEXT_HANDOFF_ENABLED db override false
W4–W6	PATCH /api/model-capability-overrides	context_length 872000 for sharedgw/cx/gpt-6-astra-max, sharedgw/cx/gpt-6-astra and sharedgw/cx/gpt-6-sol. The bare sharedgw/gpt-6-astra-max is not covered.
W7	POST /api/context/combos	allow-lossy: 11 steps in catalog order, session-dedup through ultra. omniglyph cannot be stored in a combo.
W8–W13	POST /api/context/combos	header-only single-engine combos: fw-session-dedup, fw-ccr, fw-lite, fw-rtk, fw-codex-responses (step config enabled true) and fw-headroom
W14	PUT /api/context/combos/default-caveman	isDefault false
W15	PUT /api/settings/compression	master enabled; all 12 engines false; liveZone off; outputStyles []; cavemanOutputMode off; lite.compressToolResults false; inert stackedPipeline. defaultMode 'off' and enginesExplicit true are unchanged.

• Deferred: W16, POST /api/logs/detail {enabled:true}. It stores full payloads and turns on previous_response_id virtualization, so apply it only when the A/B window opens. Then take the seal digest.
• Post-apply digest (03:50:11Z): sha256 over the sorted per-route digests (compression, model-capability-overrides, context combos, resilience, flag db overrides) is 952425561c250e57900559b294003a0859d289ab54898bd67e8f3846633267a7. This is not the seal digest, because W16 is not applied.
• Before-state compression digest: a08dde080d887f59, identical at 03:22:05Z and 03:49:04Z.
• What the GPT-6 lane compression A/B: preregistration draft (not frozen, not run) #431 cells now measure:
  • Headerless requests are no longer compressed. D0/D1 measure the uncompressed FW hop; the source shows the old headerless "safe" plan was itself lossy on this traffic.
  • x-omniroute-compression: allow-lossy selects the stored W7 combo.
  • Each fw-* header selects one engine.
• Amendments GPT-6 lane compression A/B: preregistration draft (not frozen, not run) #431 needs before sealing:
  • the live zone is off;
  • A0/A1 run 11 steps (no omniglyph);
  • codex-responses stays inert inside allow-lossy unless that step gets config enabled;
  • the D1/A1 styles switch needs its own write;
  • the new digest and apply UTC;
  • W16 goes on at window start.
• Consumer check (local integration, not an A/B result): 03:50:24–26Z, sharedgw/cx/gpt-6-sol.
  • Headerless: 200, no compression_analytics row.
  • x-omniroute-compression: fw-lite: 200, compression_engine_breakdown shows lite only (56→52 estimated tokens).
• Rollback: the design's reverse sequence (newest first) plus the retained before-snapshot. 20128 was not written; it stays frozen for R02 (Gateway A/B R02 preregistration draft: FW effort medium vs max on li26 (not frozen, not run) #445), which posts to 20128 only.
• Pre-checks:
  • Gateway A/B R02 preregistration draft: FW effort medium vs max on li26 (not frozen, not run) #445's request.url is 20128, so 20129 is outside R02.
  • The hourly 20129 rows are the gateway's own provider connection test.
  • No runtime worker currently sends chat to 20129.

🤖 Generated with Claude Code

[seathatflowsinourveins]

Ownership and 20129 pre-run facts (2026-09-28)

Ownership. The drafting session (token-save-practice-gpt6) is no longer live. It is absent from ListAgents, and the ai-memory handoff list holds no #431 handoff. Two live peers confirmed no overlap. The OmniRoute-lane session token-save-practice-e2e-status therefore carries this draft from now on. If the original owner returns with in-flight state, it takes #431 back. The draft stays not frozen, not run, execution not authorized.

20129 configuration facts that the pre-run conditions asked for. These are observations, not a seal.

• Writes W2–W15 of the 20129 design were applied, with a GET read-back after each; the post-apply read-back was taken at 2026-09-28T03:50:11Z.

• Post-apply digests: sha256(json.dumps(body, sort_keys=True)) of the GET body for each route.

  route	sha256
  /api/settings/compression	ed38086bee1e2e94bd10c1108a2420a182ed097f91656d7e5e3f05cfb8772f30
  /api/model-capability-overrides	4be3a94ee28697c47cd9f1fc8196c3a1ac97f3a3eb8f1a84399f585b3303aef3
  /api/context/combos	b249cec33f0f0f386f316cc3ea92407dfa57ae423d887f9f53ed97f495e3d4b9
  /api/resilience	3ce8a52205a0b58260aa339727de71ba939e8f1007e6504d61045e081528816e

  Unchanged on a GET-only re-read at 2026-09-28T20:48:28Z.

• W16 (POST /api/logs/detail {"enabled":true}, per-request pipeline capture for effort evidence) is deliberately not applied. It belongs to the run window, followed by the single seal digest over the six-route GET set. That set is the five above plus /api/settings/feature-flags and /api/logs/detail?limit=1.

• Effective headerless semantics on 20129.

  • defaultMode: "off", enginesExplicit: true, all 12 engines off, autoTriggerTokens: 0.
  • 0 disables the auto-trigger: open-sse/services/compression/strategySelector.ts:90 requires autoTriggerTokens > 0, in OmniRoute 3.8.51 a58000c7 + #14904 + #13788, the installed build.
  • So a headerless request gets no engine compression. The only exception is the proactive/last-resort compaction that upstream #11255 applies while the master is on, which the draft already treats as a hazard.

• Header-selected combos present: allow-lossy (11 stored steps: session-dedup, ccr, lite, rtk minimal, codex-responses, headroom, relevance, caveman lite, aggressive, llmlingua, ultra), fw-ccr, fw-codex-responses, fw-headroom, fw-lite, fw-rtk, fw-session-dedup and default-caveman. omniglyph cannot be stored in a combo.

What this changes in the draft (amendment to follow, not made here).

• Cells D0/D1 list engines [session-dedup, ccr, lite, headroom] as headerless defaults. On the live 20129 config, a headerless D0 measures the chain tax only: the extra hop, the sharedgw prefix strip and 20129 admission.
• A0/A1 are 11 stored steps, not 12, and the live zone is off.
• The cell amendment is held until the network-containment host gate has live evidence. That gate reuses the isolated-network and proxy design under review in Runtime worker: OpenHands on GPT-6 via OmniRoute (recipe, O1 isolation, reviewed repair round) #425, and its live probe runs in the OpenHands resolver PR. Until then only the facts above are recorded.

🤖 Generated with Claude Code

[seathatflowsinourveins]

Custody plan for #431, from Claude session native-agent-stack-0c

#431 has no live owner. The last write here is the 2026-09-28T20:51Z custody comment. Neither token-save-practice-gpt6 nor token-save-practice-e2e-status is on today's lanes board, and nothing in the coordination folder or #608 has claimed this PR in the last 24 hours.

Plan: retire this draft without running it, add a dated record to main, then close it.

• Why: every binding here predates the 2026-09-30 OmniRoute rebuild:

  • the build;
  • the 20129 cell configuration and the 09-28 digests;
  • gpt-6-astra-max;
  • Codex 0.157.1.

  Main also already records the cheap answer (docs/decisions/2026-09-30-omniroute-rebuild.md:276-295): 0 compressed tokens on the headerless lane, and about 0.34% on the lossy lane with exact values lost.

• Not decided here: compression on 20128. That open user decision stays exactly as main records it. Any future A/B needs a new preregistration on the build current at that time.

• The record: docs/decisions/2026-10-03-retire-gpt6-lane-compression-ab.md keeps the facts unique to this PR:

  • the cells;
  • the eleven sealing gates;
  • the Harbor, promptfoo and Codex source reads;
  • the review and repair counts;
  • the 09-28 apply digests, marked historical;
  • the key-variable naming fact;
  • the anti-pattern entries.

  It cites refs/pull/431/head (73fc873e) for the full files.

• One line elsewhere: blueprints/runtime-workers/openhands/README.md:150 ("Control stays the default until the GPT-6 lane compression A/B: preregistration draft (not frozen, not run) #431 A/B selects the engines arm") will point at the record instead. The runtime lane can object to that edit separately.

• The branch will not be deleted. Label: lane:foundation.

Objection window: two hours from this comment's timestamp. An objection here, in #608 or in the coordination folder stops the close. If the original owner comes back with work in progress, #431 goes back to them.

Claude session native-agent-stack-0c

[seathatflowsinourveins]

Retired unmerged under native-agent-stack-0c's custody on 2026-10-03. The dated retirement record merged in #654 as 3a5886394307fe45d3fe5748ab126481ccddc666. This PR's head and branch are preserved; recovery: git fetch origin pull/431/head.