Gateway A/B R02 preregistration draft: FW effort medium vs max on li26 (not frozen, not run)

[seathatflowsinourveins]

DRAFT preregistration: not frozen and not run. This is gateway A/B R02 for GPT-6 framework (FW) chat/completions effort through OmniRoute:

• arm A: cx/gpt-6-astra with no reasoning_effort, the gateway's default of medium;
• arm B: cx/gpt-6-astra-max, which gets max from the executor's built-in -max suffix alias.

The two arms run on a frozen 60-filing subset of li26's 8-K extraction. The GPT-6 reviewer found the draft freeze-ready once the gateway owner's dry read-back passes; the freeze follows that read-back.

Scope

• What this PR changes: it adds the R02 preregistration draft under blueprints/convergence-practice/gateway-ab-r02-20260927/:

  • plan.json;
  • the promptfoo configs;
  • the build, analysis, correlation and assertion scripts;
  • the README.

  It also adds the offline tests tests/test_gateway_ab_r02_20260927.py (80).

• Base commit: 37739d32 (merged into the branch).

• Lane: lane:foundation.

• Owned paths touched:

  • blueprints/convergence-practice/gateway-ab-r02-20260927/**;
  • tests/test_gateway_ab_r02_20260927.py;
  • manifests/evidence.json (registration only).

SOTA sources

• promptfoo 0.123.1 (tag commit 34f74d34):
  • the HTTP provider with maxRetries: 0;
  • the evaluator's pairing by testIdx (src/evaluator.ts:2674-2696);
  • is-json with the exact schema that was sent (dist/src/evaluator-DlYW7Rgb.js:2509-2545);
  • the timeouts PROMPTFOO_EVAL_TIMEOUT_MS and REQUEST_TIMEOUT_MS;
  • the results sanitizer and its compaction of JSON-looking vars (dist/src/logger-ChlKG5Wv.js:1049-1057).
• OmniRoute 3.8.51 at dd6e9607e:
  • the -max suffix alias (open-sse/executors/codex.ts:1424-1431) and the no-effort fallback (:1439-1441);
  • reasoning-routing rules (src/lib/reasoningRouting/policy.ts:380-404, 543-601);
  • combo mappings (src/sse/services/model.ts:686-711);
  • compression exclusions (open-sse/handlers/chatCore.ts:1429-1449);
  • call_logs saves (callLogs.ts:867-896, attemptLogging.ts:618).
• scipy 1.18.1 stats.bootstrap with paired=True (scipy/stats/_resampling.py:300, 651-683) and permutation_test (:1679), in the pinned analysis environment (Python 3.14, numpy 2.5.3, statsmodels 0.15.0).
• python-jsonschema 4.26.0 enum equality (jsonschema/_utils.py:106-153, _keywords.py:269-271), ported for the admissibility check.
• li26, the frozen scorer and acquisition (blueprints/convergence-practice/local-inference-latest-20260926/, Preregister the GPT-6 family routing test (astra/sol/luna x max/medium) on the frozen 8-K extraction task #359 and GPT-6 family tiering: record the frozen run's results (S1, gpt-6-sol at medium, for mechanical extraction) #397).

Design

• Subset. 60 of li26's 360 labelled filings, stratified by label count and split between an AB-order run and a BA-order run of 30 filings each. Each filing runs three repeats per arm, for 360 calls in all.
• Decision rule. All bounds are one-sided 95% from the paired bootstrap on filings.
  • Adopt max if the lower bound of micro-F1(B) − micro-F1(A) is above 0.
  • Otherwise keep medium if A is non-inferior within 0.02 and the paired per-filing cost leg is supported and shows A's completion tokens below B's.
  • Otherwise inconclusive.
  • Any integrity problem voids the run (analyze_r02.py exits 2).
• Gateway fingerprint. Four sanitized read-backs by the gateway owner, one before and one after each run half. Each value must equal one of the admissible values declared in plan.json gateway_fingerprint.admissible. Identical read-backs of the wrong treatment still void the run.

Evidence-class table

Claim	Evidence class	Command / receipt
The offline suite passes in the pinned analysis environment: 80 tests, none skipped.	synthetic (our integration checks on fixtures; two of them run the installed promptfoo 0.123.1 code offline)	uv run --offline --no-project --python 3.14 --with numpy==2.5.3 --with scipy==1.18.1 --with statsmodels==0.15.0 python -m unittest tests.test_gateway_ab_r02_20260927
The generated configs, frozen-file closure and promptfoo build record match.	local_integration	build_r02.py configs --check, frozen --check and promptfoo --check, each exit 0
Request shape through the running gateway: 4 calls in two wire checks at 19:49 and 21:16 UTC on 2026-09-27.	native_proven (bounded wire checks; not quality or cost)	README "Wire checks (2026-09-27)"
Rendered requests: a promptfoo execution against a local go-httpbin, with no gateway call.	local_integration	README "Echo check" (configs at 9d03c1ea)
Cited OmniRoute behaviour.	source_review (pinned commit object dd6e9607e; the running build is the gateway owner's report)	README "Evidence classes"
No scored call has been made.	—	—

Local commands run

$ python3 scripts/validate.py
{"components": 69, "hashed_files": 7384, "profiles": 4, "receipts": 159, "status": "passed"}
$ GIT_NO_LAZY_FETCH=1 python3 scripts/host_receipts.py validate
{"receipts": 182, "status": "passed"}
$ uv run --offline --no-project --python 3.14 --with numpy==2.5.3 --with scipy==1.18.1 --with statsmodels==0.15.0 python -m unittest tests.test_gateway_ab_r02_20260927
Ran 80 tests ... OK
$ python3 blueprints/convergence-practice/gateway-ab-r02-20260927/build_r02.py {configs,frozen,promptfoo} --check
exit 0 (each)

Independent review

All three rounds were GPT-6 (gpt-6-astra at max) on one Codex thread, read-only, with no gateway, model or network call.

1. Review of fae72c37: not freeze-ready, 8 findings. They were repaired in f58a4a65 and 4b7c58de; the builder also found and fixed a ninth defect, promptfoo's compaction of JSON-looking vars.

2. Re-check of 4b7c58de: not freeze-ready, 4 findings (P1, P2, P2, P3). They were repaired in 2c584098 and af85abec.

3. Focused re-check of af85abec: all four resolved, both widened wordings accepted, and no new P0–P3 defect. Verdict: "freeze-ready pending the owner's dry read-back".

   Its sandbox lacked numpy and scipy, so it ran 76 tests and skipped the 4 statistical ones. All 80 pass above in the pinned environment.

Residuals recorded in plan.json and the README:

• the admissible values are declared intent or documented values, not observed ones;
• the private server.env can override the timeouts;
• the read-backs are the gateway owner's report, not an independent observation;
• the call_logs snapshots are observed lower bounds;
• whole-call duration is not recorded.

After the re-check (884224e0, documentation only), the gateway owner reported 20128's OAuth occupancy override. At OmniRoute dd6e9607e, every chat request sets reserveOAuthSession: true (src/sse/handlers/chat.ts:1711), and the override moves a call to a more available peer connection (src/sse/services/auth.ts:2159-2176). No setting disables it.

• It is recorded as a shared account-selection condition in plan.json limitations and the README. The owner's measured rates are marked relayed.
• The owner's post-run report now also gives, per arm, the calls whose attempts were served by more than one account.
• No decision rule, integrity check or frozen input changed. The gateway_fingerprint block is byte-identical to the re-checked one.

Before freezing

1. The gateway owner runs the dry read-back:
   • read plan.json gateway_fingerprint.fields from 20128 into one sanitized JSON file, with no account or connection id, key or token;
   • run python3 blueprints/convergence-practice/gateway-ab-r02-20260927/analyze_r02.py readback <file>. Exit 0 means every field is admissible.
2. Freeze with build_r02.py frozen --update-plan, then set the status to frozen, update the tests and the README, and do the hot-file step.
3. Post the frozen hashes on Preserve OmniRoute feature research and settle October 3 remeasurement promises #423.

Decision record

The dated design record is blueprints/convergence-practice/gateway-ab-r02-20260927/README.md, with plan.json as its machine-readable form. It records the alternatives (the arms, and why moving roles to cx/gpt-6-sol is outside them) and the decision rule's overturn conditions.

Checklist

• No GitHub Actions or workflows changed.
• No secrets printed, logged or committed. The configs send no Authorization header to the keyless loopback gateway.
• No paid hosting, subscription or billing surface was added.
• Peer-owned files and worktrees were preserved.

🤖 Generated with Claude Code

[seathatflowsinourveins]

Custody notice for #445 from session native-agent-stack-0c.

No live owner found. This draft has had no comment or review since 2026-09-28T01:28Z, and no lane claims it in the coordination folder or on #608.

It cannot run as written. Its arms and admissible gateway-fingerprint values are bound to OmniRoute build dd6e9607e and cx/gpt-6-astra. The 2026-09-30 rebuild restarted 20128 with the affinity patch 045aa81f3. That ended the R02 freeze in fact (decisions.json r02-freeze-20128), but the rebuild record never names R02.

Plan:

1. Retire it with a dated record on main, docs/decisions/2026-10-03-retire-gateway-ab-r02.md. The record keeps the design, subset hashes, decision rule, wire, echo and read-back history, the three review rounds and the residuals. It states that no explicit user release of the freeze is on record, and it releases nothing.
2. Add append-only addenda to the omniroute-routing README and the 2026-09-28 roadmap.
3. Close this PR.

The branch stays, because main cites 00a94b5 and bddb007. There will be no gateway call and no host change.

Objection window: 2 hours from this comment. The user, the OmniRoute gateway owner (20128/20129) and the Codex token lane 01a0ffbf-211c in particular: object or claim R02 here, and it becomes a user decision instead.

Claude session native-agent-stack-0c

[seathatflowsinourveins]

Retired unmerged under the stale-PR custody of Claude session native-agent-stack-5f (formerly -0c). The dated retirement record merged in #656 as fdd0a247aee96f21c4eff309bacdb3af322eb873. This PR's head and branch are preserved; recovery: git fetch origin pull/445/head.