-
Notifications
You must be signed in to change notification settings - Fork 1.3k
devlog: operator visibility train roadmap unit (260825) #2520
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # 000 — Operator visibility train: baseline, scope, and work-phase map | ||
|
|
||
| Unit opened 2026-08-25. Session `01a03688-c5ee-76c2-bb0f-a7a9213345d5`. | ||
| Goalplan slug `fix-three-opencodex-operator-visibility-defects`. | ||
|
|
||
| ## Baseline | ||
|
|
||
| Verified live at unit open, immediately after the v2.32.1 publish: | ||
|
|
||
| | Ref | SHA | Meaning | | ||
| |-----|-----|---------| | ||
| | `origin/dev` | `bb89eafbe` | devlog: pin the report to the code SHA its gates describe (#2506) | | ||
| | `origin/main` | `71c57ea64` | `release: v2.32.1` | | ||
| | `origin/preview` | `f4cb9f800` | `release: v2.32.1-preview.20260825` | | ||
|
|
||
| `git merge-base --is-ancestor origin/dev origin/main` exits 0, so `dev` is an | ||
| ancestor of the shipped release and this unit starts from published code. | ||
| npm `latest` is `2.32.1`, `preview` is `2.32.1-preview.20260825`. | ||
|
|
||
| ## What this unit is | ||
|
|
||
| Three defects that share one shape: **OpenCodex knows the truth and does not | ||
| tell the operator.** None of them is a routing or execution bug. In all three | ||
| the runtime is already correct and the surface that reports to a human is | ||
| wrong, stale, or silent. | ||
|
|
||
| | # | Surface | The lie | | ||
| |---|---------|---------| | ||
| | #2457 | Management write | The picker offers Gemini, then the save rejects it as an OpenAI model | | ||
| | #2411 | `ocx status` | Green proxy while nothing routes through it | | ||
| | #2412 | Shim auto-restore | A destroyed shim returns an ineligible verdict with no message | | ||
|
|
||
| That shared shape is why they travel together and why none of them may be | ||
| "fixed" by changing behavior. Every fix in this unit is a reporting fix. | ||
|
Comment on lines
+33
to
+34
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For the inputs documented in this unit, these fixes do change behavior: Useful? React with 👍 / 👎.
Comment on lines
+22
to
+34
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win Separate the validation fix from the reporting-only fixes. The inventory states that a valid Revise the scope statement so the reporting-only constraint applies to the status and shim fixes. State explicitly that WP2 may correct management validation while preserving executor behavior. Proposed wording adjustment-Three defects that share one shape: **OpenCodex knows the truth and does not
-tell the operator.**
+Three operator-visibility defects share a common diagnostic theme. WP2 also
+corrects stale management validation for valid backend submissions.
...
-Every fix in this unit is a reporting fix.
+WP3 and WP4 are reporting fixes. WP2 corrects the management write gate while
+preserving the existing executor behavior.🤖 Prompt for AI Agents |
||
|
|
||
| ## Work-phase map | ||
|
|
||
| | Phase | Doc | Issue | Deliverable | | ||
| |-------|-----|-------|-------------| | ||
| | WP1 | this unit | — | Docs-only roadmap at diff-level precision | | ||
| | WP2 | `010` | #2457 | Submitted backend is what the pair check validates | | ||
| | WP3 | `020` | #2411 | `ocx status` prints routing and warns on unused proxy | | ||
| | WP4 | `030` | #2412 | Version-manager shim destruction is detected and reported | | ||
|
|
||
| One work-phase is one full PABCD cycle. WP2, WP3, and WP4 each produce one PR | ||
| against `dev`. | ||
|
Comment on lines
+45
to
+46
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The reviewed commit says that #2514, #2518, and #2519 have already landed, and the inspected AGENTS.md reference: AGENTS.md:L75-L78 Useful? React with 👍 / 👎. |
||
|
|
||
| ## Scope boundary | ||
|
|
||
| Out of scope, stated once so no later phase reopens it: | ||
|
|
||
| - Merging other contributors' PRs, or another npm release. | ||
| - `src/lab/` — the core-lab boundary test exists for a reason. | ||
| - The undeclared-tool guard, and any auth, OAuth, credential, workflow, or | ||
| release-automation surface. | ||
| - Auto-wrapping a version-manager-owned `codex` binary as a new original. | ||
| This is the one that is tempting and wrong; see `030`. | ||
| - The Codex-side namespaced-model error message in #2411's reproduction. That | ||
| is upstream copy, not ours. | ||
|
|
||
| ## Evidence rule | ||
|
|
||
| A remembered pass is not evidence. Every completion claim in this unit carries | ||
| exact command output, the PR number and head SHA, and the CI run id and | ||
| conclusion on that SHA. | ||
|
|
||
| ## Prior art consulted | ||
|
|
||
| - `260824_v2_32_1_hotfix_train/` — the freeze/GO discipline this unit inherits. | ||
| - `tests/repo-hygiene.test.ts` — no gitlinks, no vendored clones. | ||
| - `AGENTS.md` — focused checks during implementation, full suite before a | ||
| non-trivial PR goes review-ready. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,130 @@ | ||
| # 001 — Current-state inventory | ||
|
|
||
| Read at `bb89eafbe`. Every line anchor below was opened and read, not inferred. | ||
|
|
||
| ## #2457 — the pair check discards the union | ||
|
|
||
| The accepted union is complete. `src/server/management/config-routes.ts:591`: | ||
|
|
||
| ```ts | ||
| const WEB_SEARCH_BACKENDS_UNION = ["openai", "anthropic", "xai", "gemini", "exa"] as const; | ||
| ``` | ||
|
|
||
| The pair check nineteen lines later throws it away. `config-routes.ts:668`: | ||
|
|
||
| ```ts | ||
| const effectiveBackend = body.webSearch.backend === "anthropic" | ||
| ? "anthropic" | ||
| : body.webSearch.backend === "openai" || body.webSearch.backend === null | ||
| ? "openai" | ||
| : config.webSearchSidecar?.backend ?? "openai"; | ||
| ``` | ||
|
|
||
| A submitted `"gemini"` is not `"anthropic"`, not `"openai"`, not `null`. | ||
| It falls to the final arm and the request is validated against the **stored** | ||
| backend. With stored `openai` (or unset), `webSearchModelIsRejected("openai", | ||
| "gemini-3.7-flash", candidates)` is true, and the route returns 400 before the | ||
| persistence block at `:687` — which does honor the full union — ever runs. | ||
|
Comment on lines
+23
to
+27
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- target files ---'
git ls-files | rg '(^|/)(001_current_state_inventory|002_plan_audit)\.md$|(^|/)config-routes\.ts$|(^|/)shim\.ts$'
printf '%s\n' '--- commit summary ---'
git show --no-patch --format='%H%n%s' bb89eafbe
printf '%s\n' '--- inventory references ---'
rg -n -C 5 'config-routes\.ts|shim\.ts|persistence|destroyed-shim|preserveOnly|2043|2045|687|688' devlog/_plan/260825_operator_visibility_train/001_current_state_inventory.md devlog/_plan/260825_operator_visibility_train/002_plan_audit.md
printf '%s\n' '--- source files at bb89eafbe ---'
for path in $(git ls-tree -r --name-only bb89eafbe | rg '(^|/)config-routes\.ts$|(^|/)shim\.ts$'); do
echo "### $path"
git show "bb89eafbe:$path" | nl -ba | sed -n '675,700p;2030,2055p'
doneRepository: lidge-jun/opencodex Length of output: 13431 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- plan files at bb89eafbe ---'
for path in $(git ls-tree -r --name-only bb89eafbe | rg 'devlog/_plan/260825_operator_visibility_train/(001_current_state_inventory|002_plan_audit)\.md$'); do
echo "### $path"
git show "bb89eafbe:$path" | nl -ba | sed -n '15,115p'
done
printf '%s\n' '--- exact source matches at bb89eafbe ---'
for path in $(git ls-tree -r --name-only bb89eafbe | rg '(^|/)config-routes\.ts$|(^|/)shim\.ts$'); do
echo "### $path"
git show "bb89eafbe:$path" | rg -n -C 8 'webSearchModelIsRejected|preserveOnly|destroyed|persistence|persist|backend'
doneRepository: lidge-jun/opencodex Length of output: 28168 Align the source anchors across the plan documents. At Update all related plan references, including lines 23–27 and 97–103, to use these precise anchors. 🤖 Prompt for AI Agents |
||
|
|
||
| `src/server/management/agent-settings-routes.ts:1121` carries the same stale | ||
| ternary with a different null policy: | ||
|
|
||
| ```ts | ||
| const effectiveBackend = section.backend === "anthropic" | ||
| ? "anthropic" | ||
| : section.backend === "openai" | ||
| ? "openai" | ||
| : section.backend === null | ||
| ? config.webSearchSidecar?.backend ?? "openai" | ||
| : stored?.backend ?? config.webSearchSidecar?.backend ?? "openai"; | ||
| ``` | ||
|
|
||
| The comment directly above that block reads: *"Same module as | ||
| /api/sidecar-settings — a gate on one route and a stale copy on the other is no | ||
| gate at all."* The gate is shared; the backend resolution is not, and it drifted | ||
| exactly as the comment feared. | ||
|
|
||
| `xai` and `exa` have the identical hole. They escape notice because | ||
| backend-only submissions short-circuit on `effectiveModel` being empty. | ||
|
|
||
| The executor is already correct and must not be touched: | ||
| `resolveSidecarBackend("gemini")` returns `"gemini"` | ||
| (`src/web-search/index.ts:162`), and `planWebSearch` already defaults Gemini to | ||
| `gemini-3.7-flash` (`:285`). Writing the pair directly into `config.json` | ||
| works today, which is the reporter's own proof that only the write gate is wrong. | ||
|
|
||
| ## #2411 — status has the routing kind and never prints it | ||
|
|
||
| `collectStatus()` already computes it. `src/cli/status.ts:188`: | ||
|
|
||
| ```ts | ||
| const startup = collectStartupHealth(config, { | ||
| service, | ||
| shim: codexShim, | ||
| routingKind: getCodexRoutingKind(), | ||
| }); | ||
| ``` | ||
|
|
||
| `startup` lands on `json.startup` at `src/cli/status.ts:316`, so | ||
| `ocx status --json` **already exposes** `startup.routingKind`. The human | ||
| renderer is what drops it. `src/cli/index.ts:845`: | ||
|
|
||
| ```ts | ||
| if (status.json.proxy.pid || status.json.proxy.health.ok) { | ||
| console.log(`✅ Proxy: ${status.proxyLabel}`); | ||
| } | ||
| ``` | ||
|
|
||
| That boolean never consults `startup.routingKind`. A live PID or a good | ||
| `/healthz` is sufficient for the green check. | ||
|
|
||
| Worse, the next line reinforces it. `startupHealthSummary` | ||
| (`src/codex/autostart-health.ts:143`) renders native routing as *"native Codex | ||
| routing (no opencodex restart dependency)"*, and `deriveStartupHealth` marks it | ||
| `rebootSafe: true`. That is correct on its own terms — there is genuinely no | ||
| restart dependency when nothing routes — but printed under a green proxy it | ||
| reads as a second all-clear. | ||
|
|
||
| `ocx doctor` already prints the missing token. `src/cli/doctor.ts:986`: | ||
|
|
||
| ```ts | ||
| console.log(` routing=${startup.routingKind}, service=${...}, shim=${...}`); | ||
| ``` | ||
|
|
||
| So the fix is not new computation. It is routing the value that already exists | ||
| to the surface people actually run. | ||
|
|
||
| ## #2412 — the ineligible verdict carries no message | ||
|
|
||
| `src/codex/shim.ts:2043`: | ||
|
|
||
| ```ts | ||
| if (!existsSync(file.wrapperPath) || !hasUsableBackingPath(file)) return { status: "ineligible" }; | ||
| ``` | ||
|
|
||
| No `message` field. That is why the condition is invisible: the CLI warns only | ||
| when one exists. `src/cli/codex-shim-autorestore.ts:35`: | ||
|
|
||
| ```ts | ||
| } else if ((result.status === "deferred" || result.status === "ineligible") && result.message) { | ||
| deps.warn(`⚠️ ${result.message}`); | ||
| } | ||
| ``` | ||
|
|
||
| A mise/asdf/volta upgrade rewrites the install tree in place, destroying both | ||
| `codex` and its sibling `codex.opencodex-real` (`backupPathFor`, | ||
| `src/codex/shim.ts:601`). `hasUsableBackingPath` (`:481`) then returns false, | ||
| the silent ineligible fires, and `ocx start` / `ocx ensure` / | ||
| `ocx service repair` all proceed to report success. | ||
|
|
||
| `diagnoseCodexShim` (`src/codex/shim.ts:2156`) already produces the exact | ||
| diagnostic string the reporter pasted. The information exists; nothing routes it | ||
| to the commands that matter. | ||
|
|
||
| ## The common root | ||
|
|
||
| In all three, the correct value is computed and then discarded on the way to the | ||
| human: a validated union collapsed into a two-arm ternary, a routing kind | ||
| carried in JSON but not printed, a diagnosis produced by one command and absent | ||
| from three others. None of the three fixes changes what OpenCodex does. They | ||
| change what it admits. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| # 002 — Plan audit (A phase, WP1) | ||
|
|
||
| The dispatched read-only auditor produced nothing across four wait cycles and | ||
| was retired under the loop's failed-dispatch rule. The audit below was performed | ||
| directly by the main agent against source at `bb89eafbe`. Every anchor cited in | ||
| `001`, `010`, `020`, and `030` was re-opened and confirmed. | ||
|
|
||
| ## Anchor verification | ||
|
|
||
| | Doc claim | Verified | | ||
| |-----------|----------| | ||
| | `config-routes.ts:591` union of five backends | yes, exact | | ||
| | `config-routes.ts:668` two-arm ternary falling back to stored | yes, exact | | ||
| | `config-routes.ts:688` persistence honors the full union | yes | | ||
| | `agent-settings-routes.ts:1121` five-arm ternary | yes | | ||
| | `cli/status.ts:188` computes `routingKind` | yes | | ||
| | `cli/status.ts:316` `startup` lands in JSON | yes | | ||
| | `cli/index.ts:845` green check ignores routing | yes | | ||
| | `cli/doctor.ts:986` prints `routing=` | yes | | ||
| | `shim.ts:481` `hasUsableBackingPath` | yes | | ||
| | `shim.ts:1887` `allowFreshInstall` guard | yes | | ||
| | `cli/codex-shim-autorestore.ts:35` warns only with a message | yes | | ||
| | `autostart-health.ts:143` `startupHealthSummary` | yes | | ||
|
|
||
| One correction: `030` cites the destroyed-shim bail as `shim.ts:2043`. The | ||
| actual line is **`2045`**; `2043` is inside the `preserveOnly` branch. The | ||
| quoted code is right, the number is off by two. | ||
|
|
||
| ## Blocking findings | ||
|
|
||
| **A1 — `030` targets only one of six `ineligible` returns.** | ||
| `rg 'status: "ineligible"' src/codex/shim.ts` finds returns at `2028`, `2031`, | ||
| `2039`, `2042`, `2045`, `2049`, and `2085`. Only `2028` and `2085` carry a | ||
| message today. The plan attaches one to `2045`, but `2042` is the | ||
| `preserveOnly` sibling case and `2049` is `isHealthyShimProbe` — both are | ||
| reachable in a version-manager overwrite and both would stay silent. | ||
|
|
||
|
Comment on lines
+31
to
+37
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Correct the The audit lists seven total returns: If “six” refers to a filtered reachable subset, define that filter and identify the excluded return. Otherwise correct the denominator so WP4 coverage cannot miss a branch. 🤖 Prompt for AI Agents |
||
| Correction: WP4 must attach messages to the reachable silent returns, not just | ||
| the one the reporter happened to hit. The `preserveOnly` branch at `2042` | ||
| deserves its own wording — its condition is a missing backup **or** a resurrected | ||
| original, which is a different story from a destroyed wrapper. | ||
|
|
||
| **A2 — `020`'s truth table omits `custom-local` and `unknown`.** | ||
| `CodexRoutingKind` (`inject.ts:314`) has five members. The table covers | ||
| `opencodex-local`, `native`, and `custom-remote`. The predicate as written | ||
| returns `[]` for `custom-local` and `unknown`, which is the correct behavior — | ||
| `startupHealthSummary` already renders both as `AT RISK after restart` with a | ||
| remedy command (`autostart-health.ts:149-150`), so a second warning would be | ||
| noise. But the plan does not say so, and a later reader could "fix" the omission. | ||
|
|
||
| Correction: state the five-member coverage explicitly and record that | ||
| `custom-local`/`unknown` are intentionally silent **because** they are already | ||
| loud elsewhere. Add both to the helper's test cases so the intent is pinned. | ||
|
|
||
| ## Non-blocking findings | ||
|
|
||
| **B1 — `010`'s cast.** `WEB_SEARCH_BACKENDS_UNION.includes(x as ...)` does not | ||
| narrow `x` in TypeScript; `includes` returns `boolean`, not a type predicate. | ||
| The proposed `submittedBackend as typeof WEB_SEARCH_BACKENDS_UNION[number]` | ||
| cast in the true arm is therefore load-bearing, not decorative. It is sound | ||
| because `:591` already rejected non-members, but the doc should say that the | ||
| cast is doing real work rather than reading as noise. | ||
|
|
||
| **B2 — `webSearchModelIsRejected`'s `backend` parameter type.** If it is typed | ||
| as the narrow union, passing the widened value type-checks only because both | ||
| resolve to the same union. Confirm at implementation time; if it is narrower, | ||
| the signature is the thing to widen, not the call site to cast. | ||
|
|
||
| **B3 — line-number drift.** `030` says `2043`, actual `2045`. Corrected in this | ||
| document rather than by rewriting `030`, so the drift stays visible. | ||
|
|
||
| ## Verified correct | ||
|
|
||
| - The #2457 mechanism, end to end: union at `:591`, ternary at `:668`, | ||
| persistence at `:688`. A submitted `gemini` provably reaches the stored-backend | ||
| arm. | ||
| - Both null policies genuinely differ between the two routes. `010`'s refusal to | ||
| unify them is right. | ||
| - `startup.routingKind` is already in `status --json`. `020`'s claim that no | ||
| schema change is needed holds. | ||
| - `allowFreshInstall: false` at `1887` is the invariant that blocks adoption. | ||
| `030`'s refusal to relax it is correct, and it is what makes A1 a | ||
| message-plumbing fix rather than a behavior change. | ||
|
|
||
| ## Verdict | ||
|
|
||
| **PASS with two required amendments.** A1 and A2 are corrections to WP4 and WP3 | ||
| scope respectively; neither invalidates the plan's shape, and both are folded | ||
| into this document rather than silently patched into the originals. B1–B3 are | ||
| notes for the implementer. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Reconcile the issue IDs with the PR objective.
The PR objective names
#2514,#2518, and#2519, while this roadmap and the WP2 document use#2457,#2411, and#2412. Add an explicit mapping if these are migrated issue IDs; otherwise update the roadmap headings and references before merge so implementation work cannot be associated with the wrong tracking items.📍 Affects 2 files
devlog/_plan/260825_operator_visibility_train/000_baseline_and_scope.md#L27-L32(this comment)devlog/_plan/260825_operator_visibility_train/010_wp2_issue2457_sidecar_backend_resolution.md#L1-L1🤖 Prompt for AI Agents