fix: surface real codex errors, seed a ChatGPT-auth model, show step failures inline

CHANGELOG.md

@@ -16,8 +16,15 @@ All notable changes to Agrippa are documented here. The format follows
   - *Role-scoped, single-provider slot resolution.* Slots resolve only the model roles their steps (and subagents) reference, and provider-constrained slots resolve all of them from one provider (credentialed provider first, then cheapest, deterministic) — a step's base URL is process-wide, so mixed-provider slots could never execute. A slot blocked only by a missing credential fails submission with the new `provider_credential_required` error — retries re-assert the same gate against the frozen resolution, and a credential deleted mid-run fails the run with the same code before the executor is invoked; demo/`fake` resolution is unchanged.
   - *Keyless workers defer instead of failing.* Executors advertise which providers their worker env can authenticate (`envAuthProviders`); a run whose providers neither a project credential nor the claiming worker's env covers is declined before any status transition (the existing `run.deferred` heterogeneous-fleet path), so it waits for a capable worker rather than burning mid-run. The decline applies only to the pre-claim states the worker can actually re-enqueue (`queued`/`waiting_approval`) — a crash-recovered running run proceeds and, if its auth is truly unusable (including an env-policy credential deleted after claim on a keyless worker), fails actionably per-step with `provider_credential_required` instead of a generic error. Preflight checks only credential-row/secret presence; endpoint validation and decryption happen after claim, where bad configuration fails actionably and infrastructure errors retry.
 
+- **`gpt-5.6-sol` joins the seeded model catalog** — the first seeded OpenAI model that works under a ChatGPT-account Codex login (`CODEX_HOME`); the existing `gpt-*-codex` ids are API-key-only and the backend 400s them under account auth, which left ChatGPT-login deployments with no runnable reviewer model. Context window and pricing follow OpenAI's model catalog (1,050,000 tokens, $5.00/$30.00 per MTok); a new CONTRIBUTING recipe covers adding builtin models.
+
 ### Fixed
 
+- **Codex failures now surface their real cause** — a live requirement-delivery run died with the useless "codex reported an error" while the backend had said exactly what was wrong ("The 'gpt-5.1-codex' model is not supported when using Codex with a ChatGPT account"):
+  - *Error items carry `message`, not `text`.* codex-cli 0.145 puts error-item text in `message`; the mapper read only `text`, dropping the detail. It now reads both.
+  - *Warnings no longer mask or fabricate failures.* The CLI emits non-fatal error items ("Model metadata … Defaulting to fallback") and then completes the turn fine; first-error-wins let that warning shadow the fatal `turn.failed`, and any recorded error failed the step even on a clean exit. Fatal stream errors (`turn.failed` / top-level `error`, JSON blobs unwrapped to the inner sentence) and the exit code now decide failure; item errors only refine the message, and a step that completed with a lone warning succeeds (logged, not failed).
+  - *Silent deaths get a usable message.* `stderr ?? fallback` never fired (stderr is always a string), so a CLI that died quietly produced an empty error message; failure paths also now log stderr through the executor logger.
+  - *The timeline shows the failure where it happened.* `step.failed` only flipped a red icon on the turn — the error text appeared solely in the run-level banner, and only once the whole run had failed (a retried-then-recovered attempt's message was never shown anywhere). Each attempt's turn now renders its own `code: message` detail inline.
 - **Fourth review round (PR #5)** — the evidence boundary now excludes all agent-writable Git metadata:
   - *Platform Git owns a complete private gitdir and index.* Checkout gives the agent a copy of `.git` while the pristine original moves to a sibling sidecar. Evidence and publication rebuild the sidecar index from the trusted clone base and explicitly exclude `.claude`, `.mcp.json`, and `.agrippa`, so a config symlink/special file or agent edits to the index and `info/exclude` cannot alter the platform view. This supersedes the third round's config-restore design.
   - *Publication is one verified snapshot commit.* The platform compares stored and current patch evidence without truthiness shortcuts, treats Git failures as errors, refuses empty snapshots, stages once more inside the SCM operation, and publishes one idempotent Agrippa-authored commit whose tree matches the approved patch. Agent-local commits remain working checkpoints but are intentionally not preserved in the PR history (ADR-0012).

CONTRIBUTING.md

@@ -51,6 +51,12 @@ bun run build                # SPA build
 3. Add the template head + task type rows to `packages/db/src/seed/index.ts` (slug, i18n names, default Faber, scenario).
 4. Boot the api (or `bun run db:seed && bun run templates:seed`) — the template publishes automatically; the submission form appears with zero frontend work.
 
+### Add a builtin model
+
+1. Append a row to `modelRows` in `packages/db/src/seed/index.ts` — every field: `provider`, `providerModelId`, `displayName`, `tier`, `contextWindow`, `inputCostPerMtok`, `outputCostPerMtok` (list pricing; the budget meter charges from these). Verify the id against the live provider first — ids can be auth-mode-dependent (e.g. the `gpt-*-codex` ids 400 under a ChatGPT-account Codex login).
+2. `bun run db:seed` (or boot the api) — inserts are idempotent by provider model id.
+3. Grant it to projects that should use it (Settings → Resources). Org-specific one-offs can skip the seed entirely: admins add rows at `/admin/models`.
+
 ### Add a builtin skill
 
 1. Create `templates/_shared/skills/<name>/SKILL.md` (frontmatter: `name`, `description`).

apps/web/src/features/runs/RunTimeline.tsx

@@ -35,6 +35,7 @@ type TurnItem = {
   tools: string[];
   done: boolean;
   failed: boolean;
+  error: { code: string; message: string } | null;
 };
 
 type TimelineItem =
@@ -87,6 +88,7 @@ function buildTimeline(
           tools: [],
           done: false,
           failed: false,
+          error: null,
         };
         openTurns.set(stepKey, turn);
         items.push(turn);
@@ -120,6 +122,12 @@ function buildTimeline(
         if (turn) {
           turn.done = true;
           turn.failed = true;
+          // each retry attempt re-opens its own turn, so this failure detail
+          // stays attached to the attempt that produced it
+          const error = p.error as { code?: string; message?: string } | undefined;
+          turn.error = error
+            ? { code: String(error.code ?? "error"), message: String(error.message ?? "") }
+            : null;
         }
         break;
       }
@@ -248,6 +256,12 @@ function TurnBlock({ turn, run }: { turn: TurnItem; run: Run }) {
             {shown}
           </pre>
         ) : null}
+        {turn.error ? (
+          <p className="rounded-md border border-destructive/40 bg-destructive/5 px-2.5 py-1.5 text-xs text-destructive">
+            {turn.error.code}
+            {turn.error.message ? `: ${turn.error.message}` : ""}
+          </p>
+        ) : null}
         {collapsed || (turn.done && turn.text.length > 600 && expanded) ? (
           <button
             type="button"

docs/design/03-executor-abstraction.md

@@ -127,6 +127,7 @@ All executor work happens in the **worker container** (`apps/worker`), one run p
 | Config isolation | `--ignore-user-config` (no `~/.codex/config.toml` MCP servers riding in; auth still uses `CODEX_HOME`) + `--ignore-rules`; the boot probe refuses CLIs lacking these flags |
 | Streaming | `thread.started` → `step.started{sessionId}`; `item.*` agent messages / command executions → `message.completed` / `tool.*` |
 | Usage | `turn.completed.usage` — `input_tokens` is cached-inclusive and split into `inputTokens`/`cacheReadTokens` |
+| Errors | fatal = `turn.failed` / top-level `error` (JSON-blob messages unwrapped to the inner sentence) or a nonzero exit; item-level `error` items carry `message` and are often non-fatal warnings ("Model metadata … Defaulting to fallback") — they refine the failure message but never fail a completed turn |
 | Resume | `codex exec resume <thread_id>` (same-step crash recovery only, per ADR-0005) |
 | Env | `buildScrubbedEnv` (OPENAI_API_KEY/CODEX_API_KEY on the provider-auth allow-list) |
 

docs/design/09-testing-and-ci.md

@@ -40,6 +40,11 @@ Auth flows, RBAC allow/deny matrix per role × endpoint class, transactional tas
 - Unit: mocked SDK `query()` asserting the full option-mapping table from [03](03-executor-abstraction.md) (subagents, skills materialization path, MCP config, resume); the isolation policy (`evaluateToolCall` — Bash and write containment, read-only enforcement, boundary-safe check) and env scrubbing; contract-scoped artifact collection (patch and uncontracted files skipped).
 - One live smoke test behind `ANTHROPIC_API_KEY`, excluded from CI, run manually before releases.
 
+### Codex executor
+
+- Unit, against a fake-CLI fixture replaying JSONL shapes pinned to codex-cli 0.145.0: stream→event mapping (one terminal, cache-split usage), argv/sandbox/resume construction, prompt assembly, read-only artifact synthesis, env scrubbing and project-credential overlay (`CODEX_HOME` redirect), and error semantics — a fatal `turn.failed`/`error` surfaces the unwrapped backend message, a non-fatal error item never fails a completed turn, and a silent CLI death still yields a non-empty message.
+- CLI registration is gated on `probeCodexCli` (version + config-isolation flags); no live smoke in CI.
+
 ### Worker adapters
 
 - `DiskArtifactStore` path containment: normal file stored, escaping symlink rejected, missing source is not a zero-byte artifact.

packages/db/src/seed/index.ts

@@ -363,7 +363,9 @@ export async function seed(db: Db): Promise<void> {
     },
     // OpenAI models for the codex-cli executor (requirement-delivery reviewer
     // slot). Verify current ids/pricing against the OpenAI price list when
-    // rolling out — admins can adjust rows in the registry.
+    // rolling out — admins can adjust rows in the registry. Model ids are
+    // auth-mode-dependent: the gpt-*-codex ids serve API-key auth but 400
+    // under a ChatGPT-account login (CODEX_HOME), which gpt-5.6-sol serves.
     {
       provider: "openai",
       providerModelId: "gpt-5.1-codex",
@@ -382,6 +384,18 @@ export async function seed(db: Db): Promise<void> {
       inputCostPerMtok: "0.25",
       outputCostPerMtok: "2.00",
     },
+    {
+      // codex-cli 0.145's default model; verified working under ChatGPT-account
+      // auth (the gpt-*-codex ids above are rejected there). Context/pricing
+      // from OpenAI's model catalog (developers.openai.com/api/docs/models).
+      provider: "openai",
+      providerModelId: "gpt-5.6-sol",
+      displayName: "GPT-5.6 Sol",
+      tier: "strong" as const,
+      contextWindow: 1_050_000,
+      inputCostPerMtok: "5.00",
+      outputCostPerMtok: "30.00",
+    },
     // Qwen via Aliyun Bailian (DashScope); runs need a per-project dashscope
     // credential (project settings → providers). Prices are entry-tier list
     // USD per MTok — Bailian tiers rates by input length, so verify against

packages/executor-codex/README.md

@@ -26,13 +26,28 @@ JSONL events observed (probe samples):
 {"type":"turn.completed","usage":{"input_tokens":30961,"cached_input_tokens":25088,"cache_write_input_tokens":0,"output_tokens":129,"reasoning_output_tokens":0}}
 ```
 
+Error stream (probed live: a model id the auth mode doesn't serve):
+
+```json
+{"type":"thread.started","thread_id":"…"}
+{"type":"item.completed","item":{"id":"item_0","type":"error","message":"Model metadata for `gpt-5.1-codex` not found. Defaulting to fallback metadata; this can degrade performance and cause issues."}}
+{"type":"turn.started"}
+{"type":"error","message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-5.1-codex' model is not supported when using Codex with a ChatGPT account.\"}}"}
+{"type":"turn.failed","error":{"message":"{…same JSON blob…}"}}
+```
+
 Notes:
 
 - `input_tokens` **includes** `cached_input_tokens`; the mapper splits them so
   pricing never double-charges cache reads.
 - `thread_id` is the resume handle (`codex exec resume <id>`), used only for
   retry/crash-resume of the same step per ADR-0005.
 - Unknown event/item kinds (reasoning, web_search, todo_list…) are ignored.
+- Error items carry their text in `message` (not `text`) and are **not always
+  fatal** — the CLI emits metadata warnings this way and then completes the
+  turn fine. Only `turn.failed` / top-level `error` (or a nonzero exit) fail
+  the step; the mapper keeps the two channels separate and unwraps JSON-blob
+  messages to the inner human sentence.
 
 ## Capabilities & isolation
 

packages/executor-codex/src/events.ts

@@ -19,6 +19,7 @@ export type CodexItem = {
   id?: string;
   type?: string;
   text?: string;
+  message?: string;
   command?: string;
   aggregated_output?: string;
   exit_code?: number | null;
@@ -34,14 +35,35 @@ export type CodexThreadEvent = {
   error?: { message?: string };
 };
 
+/**
+ * The CLI forwards backend failures as a JSON blob in the message field
+ * (`{"type":"error","status":400,"error":{"message":"…"}}`) — pull the human
+ * sentence out so the step error reads as prose, not wire format.
+ */
+function humanMessage(raw: string): string {
+  try {
+    const parsed = JSON.parse(raw) as { error?: { message?: string }; message?: string };
+    return parsed.error?.message ?? parsed.message ?? raw;
+  } catch {
+    return raw;
+  }
+}
+
 /**
  * Stateful JSONL → ExecutorEvent mapper for one step. Tracks the session id,
- * the final agent message (the step output), and the first error message.
+ * the final agent message (the step output), and two error channels: fatal
+ * stream errors (`turn.failed`, top-level `error`) that must fail the step,
+ * and item-level error items, which the CLI also uses for non-fatal warnings
+ * ("Model metadata … not found. Defaulting to fallback") before carrying on —
+ * those only color the message of a failure the exit code already signals.
  */
 export class CodexEventCollector {
   sessionId: string | null = null;
   lastAgentMessage = "";
-  errorMessage: string | null = null;
+  /** First fatal error (`turn.failed` / `error`) — usually the same failure twice. */
+  fatalMessage: string | null = null;
+  /** Last item-level error text — the item nearest the death is the most specific. */
+  itemErrorMessage: string | null = null;
 
   constructor(private readonly providerModelId: string) {}
 
@@ -95,7 +117,9 @@ export class CodexEventCollector {
           ];
         }
         if (item.type === "error") {
-          this.errorMessage ??= item.text ?? "codex reported an error";
+          // codex-cli 0.145 puts the text in `message` (older shapes used `text`)
+          const text = item.message ?? item.text;
+          if (text !== undefined) this.itemErrorMessage = text;
         }
         return [];
       }
@@ -117,11 +141,11 @@ export class CodexEventCollector {
         ];
       }
       case "turn.failed": {
-        this.errorMessage ??= event.error?.message ?? "codex turn failed";
+        this.fatalMessage ??= humanMessage(event.error?.message ?? "codex turn failed");
         return [];
       }
       case "error": {
-        this.errorMessage ??= event.message ?? "codex error";
+        this.fatalMessage ??= humanMessage(event.message ?? "codex error");
         return [];
       }
       default:

packages/executor-codex/src/executor.test.ts

@@ -182,6 +182,34 @@ describe("codex executor", () => {
     expect(events.some((e) => e.type === "step.completed")).toBe(false);
   });
 
+  it("surfaces the backend failure, not the metadata warning that preceded it", async () => {
+    const events = await collect(makeReq(makeWorkspace("model-rejected")));
+    const terminal = events.at(-1);
+    expect(terminal?.type).toBe("step.failed");
+    if (terminal?.type === "step.failed") {
+      expect(terminal.error.code).toBe("model_error");
+      expect(terminal.error.message).toContain(
+        "not supported when using Codex with a ChatGPT account",
+      );
+      expect(terminal.error.message).not.toContain("Model metadata");
+      expect(terminal.error.message).not.toContain("invalid_request_error"); // prose, not the JSON blob
+    }
+  });
+
+  it("a non-fatal error item does not fail a step whose turn completed", async () => {
+    const events = await collect(makeReq(makeWorkspace("warning-then-success")));
+    const terminals = events.filter((e) => e.type === "step.completed" || e.type === "step.failed");
+    expect(terminals).toEqual([{ type: "step.completed", output: "All done" }]);
+  });
+
+  it("falls back to a non-empty message when the CLI dies silently", async () => {
+    const events = await collect(makeReq(makeWorkspace("die-silent")));
+    const terminal = events.at(-1);
+    expect(terminal?.type === "step.failed" && terminal.error.message).toBe(
+      "codex produced no output",
+    );
+  });
+
   it("kills the subprocess and reports aborted on cancellation", async () => {
     const controller = new AbortController();
     setTimeout(() => controller.abort(), 150);

packages/executor-codex/src/executor.ts

@@ -224,24 +224,39 @@ export function createCodexExecutor(options: CodexExecutorOptions = {}): Executo
         if (!started) {
           // the CLI died before announcing a thread (bad auth, bad flags…)
           const stderr = (await stderrPromise).trim().slice(-2000);
+          ctx.logger.warn("codex died before starting a thread", { exitCode, stderr });
           yield {
             type: "step.failed",
             error: {
               code: "model_error",
-              message: collector.errorMessage ?? stderr ?? "codex produced no output",
+              message:
+                collector.fatalMessage ??
+                collector.itemErrorMessage ??
+                (stderr || "codex produced no output"),
             },
           };
           return;
         }
-        if (exitCode !== 0 || collector.errorMessage) {
+        // Item-level error items alone don't fail the step — the CLI also
+        // emits them as non-fatal warnings and then completes the turn fine.
+        if (exitCode !== 0 || collector.fatalMessage) {
           const stderr = (await stderrPromise).trim().slice(-2000);
-          const message = collector.errorMessage ?? stderr ?? `codex exited with ${exitCode}`;
+          const message =
+            collector.fatalMessage ??
+            collector.itemErrorMessage ??
+            (stderr || `codex exited with ${exitCode}`);
+          ctx.logger.warn("codex step failed", { exitCode, stderr });
           yield {
             type: "step.failed",
             error: { code: normalizedErrorCode(message), message: message.slice(0, 2000) },
           };
           return;
         }
+        if (collector.itemErrorMessage) {
+          ctx.logger.warn("codex emitted a non-fatal error item", {
+            message: collector.itemErrorMessage,
+          });
+        }
 
         yield* collectStepArtifacts(req, collector.lastAgentMessage);
         yield { type: "step.completed", output: collector.lastAgentMessage };