Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .claude/research/harness-upgrade-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,17 +63,18 @@ The generic, one-mechanism fix for "why do you violate instructions" — targets
- [x] Added `effort:` to the subagents: `parity-checker`/`i18n-syncer` → `low` (already `model: haiku`); `contract-aligner`/`security-reviewer`/`design-reviewer` → `medium`. **design-reviewer** also got `model: sonnet` — it carried no `model` and would have inherited the Opus driver, which defeats the routing for an auto-firing reviewer; pinning it to sonnet matches its two effort-tier siblings (a reasoned extension of the literal checkbox, flagged). **Lockstepped** the same `effort: medium` onto orbit-api's own (body-drifted) copies of `contract-aligner`/`security-reviewer` so Stage 3 did not widen the drift Stage 6 will dedupe. Frontmatter order follows the Stage-2 `Explore` convention (`model:` then `effort:`).
- [x] Converted `/audit-security`, `/audit-tests`, `/audit-performance`, `/audit-code-quality`, and `/prod-readiness` to **dynamic workflows**. New `.claude/workflows/audit.mjs` — ONE generic engine keyed by `kind` (per rule 10 DRY; ≥4 uses justify the abstraction): a **Haiku finder per surface** (reads that audit's rubric/checklist) → a **Haiku default-refuted skeptic** per serious finding (Tier 1/2 or Critical/High) → a **completeness-critic loop-until-dry** (cap 2 dry rounds, 4-round hard cap) → returns verified structured findings + coverage + deferred. New `.claude/workflows/prod-readiness.mjs` — runs the four audit workflows in parallel via one-level `workflow()` nesting (scriptPath form, name-resolution-independent) + the ops-layer fan-out (observability/multi-instance/background-durability/staging; backups always Deferred) + ops skeptic verify. The five skills became **thin non-fork drivers**: resolve scope → invoke `Workflow` (the skill's own instructions are the sanctioned opt-in per the Workflow gate) → **Opus synthesizes** the report from the compact return. **This is the real quota win**: the entire multi-phase fan-out/verify/loop *reasoning* the Opus `context: fork` used to do is now free deterministic JS; Opus spends only on synthesis, every leaf agent is Haiku. Dropped `context: fork` (the driver must be main-loop to call `Workflow`); each skill keeps a **headless/CI fallback** (inline `Explore` fan-out) because the Workflow tool is absent from CI tool-allowlists (verified: only `/pr-review` runs in CI, via `Task`, and the audits run interactively only). Both scripts full-syntax-validated (the `node --check` "illegal return" is a false positive — the Workflow runtime wraps scripts so top-level `return` is legal). Live end-to-end fires on the next real `/audit-*` invocation; the core `agent({model:'haiku', schema})` + file-tool mechanics were smoke-tested via a 1-agent probe workflow.
- [x] Confirmed **no `opusplan`** and **no `"model"` pin** in any `.claude` config (grepped) — Opus stays the session driver. `ultrathink`-per-hard-turn is a usage habit, nothing to configure. Model routing lives entirely at the subagent/workflow tier, per the locked decision.
- [x] **Verified live (2026-07-09, Stage-4 session): `/audit-security api` end-to-end.** Found + fixed four issues the conversion carried: (1) **named resolution is unavailable in this build** — `Workflow({name:'audit'})` → "Available: (none)"; repointed all five skills (4 audits + prod-readiness) to the **`scriptPath`** form (prod-readiness.mjs already nested via scriptPath). (2) **Windows `autocrlf=true` gives the working-tree `.mjs` CRLF**, which the Workflow approval dialog rejects as "control characters" — added a `.gitattributes` gate (`.claude/workflows/*.mjs text eol=lf`) + renormalized; durable across re-checkout. (3) **The top-level Workflow tool delivers `args` as a JSON string** (not a parsed object) → `args?.kind` was undefined; both scripts now normalize at the boundary (`typeof args === 'string' ? JSON.parse(args) : args`). (4) **`scope:'api'` under-covered security** — `isApiSurface` inferred repo from a substring match on each surface's `where`, so injection/secrets/rate-limit/error-leakage (all real API surfaces) were dropped and the completeness critic expensively regenerated them (dragging in Tier-3 noise: GDPR/CVE-scan/monitoring). Fixed with explicit `repos: 'api'|'ui'|'both'` tags on the security surfaces + a tier-calibrated, ≤6-gap critic prompt. (5) **The adversarial skeptic pass was silently bypassed for ~all findings** — the Haiku finders emit `CRITICAL/HIGH/MEDIUM` severities, but `isSerious` gated verification on the security ladder's `Tier 1/Tier 2` strings via exact set-membership, so the default-refuted verify never fired on the 21 "CRITICAL"/26 "HIGH" findings. Fixed `isSerious` to be `rank()`-based (vocabulary-agnostic — `rank()` already normalizes both `CRITICAL`↔`Tier 1`; validated by construction, no costly re-run) and deleted the now-dead `KIND.*.verify` config. The re-run itself completed (43 agents, 0 errors, 4 critic rounds, 184 raw findings) and validated the full plumbing; its findings are UNVERIFIED (produced pre-fix), so `.claude/audits/security-api.md` is written as an explicit **smoke-test validation artifact**, not an authoritative report — a clean re-run is the source of truth. The other three kinds classify correctly via their `api-` label prefixes (unaffected).

## Stage 4 — Proactive standards + second opinion + investigate (one session, may split)

- [ ] `~/.claude/rules/` — unscoped rule(s) for proactive tooling defaults (the rule-tier sibling of the Stage-1 guard). This is the mechanism for "always use tool X" without repeating it.
- [ ] **`/investigate` skill:** Sentry issue → Render deploy/logs → Postgres rows → localize to code (needs `csharp-lsp` from Stage 2) → root cause + minimal fix, gated. A runbook skill (Anthropic keeps these). Consider running it on Fable.
- [ ] **`/second-opinion` skill (deferred until opencode is back):** `opencode run --model <glm-provider>/glm-5.2 --format json "<claim + diff>"`. Wire it **inside `/pr-review`** so it auto-fires on each Critical finding that survives the skeptic; if GLM disagrees, mark `CONTESTED` and surface both verdicts (do not force a merge). Confirm the exact GLM provider slug from opencode's catalog at build time.
- [x] `~/.claude/rules/` — proactive tooling defaults. Confirmed (claude-code-guide, cited docs) that `.claude/rules/*.md` is a **native auto-load** — files load at session start, survive compaction, and re-inject post-compaction; unscoped (no `paths` frontmatter) = loads unconditionally. **No loader hook needed.** Wrote `~/.claude/rules/tooling-defaults.md` — the "always reach for tool X" defaults for the live servers: Sentry MCP (error triage), Render MCP (deploys/logs/metrics, workspace-select-first guardrail), `query_render_postgres` (read-only data), csharp-lsp (orbit-api C# nav), `gh` CLI (issues/PRs/CI). Pairs with, does not duplicate, the Stage-1 proactivity reminder (disposition) — this names the concrete servers.
- [x] **`/investigate` skill** — `.claude/skills/investigate/SKILL.md`. Gated runbook: Sentry issue → Render deploy/logs → Postgres rows (read-only) → localize via csharp-lsp → root cause → **minimal fix behind a human gate** (read-only until the gate). Uses the real MCP servers (verified live this session: Sentry org `thomasluizon` ✓, Render MCP ✓ — needs a workspace select, encoded in the runbook). **No `model:` pin — inherits the session model (Opus 4.8 at `xhigh`).** (Skills *can* pin a model via frontmatter; deliberately not here.) Rationale: incident root-cause is low-frequency, high-stakes judgment — model routing pays off on high-*volume* cheap work (the Haiku audit leaves, 40+/run), not a once-per-incident runbook, and a wrong diagnosis costs time even behind the human gate. Dropping the pin also permanently removes the model-churn fragility that surfaced when the first `claude-fable-5` pin met Fable's **2026-07-12 disablement** (a pinned-to-dead-model skill errors or silently falls back). The runbook's value is the gated Sentry→Render→Postgres→csharp-lsp structure + the human gate, not a model tier. (Briefly repointed Fable→Sonnet-5 before landing on drop-the-pin.)
- [ ] **`/second-opinion` skill — UNBLOCKED (opencode is back).** opencode is now installed (`v1.14.48`, verified this session) — the "deferred until opencode is back" condition is void. Held for a follow-up session per this session's scope (rules + investigate). When built: `opencode run --model <glm-provider>/glm-5.2 --format json "<claim + diff>"`, wired **inside `/pr-review`** to auto-fire on each Critical finding that survives the skeptic; GLM disagreement → mark `CONTESTED`, surface both verdicts (never force a merge). Confirm the exact GLM provider slug from opencode's catalog at build time.

## Stage 5 — Workflow docs (one session)

- [ ] `WORKFLOW.md`: add the **slice-vs-campaign router** (campaign = looped / multi-session / externally-tracked, e.g. #243workflows + fresh sessions; slice = `/execute`) and the **plan→implement fresh-session split** rule for cross-repo/risky work (persist plan file → `/clear` → fresh `/prime` + `/implement <issue>`).
- [ ] Document when to use `/execute` vs the manual split; both stay.
- [x] `WORKFLOW.md`: added the **slice-vs-campaign router** as the top-level Step 1 (litmus: converging-metric finish linecampaign; known diff → slice), a **Campaign pattern** section (assess via workflow → fix a batch via `/implement` → PR → `/clear` → re-run, fresh sessions; the #243 worked example), and the **plan→implement fresh-session split** rule for cross-repo/risky slices. Kept the four slice paths intact. Landed in ui#438 (bundled into the Stage-4 PR per the minimum-PRs preference, not a separate PR).
- [x] Documented `/execute` vs the split: `/execute` stays the **default for slices**; the split is for cross-repo/risky slices; the campaign loop is for converging-metric work. Both `/execute` and the manual path remain. (Triggered by the user's "I was used to running `/execute` for everything" — the doc now teaches when NOT to.)

## Stage 6 — Drift hygiene (one session)

Expand Down
4 changes: 2 additions & 2 deletions .claude/skills/audit-code-quality/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,10 @@ executes §1/§2/§3; you emit the Verify summary + Deferred ledger (§4/§5).
Invoke the `Workflow` tool (this skill's instructions are the opt-in):

```
Workflow({ name: 'audit', args: { kind: 'code-quality', scope: '<resolved {scope}>' } })
Workflow({ scriptPath: '.claude/workflows/audit.mjs', args: { kind: 'code-quality', scope: '<resolved {scope}>' } })
```

(If `name` does not resolve, use `scriptPath: '.claude/workflows/audit.mjs'`.)
(`scriptPath` is canonical — named workflow resolution is not available in this Claude Code build.)

It fans out **one Haiku finder per area** — `apps/web` · `apps/mobile` · `packages/shared`
· `orbit-api/src/Orbit.Application` · `orbit-api/src/{Orbit.Domain,Orbit.Infrastructure,
Expand Down
4 changes: 2 additions & 2 deletions .claude/skills/audit-performance/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,10 @@ emit the Verify summary + Deferred ledger (§4/§5).
Invoke the `Workflow` tool (this skill's instructions are the opt-in):

```
Workflow({ name: 'audit', args: { kind: 'performance', scope: '<resolved {scope}>' } })
Workflow({ scriptPath: '.claude/workflows/audit.mjs', args: { kind: 'performance', scope: '<resolved {scope}>' } })
```

(If `name` does not resolve, use `scriptPath: '.claude/workflows/audit.mjs'`.)
(`scriptPath` is canonical — named workflow resolution is not available in this Claude Code build.)

It fans out **one Haiku finder per slice** — `api-queries` (N+1, index coverage vs the
migrations) · `api-requestpath` (sync slow work, blocking async, over-fetch, AsNoTracking) ·
Expand Down
4 changes: 2 additions & 2 deletions .claude/skills/audit-security/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,10 @@ return (§4/§5).
Invoke the `Workflow` tool (this skill's instructions are the opt-in):

```
Workflow({ name: 'audit', args: { kind: 'security', scope: '<resolved {scope}>' } })
Workflow({ scriptPath: '.claude/workflows/audit.mjs', args: { kind: 'security', scope: '<resolved {scope}>' } })
```

(If `name` does not resolve, use `scriptPath: '.claude/workflows/audit.mjs'`.)
(`scriptPath` is canonical — named workflow resolution is not available in this Claude Code build.)

It fans out **one Haiku finder per attack surface** — authz & data-isolation · AI/MCP tool
scoping · injection · secrets & config · rate-limit & AI-abuse · error-leakage & web/mobile
Expand Down
4 changes: 2 additions & 2 deletions .claude/skills/audit-tests/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,10 @@ the workflow tags it **Critical**.
Invoke the `Workflow` tool (this skill's instructions are the opt-in):

```
Workflow({ name: 'audit', args: { kind: 'tests', scope: '<resolved {scope}>' } })
Workflow({ scriptPath: '.claude/workflows/audit.mjs', args: { kind: 'tests', scope: '<resolved {scope}>' } })
```

(If `name` does not resolve, use `scriptPath: '.claude/workflows/audit.mjs'`.)
(`scriptPath` is canonical — named workflow resolution is not available in this Claude Code build.)

It fans out **one Haiku finder per suite area** — `web` · `mobile` · `shared` ·
`api-application` · `api-domain` — each scoring tests against `rubric.md` (Behavior / Edge /
Expand Down
84 changes: 84 additions & 0 deletions .claude/skills/investigate/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
name: investigate
description: Root-cause a production incident end to end — pull the Sentry issue, correlate it to a Render deploy and its runtime logs, inspect the Postgres rows it implicates, localize the fault to code via csharp-lsp, then propose the minimal fix behind a human gate. Use when a production error, crash, Sentry alert, or "why is X broken in prod" needs diagnosing. Not for a local dev bug you already have a reproducible stack trace for — just read the code.
argument-hint: <sentry-issue-id | sentry-url | short description of the incident>
---

# Investigate: Production Incident Root-Cause

**Input**: $ARGUMENTS

Diagnose a live incident against the real systems — Sentry, Render, Postgres, and the code — and hand back a root cause plus the **minimal** fix. Every claim is pinned to evidence a tool returned; nothing is guessed.

## Operating rules

- **Read-only until the gate.** Phases 0–5 only *inspect* (Sentry, Render logs, a Postgres `SELECT`, code navigation). No edits, no data mutation.
- **Human gate before Phase 6.** Present the root cause + proposed fix and **STOP**. Do not touch a file until I approve. (Autonomy within the investigation; a gate before the change.)
- **Root cause, not symptom** (CLAUDE.md rule 1). Name the upstream cause — a missing validator, a nullable that should not be, contract drift, a migration gap. No workaround that masks it.
- **Verify, don't guess.** Each finding traces to a Sentry event, a Render log line, a DB row, or a code line. If a step's tool is unreachable, say so and continue — don't invent its output.

## Phase 0 — Frame the incident

Parse `$ARGUMENTS`:
- A Sentry issue id / short-id / URL → go straight to it.
- A description ("signup 500s", "widget crash on ColorOS") → search Sentry for the matching issue.

Confirm the Sentry org is `thomasluizon` and identify the project (api / web / mobile).

## Phase 1 — Sentry: what broke

- `mcp__sentry__search_issues` (or `mcp__sentry__get_sentry_resource` with the id) → exception type, culprit, level, first/last seen, event count, affected releases + environments.
- Pull a representative event → full stack trace, breadcrumbs, request context, tags, affected-user data.
- Optionally `mcp__sentry__analyze_issue_with_seer` for an AI root-cause hypothesis — treat it as a lead to verify against evidence, never as fact.

**Capture:** the exact exception + message, the top **in-app** stack frame (file:line), the release it started in, and how often / who it hits.

## Phase 2 — Render: what changed and what the runtime saw

- Ensure a Render workspace is selected: `mcp__render__get_selected_workspace`; if none, `list_workspaces` → **confirm the choice with me** → `select_workspace`. Never auto-pick.
- `list_services` → the orbit-api service. `list_deploys` / `get_deploy` → find the deploy whose window brackets the issue's first-seen. Did the error start right after a deploy? Which commit?
- `list_logs` across the incident window for that service → the runtime lines around the Sentry event timestamps (log context Sentry may lack).

**Capture:** the suspect deploy + commit if the error is deploy-correlated (or "not deploy-correlated"), and any runtime log detail the Sentry event is missing.

## Phase 3 — Postgres: what the data says (only if the fault implicates data)

- `mcp__render__query_render_postgres` — **read-only**. Inspect the rows the stack trace implicates: the offending record, an unexpected null, a violated constraint, a duplicate, the affected user's state.
- Never `UPDATE` / `DELETE` / `INSERT`. Confirm or kill the data hypothesis with a `SELECT`.

**Capture:** the row-level evidence, or "data not implicated."

## Phase 4 — Localize to code

- **orbit-api (C#) → csharp-lsp.** From the top in-app frame: `mcp__csharp-lsp__find_symbol` → the method; `find_callers` / `find_references` → how it is reached with the bad input; `get_diagnostics` on the file; `get_type_hierarchy` / `find_implementations` when the frame is an interface / virtual dispatch.
- **web / mobile (TS) →** Grep/Read the implicated `apps/web` or `apps/mobile` module and its shared types in `packages/shared`.

**Capture:** the exact repo/path:line of the fault and the code path that triggers it.

## Phase 5 — Root cause

One tight paragraph: the exact line + the condition that triggers it + why the deploy / data / input produced it *now*. Point at the upstream cause, not the thrown symptom.

## Phase 6 — Propose the minimal fix — THEN STOP (human gate)

Present, and wait for my approval before any edit:

- **Root cause** — one paragraph.
- **Minimal fix** — file:line → the smallest correct change that removes the cause (not a defensive branch around it). Flag: cross-platform parity if it is a UI/shared change; backend-source-of-truth if it is validation; the append-only contract if it is a DTO.
- **Regression test** — the test that would have caught this, to add with the fix.
- **Blast radius / verification** — what to run after (`/validate`, a targeted test), and what else the change touches.

On approval: implement with parity + the regression test, then hand to `/validate` (or `/pr-review`). Do not edit before approval.

## Output — the incident dossier (rendered before the gate)

```
## Incident: {exception} ({sentry-short-id})

- **Seen**: first {…} · last {…} · {N} events · {environment} · {who}
- **Deploy correlation**: {commit / deploy} — or "not deploy-correlated"
- **Data**: {row evidence} — or "not implicated"
- **Fault**: {repo}/{path}:{line} — {the trigger path}
- **Root cause**: {one paragraph}
- **Proposed fix (awaiting approval)**: {file:line → change} + {regression test} + {verification}
```
4 changes: 2 additions & 2 deletions .claude/skills/prod-readiness/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,10 @@ Parse `$ARGUMENTS` into one `{scope}` token, forwarded to every child audit:
Invoke the `Workflow` tool (this skill's instructions are the opt-in):

```
Workflow({ name: 'prod-readiness', args: { scope: '<resolved {scope}>' } })
Workflow({ scriptPath: '.claude/workflows/prod-readiness.mjs', args: { scope: '<resolved {scope}>' } })
```

(If `name` does not resolve, use `scriptPath: '.claude/workflows/prod-readiness.mjs'`.)
(`scriptPath` is canonical — named workflow resolution is not available in this Claude Code build.)

It runs the four `audit` workflows in parallel (each self-caps its own fan-out + verify +
loop), then fans out the **ops-layer** checks — observability · multi-instance · background
Expand Down
Loading
Loading