From 5e7b1f8e5e34458f35f6d87c36828f105f930955 Mon Sep 17 00:00:00 2001 From: Thomas Luizon Rodrigues Gregorio Date: Tue, 14 Jul 2026 19:46:15 -0300 Subject: [PATCH] chore(harness): tools/ dir + /make-tool + doc registry + TESTING.md (#461 A1+A3+A4) A1: stand up a tracked tools/ directory with README + CONVENTIONS, an agent-review .sh/.ps1 wrapper over second-opinion.mjs, the two reusable merge-sweep scripts promoted out of the gitignored night-run dir, a /make-tool skill, and a CLAUDE.md pointer. A3: a greppable "At a glance" header on the 7 system docs plus a Docs registry in CLAUDE.md. A4: a root TESTING.md suite catalog, linked from CLAUDE.md, /audit-tests, /pr-review. Docs, one skill, and thin shell wrappers only; no product code, so no parity or i18n. The merge-sweep scripts were copied verbatim because the source night-run dir is gitignored and untracked (git mv was not possible); their pre-existing comment em dashes are left as-is (retrofit is a non-goal). A tools/*.sh eol=lf gitattributes rule keeps the bash scripts LF on Windows. Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude/pending-lessons.md | 4 +- ...agent-memory-guardrails-operating-model.md | 7 +++ .claude/research/ai-environment-overhaul.md | 7 +++ .claude/research/harness-upgrade-plan.md | 7 +++ .../opencode-glm52-go-zen-vs-openrouter.md | 7 +++ .claude/skills/audit-tests/SKILL.md | 2 + .claude/skills/make-tool/SKILL.md | 34 +++++++++++ .claude/skills/pr-review/SKILL.md | 1 + .gitattributes | 1 + CLAUDE.md | 18 +++++- DESIGN.md | 7 +++ FEATURES.md | 7 +++ TESTING.md | 47 +++++++++++++++ research.md | 7 +++ tools/CONVENTIONS.md | 29 ++++++++++ tools/README.md | 19 ++++++ tools/agent-review.ps1 | 49 ++++++++++++++++ tools/agent-review.sh | 48 +++++++++++++++ tools/merge-sweep-cov.sh | 58 +++++++++++++++++++ tools/merge-sweep.sh | 27 +++++++++ 20 files changed, 383 insertions(+), 3 deletions(-) create mode 100644 .claude/skills/make-tool/SKILL.md create mode 100644 TESTING.md create mode 100644 tools/CONVENTIONS.md create mode 100644 tools/README.md create mode 100644 tools/agent-review.ps1 create mode 100755 tools/agent-review.sh create mode 100755 tools/merge-sweep-cov.sh create mode 100755 tools/merge-sweep.sh diff --git a/.claude/pending-lessons.md b/.claude/pending-lessons.md index 6fb1f27b3..9e149d52e 100644 --- a/.claude/pending-lessons.md +++ b/.claude/pending-lessons.md @@ -9,9 +9,9 @@ Reviewed and promoted via `/lesson`. Delete each entry once promoted to a rule/h - Draft: When the user frames a GLM-5.2 scenario as "opencode + Zen" / "opencode go + zen", the access + billing route is the **opencode Zen gateway** (provider "OpenCode Go", `opencode.ai/zen`, slug `opencode-go/glm-5.2`, metered pay-as-you-go). Do NOT price it as Z.ai's direct GLM Coding Plan subscription. GLM-5.2 is upstream a Z.ai/Zhipu model — cite Z.ai only as the model maker / benchmark source, never as the billing route unless the user explicitly says "Z.ai" or "GLM Coding Plan". ## 2026-07-14 — sweep-merge can race a re-triggered review on a BEHIND PR → merges past CHANGES_REQUESTED -- Trigger: `.claude/night-run/merge-sweep-cov.sh` merging any PR that is BEHIND main (require-up-to-date) and therefore needs an update-branch. Hit once on orbit-api #403 (a HIGH backend-contract finding shipped to main + deployed before the re-review landed; the fix went to the orphaned head branch, not main). +- Trigger: `tools/merge-sweep-cov.sh` merging any PR that is BEHIND main (require-up-to-date) and therefore needs an update-branch. Hit once on orbit-api #403 (a HIGH backend-contract finding shipped to main + deployed before the re-review landed; the fix went to the orphaned head branch, not main). - Type: checkable (the sweep script can enforce this deterministically). -- Proposed home: a guard inside `merge-sweep-cov.sh` (gitignored night-run tooling) — after its update-branch step, re-poll `gh pr view --json reviewDecision` until the re-triggered `review` check reaches a terminal state, and BLOCK the merge unless it re-settles to APPROVED (never merge on the pre-update APPROVED snapshot). Secondary signal to detect a past occurrence: the PR's head branch survives deletion (a post-merge push re-created it) = an orphaned fix that never reached main — scan for surviving head branches after a sweep. +- Proposed home: a guard inside `tools/merge-sweep-cov.sh` — after its update-branch step, re-poll `gh pr view --json reviewDecision` until the re-triggered `review` check reaches a terminal state, and BLOCK the merge unless it re-settles to APPROVED (never merge on the pre-update APPROVED snapshot). Secondary signal to detect a past occurrence: the PR's head branch survives deletion (a post-merge push re-created it) = an orphaned fix that never reached main — scan for surviving head branches after a sweep. - Draft: In the sweep, sequence = update-branch → wait-for-checks-terminal (INCLUDING `review`) → re-read reviewDecision → if APPROVED and required checks green (or coverage-only), merge; else abort + report. Do not read reviewDecision once before the update-branch and reuse it. - Interim operational guard (until promoted): for every BEHIND-PR sweep tonight, after merge re-check `reviewDecision` + whether the head branch still exists; if flipped/orphaned, fix-forward onto main. diff --git a/.claude/research/agent-memory-guardrails-operating-model.md b/.claude/research/agent-memory-guardrails-operating-model.md index e1106c99b..602cecf3a 100644 --- a/.claude/research/agent-memory-guardrails-operating-model.md +++ b/.claude/research/agent-memory-guardrails-operating-model.md @@ -1,5 +1,12 @@ # Operating model: why the "know-better-but-ship-worse + write-a-memory" loop happens, and the fix +> **At a glance** - why the "knows the rule but ships the worse default, then writes a memory" loop happens, and the fix. +> - The one change: stop writing "never do X" memory rules; encode every checkable rule as a deterministic guardrail (hook / lint / CI / type). +> - Keep memory and CLAUDE.md a lean, pruned pointer-index of load-bearing FACTS, not behavioral commandments. +> - For unmechanizable judgment, only external-signal verification helps (run the build / tests / an independent critic), never a remembered rule. +> - More rules lower adherence monotonically; prohibitions decay worse than requirements (all cited). +> - Read the whole doc for the mechanism, the citations, and what was applied to this repo. + Deep-research session 2026-07-06 (7 parallel research subagents, primary sources verified live). Triggered by the recurring failure: agent picks the conventional/worse default (JWT-claim admin auth over a live DB check; emits banned em-dashes it "knows" are banned), gets corrected, writes a "never do X" memory rule, repeats. ## The one operating change diff --git a/.claude/research/ai-environment-overhaul.md b/.claude/research/ai-environment-overhaul.md index e89725784..5ba139cf3 100644 --- a/.claude/research/ai-environment-overhaul.md +++ b/.claude/research/ai-environment-overhaul.md @@ -1,5 +1,12 @@ # AI Environment Overhaul: Plan, Runbook & Evidence +> **At a glance** - the locked plan and runbook for the 2026-07-07 AI environment overhaul. +> - Core principle: an instruction is a probabilistic bias that decays; reliability comes from architecture (where a rule lives), not phrasing. +> - Minimize what is always loaded, push every machine-checkable rule to a deterministic gate, load everything else just in time. +> - Two layers: a ~50-line global behavioral CLAUDE.md, and the Orbit-specific stack loaded on top (scoped CLAUDE.md + `.claude/rules/` + the enforcement stack). +> - Phased: wiring, budget cut, prose-to-gates, memory rebuild, then design plus the gated graduating `/lesson` loop. +> - Read the whole doc for the per-phase runbook and the cited evidence. + Source: `/deep-research` (6 parallel research agents, all live-doc verified), 2026-07-07. Status: decisions locked; executing locally, phase by phase, no commits. diff --git a/.claude/research/harness-upgrade-plan.md b/.claude/research/harness-upgrade-plan.md index 90e429d6c..38171dc25 100644 --- a/.claude/research/harness-upgrade-plan.md +++ b/.claude/research/harness-upgrade-plan.md @@ -1,5 +1,12 @@ # Harness Upgrade — Execution Checklist +> **At a glance** - the actionable, stage-by-stage execution checklist for the harness upgrade (companion to `research.md`). +> - Run one stage per fresh session, keep each under ~100k tokens; this file IS the durable state a new session resumes from. +> - Guiding principle: autonomy within a phase, a human gate between phases, never a headless self-improving loop. +> - Locked (do not re-litigate): no Codex; opencode returns later; `opusplan` is out; keep `/execute` and `/feature`. +> - Encode each rule at its authority tier: hook = enforcement, rule = proactive guidance, skill = procedure, CLAUDE.md = fact. +> - Read the whole doc to pick up the next unchecked stage. + **Companion to `research.md` (repo root)**, which holds the full rationale + sources. This file is the *actionable* plan. Date: 2026-07-09. ## How to use this file diff --git a/.claude/research/opencode-glm52-go-zen-vs-openrouter.md b/.claude/research/opencode-glm52-go-zen-vs-openrouter.md index b966c0139..6ce30f7e5 100644 --- a/.claude/research/opencode-glm52-go-zen-vs-openrouter.md +++ b/.claude/research/opencode-glm52-go-zen-vs-openrouter.md @@ -1,5 +1,12 @@ # OpenCode + GLM‑5.2: Go + Zen vs OpenRouter +> **At a glance** - Go + Zen vs OpenRouter for running GLM-5.2 in the opencode agent, for a solo dev. +> - Bottom line: OpenCode Go primary + Zen spillover wins; the flat $10/mo (up to ~$60/mo at-cost draw) beats OpenRouter's per-token edge at realistic volume. +> - Go and Zen serve vetted full-precision weights; OpenRouter is a quantization lottery unless you pin the provider. +> - Make-or-break: model-routing discipline - reserve GLM-5.2 for hard tasks or you hit the caps in hours. +> - "Go" and "Zen" are OpenCode's own products (the opencode Zen gateway), not Z.ai's subscription. +> - Re-verify pricing before betting on it, it moves. Read the whole doc for the crossover math and the config. + **Question:** For running GLM‑5.2 and other open‑weight models in the OpenCode coding agent, which is the better setup for a solo dev — **(A)** OpenCode **Go** (flat sub) spilling over to **Zen** when capped, or **(B)** OpenCode + **OpenRouter** pay‑as‑you‑go? **Researched:** 2026‑07‑09 (all facts verified live that day; re‑verify pricing before betting on it — it moves). diff --git a/.claude/skills/audit-tests/SKILL.md b/.claude/skills/audit-tests/SKILL.md index b0e11d49a..273ef4cca 100644 --- a/.claude/skills/audit-tests/SKILL.md +++ b/.claude/skills/audit-tests/SKILL.md @@ -51,6 +51,8 @@ Parse `$ARGUMENTS` into a `{scope}` token: blank → `both`; `api`/`backend` → | `orbit-ui-mobile` | Vitest. `apps/web/**/*.{test,spec}.{ts,tsx}`, `apps/mobile/**`, `packages/shared/**/__tests__/`. Factories: `packages/shared/src/__tests__/factories.ts`. Configs: `*/vitest.config.ts`. | | `orbit-api` | xUnit + FluentAssertions in `tests/`. Test accounts via `TEST_ACCOUNTS` env. | +`TESTING.md` (repo root) is the canonical suite catalog: where each suite lives, its exact command, and what it proves. + Load `rubric.md` (the workflow's finders read it — it defines what "intelligent" means) and **`.claude/skills/_shared/verification-protocol.md`**. **Unit only** — if the return flags an integration/E2E/real-DB harness, treat it as out-of-policy (it was deliberately diff --git a/.claude/skills/make-tool/SKILL.md b/.claude/skills/make-tool/SKILL.md new file mode 100644 index 000000000..a6473edbb --- /dev/null +++ b/.claude/skills/make-tool/SKILL.md @@ -0,0 +1,34 @@ +--- +name: make-tool +description: Turn a repeated shell incantation into a reusable, agent-callable script under tools/. Use when you have run the same multi-flag command twice, or when a one-liner has grown into a pipeline worth naming and re-running. Not for a genuine one-off (that stays in your shell history or the scratchpad). +argument-hint: [what the tool should do] +--- + +# Make tool: promote a repeated incantation into `tools/` + +**Input**: $ARGUMENTS (what the tool should do) + +## When to reach for this + +- You have typed the **same multi-flag command a second time** (rule 6: the third real use is the extract point, so the second is the signal to build it now). +- A one-liner has become a **pipeline worth a name** that you will run again. +- A future agent would need to rediscover an incantation you already worked out. + +Do **not** build a tool for a true one-off. That stays in your shell history or the scratchpad. `tools/` is for scripts that earn their keep by running more than once. + +## Steps + +1. **Name the single purpose.** One sentence, one verb. If it needs an "and", that is two tools. +2. **Write it to the `tools/` contract** in `tools/CONVENTIONS.md`: `--help`/`-h`, meaningful exit codes, non-interactive, cwd-safe (resolve paths from the script location), stdin for large payloads, no secrets in argv. +3. **Pick the shells.** Author the POSIX `.sh` (the baseline: night-run bash loop, CI, Git Bash). Add a `.ps1` twin **only when the tool must run in the user's PowerShell shell** as an interactive path; the twin mirrors the `.sh` flags, stdin shape, and exit codes exactly. Give any `.sh` LF line endings. +4. **Prefer delegating over reimplementing.** If a vetted helper already does the hard part (a `.mjs`, a `gh` call), the tool is a thin wrapper over it. Do not re-derive its logic in shell. +5. **Catalog it.** Add a row to `tools/README.md` (tool, what it does, usage) in the same change. +6. **Point to it from `CLAUDE.md` only if broadly useful** across the workflow. A niche tool just lives in the `tools/` catalog. +7. **Prove it.** Run `--help` in each shell you shipped (exit 0) and one real smoke of the happy path. Fix the cause of any failure; do not paper over it. + +## Guardrails + +- **No premature abstraction (rule 6).** Do not build a shared shell library for the first two small wrappers. Extract on the third real use. +- **stdin over argv** for a claim, a diff, a file list, or any large payload. +- **No secrets in argv** (process table + shell history leak them). Read them from the environment or a file. +- **One purpose per script.** A flag matrix that forks behavior is a sign you are hiding two tools in one file. diff --git a/.claude/skills/pr-review/SKILL.md b/.claude/skills/pr-review/SKILL.md index aef7338ad..6da8ff4ff 100644 --- a/.claude/skills/pr-review/SKILL.md +++ b/.claude/skills/pr-review/SKILL.md @@ -83,6 +83,7 @@ In parallel: - `C:\Users\thoma\Documents\Programming\Projects\orbit-api\CLAUDE.md` (root + scoped project `CLAUDE.md`) — only if the diff touches backend. - `DESIGN.md` — only if the diff touches `apps/*` UI files. +- `TESTING.md` - the suite catalog and test-writing conventions; consult it for suite semantics when the diff touches tests. - The plan in `.claude/plans/completed/` if the PR body references one. - **`.claude/skills/pr-review/rubric.md`** — the dimensions, severities, and finding template this review walks. diff --git a/.gitattributes b/.gitattributes index 1a1ad5d1d..37a3d2303 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,3 +1,4 @@ packages/shared/src/types/__generated__/*.generated.ts text eol=lf .claude/workflows/*.mjs text eol=lf .claude/skills/**/*.mjs text eol=lf +tools/*.sh text eol=lf diff --git a/CLAUDE.md b/CLAUDE.md index 493f1fa1c..1e82e02c9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -45,10 +45,26 @@ Auth: web cookie is httpOnly + sameSite strict + secure; mobile tokens live in S - Never create an `AGENTS.md` — opencode reads this CLAUDE.md natively; an AGENTS.md would shadow it. - `.opencode/agents/*.md` are thin pointers to the `.claude/agents/*.md` bodies — when adding an agent, create BOTH. Hooks and memory are Claude-Code-only machinery (inert under opencode). - C# LSP for orbit-api is wired via `.mcp.json` — the Roslyn-backed CWM.RoslynNavigator MCP server (install once: `dotnet tool install -g CWM.RoslynNavigator`) pointed at orbit-api's `Orbit.slnx`; copy from `.claude/mcp.json.example`. +- Reusable agent scripts live in `tools/` (`agent-review`, the merge-sweep scripts) under the `tools/CONVENTIONS.md` contract; build a new one with `/make-tool` once you have run the same incantation twice; a one-off stays in the scratchpad. - Git: one feature/fix per PR (cross-repo work opens paired PRs, cross-linked); branches `feature/`|`fix/`|`chore/`; `main` is protected (no direct or force push — enforced by the `git-guardrails` hook); squash-merge only; never `--no-verify`/`--no-gpg-sign`; never reuse a squash-merged branch. -- Testing: Vitest unit tests only; every feature needs behavior tests. The only sanctioned E2E is the post-deploy web smoke suite. Configs live in each workspace. +- Testing: Vitest unit tests only; every feature needs behavior tests. The only sanctioned E2E is the post-deploy web smoke suite. Configs live in each workspace. `TESTING.md` (repo root) is the suite catalog + how to write a test here. - `/pr-review` is the canonical local diff review (orchestrates security-reviewer / contract-aligner / parity-checker / i18n-syncer + the backward-compat guard). +## Docs registry + +Grep a doc's `At a glance` header before loading the whole file. + +| Doc | Purpose | +|---|---| +| `DESIGN.md` | Authoritative UI spec: navy-violet anchor, semantic tokens, 412px shell. | +| `FEATURES.md` | Code-derived, gating- and platform-aware map of every capability. | +| `WORKFLOW.md` | Path-picking guide (tiny bug / real bug / medium feature / multi-issue). | +| `TESTING.md` | How to write tests here + the catalog of every suite and what each proves. | +| `research.md` | Agentic-harness research + the locked harness design decisions. | +| `.claude/research/*` | Deep-research dossiers behind the harness (memory model, env overhaul, opencode/GLM, upgrade plan). | + +When you change a doc, update its `At a glance` header and this registry in the same edit. + ## Path-picking & delegation `WORKFLOW.md` (repo root) is the path-picking guide (tiny bug / real bug / medium feature / multi-issue) — read it before non-trivial work. Delegate independent/heavy work by default (3 concurrent cap; `Explore` for audits; paired worktrees for multi-issue). diff --git a/DESIGN.md b/DESIGN.md index 9e57cbbd9..ab10b6cbe 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -1,5 +1,12 @@ # Orbit Design System +> **At a glance** - the authoritative spec for every Orbit UI surface; it overrides generic and user-global design defaults. +> - Anchor (locked): navy-violet orbital - slate-950 canvas, violet-500 accent, violet gradient headers, pill CTAs, deliberate emoji. +> - Semantic tokens only (`--bg`, `--bg-elev`, `--fg-1..4`, `--primary`, `--primary-rgb`, `--gradient-header`, ...); no raw hex in UI. +> - Ships light AND dark, all 6 color schemes; mobile-first 412px shell. +> - Source of truth is the vendored handoff at `design/handoff/`; tokens live in `apps/web/app/globals.css` + `apps/mobile/lib/theme.ts` + `packages/shared/src/theme/`. +> - Read the whole doc before shaping, reviewing, or theming any surface. + Source of truth: the vendored handoff at `design/handoff/` (see its README for the canon table). This document is the distilled, authoritative spec. The token values live in `design/handoff/orbit/project/orbit-fig.css`; the primitive dimensions in `orbit-kit.jsx`; the screen compositions in the artboards of `Orbit App - Figma.html`. **Authority note:** this DESIGN.md is authoritative over any generic or user-global design defaults — including the global anti-Inter and anti-violet rules. The handoff adopts Inter for display type and a violet accent deliberately; that deviation is documented here once and applies repo-wide. Deliberate emoji use (habit emojis, stat tiles, streak flame, celebrations) is part of the language and overrides the global anti-emoji rule for UI surfaces where the artboards show them. diff --git a/FEATURES.md b/FEATURES.md index 8ec4302dd..9dd09073c 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -1,5 +1,12 @@ # Orbit — Feature Inventory +> **At a glance** - the single code-derived, gating- and platform-aware map of every Orbit capability. +> - Downstream copy (Play listing, landing page, QA matrix) derives its rows from here, so nothing is undersold. +> - Gating: Free / Trial (7-day, full Pro except Retrospective) / Pro / Yearly-Pro, computed from `User` flags in `PayGateService.cs`. +> - Free limits: 10 top-level habits and 20 AI messages/month; Both = web + mobile (Expo, Android-only), with no iOS app. +> - Headline surfaces: Astra (61 AI tools), the MCP server (79 tools / 15 classes), the opt-in social layer, the core tracker. +> - Read the whole doc before writing store, marketing, or QA copy. + The single, code-derived, gating- and platform-aware map of everything Orbit does. Downstream copy — the Play listing, the landing page, the store description, and the pre-launch QA matrix — derives its rows from here, so no feature is undersold or invisible and nothing needs feature archaeology. **How to read this doc.** Every capability lives in a per-domain table with five columns: **Feature**, **Description** (one line), **Gating**, **Platform**, and **Locale notes**. Astra and MCP are listed at feature/category granularity with tool counts, not one row per tool. diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 000000000..b2378c04b --- /dev/null +++ b/TESTING.md @@ -0,0 +1,47 @@ +# Orbit Testing + +> **At a glance** - how to write a test in orbit-ui-mobile and the catalog of every suite. +> - Unit-only policy (Vitest); the only sanctioned E2E is the post-deploy web smoke suite. +> - Assert behavior and data-attributes, never class names or implementation details. +> - Six suites: web / mobile / shared unit, web Playwright e2e (which IS the post-deploy smoke), Stryker mutation. +> - orbit-api has its own xUnit suite, documented in that repo. +> - Read the whole doc before adding a suite, a CI test job, or an e2e spec. + +Every feature ships behavior tests. A test that cannot fail when the behavior breaks is worse than no test. This doc is the canonical suite catalog; `/audit-tests` and `/pr-review` judge tests against the rubric it points to. + +## How to write a test here + +- **Behavior, not implementation.** Assert what the user or caller observes: rendered text, a `data-*` attribute, a returned value, a thrown error. Never assert class names, call order, or private state. Those pass while the behavior is broken and block honest refactors. +- **Three axes.** A real test covers the happy path **and** an edge case **and** a failure case. Invalid input must be *rejected*, not just valid input accepted. +- **Factories over literals.** Build fixtures with `packages/shared/src/__tests__/factories.ts` so a schema change updates every test in one place. +- **Mock at the boundary.** Mobile: mock at the hook level (query by role or `testID`). Web: mock `next/navigation` and server actions, query by role or `data-*`. Shared: no mocks, pin the pure logic directly. +- **Parity.** A behavior that lands on both platforms gets a test on both platforms in the same change (the cross-platform parity rule). +- **Property tests** for pure shared logic (`@fast-check/vitest`) when a value range matters more than a single example. + +## What to avoid (the anti-patterns `/audit-tests` encodes) + +Happy-path-only; rubber-stamp / assertion-free; "asserts a mock was called" tautologies; over-mocked tests that exercise the mock instead of the code; implementation-coupled tests (call-order or private state); snapshot-as-crutch. The full rubric with severities is `.claude/skills/audit-tests/rubric.md`. + +## Suite catalog + +| Suite | Where | Command | What it proves | +|---|---|---|---| +| Web unit | `apps/web` | `npm test -w @orbit/web` (`vitest run`) | web component / hook / server-action behavior via data-attributes, not classes | +| Mobile unit | `apps/mobile` | `npm test -w @orbit/mobile` (`vitest run`) | mobile component / hook behavior (`@testing-library/react-native`, query by role / testID) | +| Shared unit | `packages/shared` | `npm test -w @orbit/shared` (`vitest run`, + `@fast-check/vitest` property tests) | the Zod contract, utils, validation, query keys, and theme data | +| All unit | root | `npm test` (`turbo run test`) | the three unit suites above; CI adds coverage thresholds | +| Web Playwright e2e / post-deploy smoke | `apps/web/e2e` | `npm --workspace @orbit/web run test:smoke` (`playwright test`, needs `SMOKE_BASE_URL`) | the real core flows (auth, create habit, log habit, Astra create-habit, paywall) against the live deployment | +| Stryker mutation | `packages/shared` | `npm run mutation -w @orbit/shared` (`stryker run`) | that the shared unit tests actually kill mutants (effectiveness, not coverage percent) | + +**The e2e suite and the post-deploy smoke suite are one and the same.** There is no second, PR-time Playwright suite. Playwright specs live in `apps/web/e2e/`, run under the `smoke` project, require `SMOKE_BASE_URL`, and execute against the live production deployment, never localhost. This is the only sanctioned E2E. + +## CI mapping + +- **`.github/workflows/test.yml`** - build, unit tests with coverage thresholds (`turbo run test -- --coverage`), type-check, lint, dependency-audit, design-guard, and contract-drift, on PRs to `main`. +- **`.github/workflows/mutation.yml`** - PR-incremental Stryker run on `packages/shared`, report-only. +- **`.github/workflows/nightly.yml`** - full-scope Stryker mutation run. +- **`.github/workflows/smoke-prod.yml`** - the Playwright smoke suite, post-deploy against the live production deployment. + +## orbit-api + +The backend (`orbit-api`, a sibling repo) has its own xUnit + FluentAssertions unit suite under its `tests/` folder, with test accounts wired via the `TEST_ACCOUNTS` env var. It is unit-only too (the integration suite was removed). See that repo's `CLAUDE.md` and `tests/CLAUDE.md` for its conventions; it is out of scope for this catalog. diff --git a/research.md b/research.md index 060c50750..f45bb0b49 100644 --- a/research.md +++ b/research.md @@ -1,5 +1,12 @@ # Agentic Harness Research and Design +> **At a glance** - 2026-07-09 deep-research on the best local agentic harness for a solo dev on the Orbit stack. +> - Verdict: the harness is already top-few-percent; do not rebuild - fix config drift and route models by task. +> - Through-line: maximum autonomy WITHIN a phase, a hard human gate BETWEEN phases (never a headless loop). +> - Adds: an `/investigate` skill and a proactive `.claude/rules/` layer; split plan and implement into separate sessions. +> - The actionable stage-by-stage checklist is the companion `.claude/research/harness-upgrade-plan.md`. +> - Read the whole doc for the setup audit, the external evidence, and the sources. + **Date:** 2026-07-09 **Scope:** The best local agentic harness / multi-agent orchestration setup for a solo developer (you) using Claude Code (plus Codex / opencode) across the Orbit stack (`orbit-ui-mobile`, `orbit-api`, `orbit-landing-page`), given that every session starts from `orbit-ui-mobile`. diff --git a/tools/CONVENTIONS.md b/tools/CONVENTIONS.md new file mode 100644 index 000000000..fa34d2f4e --- /dev/null +++ b/tools/CONVENTIONS.md @@ -0,0 +1,29 @@ +# tools/ conventions + +> **At a glance** - the contract every script in `tools/` follows so an agent can call it blind. +> - One clear purpose per script; if it grows a second job, split it. +> - Always: `--help`/`-h`, meaningful exit codes, non-interactive, cwd-safe. +> - POSIX `.sh` is the baseline; add a `.ps1` twin only when it must run in the user's PowerShell shell. +> - Large payloads come in on stdin, never argv; secrets never appear in argv. + +A tool here is something an agent invokes without reading its source. That only works if every tool obeys the same contract. + +## The contract + +- **Single purpose.** One script does one thing. A second responsibility means a second script (or a shared step), not a flag matrix. +- **`--help` / `-h`.** Print usage and exit `0`. Cover every flag, the stdin shape, and the exit codes. This is the tool's spec. +- **Meaningful exit codes.** `0` on success; non-zero on failure, with distinct codes for distinct failure classes when a caller would branch on them. Write errors to stderr, results to stdout, so output can be piped. +- **Non-interactive.** Never prompt. No `read`, no `Read-Host`, no confirmation gates. Every input arrives as a flag, an argument, or stdin. A destructive action takes an explicit `--yes`-style flag rather than asking. +- **Cwd-safe.** Resolve paths from the script's own location (`dirname "$0"` in bash, `$PSScriptRoot` in PowerShell), not the caller's working directory, so the tool runs from anywhere. +- **stdin for big payloads.** A claim, a diff, a dossier, a file list goes in on stdin, not as a giant argv string. Small scalars (a repo slug, a PR number, `--model`) are fine as arguments. +- **No secrets in argv.** Tokens and keys are visible in the process table and shell history. Read them from the environment or a file, never a positional argument or flag value. + +## POSIX vs PowerShell + +- **`.sh` is the baseline.** It runs in the night-run bash loop, in CI, and in Git Bash on Windows. Keep it POSIX-ish and give it LF line endings. +- **Add a `.ps1` twin only when the tool must run in the user's primary PowerShell shell** (the interactive path). The twin mirrors the `.sh` interface exactly: same flags, same stdin shape, same exit codes. Do not maintain a `.ps1` for a script that only ever runs in the bash loop. +- Prefer delegating to an existing vetted helper (a `.mjs`, a `gh` call) over reimplementing its logic in shell. The wrappers stay thin. + +## Adding one + +Use `/make-tool`. It scaffolds the script to this contract and adds the catalog row to `README.md`. Do not abstract across tools on the first shared line (rule 6, extract on the third real use); two twenty-line wrappers do not need a shared library. diff --git a/tools/README.md b/tools/README.md new file mode 100644 index 000000000..4bad45442 --- /dev/null +++ b/tools/README.md @@ -0,0 +1,19 @@ +# tools/ + +> **At a glance** - reusable, agent-callable scripts for the Orbit harness, versioned with the repo. +> - A script lives here when it is worth running more than once. Throwaways go to the scratchpad, never here. +> - Every tool is non-interactive, supports `--help`, and returns meaningful exit codes (see `CONVENTIONS.md`). +> - POSIX `.sh` is the baseline; add a `.ps1` twin only when it must run in the user's PowerShell shell. +> - Add a new tool with `/make-tool`; add its row to the catalog below in the same change. + +Reusable scripts an agent (or a human) invokes from the CLI. The bar for landing a file here: it has a single clear purpose and you will run it again. One-off commands stay in your shell history or the scratchpad. + +Read `CONVENTIONS.md` before adding one. Use the `/make-tool` skill to scaffold it. + +## Catalog + +| Tool | What it does | Usage | +|---|---|---| +| `agent-review.sh` / `agent-review.ps1` | Cross-model second opinion (GLM-5.2 via opencode) on one claim or review finding. Thin wrapper over `.claude/skills/second-opinion/second-opinion.mjs`; prints one line of JSON (`AGREE` / `DISAGREE` / `UNSURE`, or a graceful `UNAVAILABLE`). | `agent-review --claim ""` or `agent-review < dossier.txt`; `--help` for options | +| `merge-sweep.sh` | Require-up-to-date server-side merge sweep: per PR, update-branch then poll `mergeStateStatus` until it is decidable and squash-merge. Skips on a failed required check or timeout. | `bash merge-sweep.sh ` | +| `merge-sweep-cov.sh` | Coverage-aware merge sweep: like `merge-sweep.sh`, but admin-overrides a SonarCloud failure that is solely new-code coverage (verified from the check-run summary), and skips anything more. | `bash merge-sweep-cov.sh ` | diff --git a/tools/agent-review.ps1 b/tools/agent-review.ps1 new file mode 100644 index 000000000..554d79d1b --- /dev/null +++ b/tools/agent-review.ps1 @@ -0,0 +1,49 @@ +#!/usr/bin/env pwsh +# Thin CLI over second-opinion.mjs: pipe one claim or dossier, get GLM-5.2's AGREE/DISAGREE/UNSURE verdict as one line of JSON. +$root = Split-Path -Parent $PSScriptRoot +$helper = Join-Path $root '.claude/skills/second-opinion/second-opinion.mjs' + +function Show-Usage { + @' +agent-review - cross-model second opinion (GLM-5.2 via opencode) on one claim. + +Usage: + agent-review --claim "" # claim as arg, dossier built for you + agent-review < dossier.txt # full dossier (title/severity/path:line/code) on stdin + agent-review --claim "" --model --timeout + agent-review --help | -h + +Behavior: + Resolves the repo root from the script location, so it runs from any cwd. + With --claim and empty stdin, sends the claim as the dossier; otherwise forwards stdin verbatim. + Delegates to second-opinion.mjs, prints its single-line JSON, and passes the exit code through. + +Exit codes: + 0 a JSON verdict was printed (OK) or opencode was UNAVAILABLE (graceful, still JSON) + 1 usage error: no --claim and empty stdin, or an unknown flag +'@ +} + +$claim = '' +$pass = @() +$i = 0 +while ($i -lt $args.Count) { + switch ($args[$i]) { + { $_ -in '-h', '--help' } { Show-Usage; exit 0 } + '--claim' { $claim = $args[$i + 1]; $i += 2 } + '--model' { $pass += @('--model', $args[$i + 1]); $i += 2 } + '--timeout' { $pass += @('--timeout', $args[$i + 1]); $i += 2 } + default { [Console]::Error.WriteLine("agent-review: unknown argument: $($args[$i]) (run --help)"); exit 1 } + } +} + +$dossier = '' +if ([Console]::IsInputRedirected) { $dossier = [Console]::In.ReadToEnd() } +if ([string]::IsNullOrEmpty($dossier)) { $dossier = $claim } +if ([string]::IsNullOrEmpty($dossier)) { + [Console]::Error.WriteLine('agent-review: no --claim and empty stdin (run --help)') + exit 1 +} + +$dossier | & node $helper @pass +exit $LASTEXITCODE diff --git a/tools/agent-review.sh b/tools/agent-review.sh new file mode 100755 index 000000000..aa63e4465 --- /dev/null +++ b/tools/agent-review.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# Thin CLI over second-opinion.mjs: pipe one claim or dossier, get GLM-5.2's AGREE/DISAGREE/UNSURE verdict as one line of JSON. +root="$(cd "$(dirname "$0")/.." && pwd)" +helper="$root/.claude/skills/second-opinion/second-opinion.mjs" + +usage() { + cat <<'EOF' +agent-review - cross-model second opinion (GLM-5.2 via opencode) on one claim. + +Usage: + agent-review --claim "" # claim as arg, dossier built for you + agent-review < dossier.txt # full dossier (title/severity/path:line/code) on stdin + agent-review --claim "" --model --timeout + agent-review --help | -h + +Behavior: + Resolves the repo root from the script location, so it runs from any cwd. + With --claim and empty stdin, sends the claim as the dossier; otherwise forwards stdin verbatim. + Delegates to second-opinion.mjs, prints its single-line JSON, and passes the exit code through. + +Exit codes: + 0 a JSON verdict was printed (OK) or opencode was UNAVAILABLE (graceful, still JSON) + 1 usage error: no --claim and empty stdin, or an unknown flag +EOF +} + +claim="" +pass=() +while [ $# -gt 0 ]; do + case "$1" in + -h|--help) usage; exit 0 ;; + --claim) claim="${2:-}"; shift 2 ;; + --model) pass+=(--model "${2:-}"); shift 2 ;; + --timeout) pass+=(--timeout "${2:-}"); shift 2 ;; + *) echo "agent-review: unknown argument: $1 (run --help)" >&2; exit 1 ;; + esac +done + +dossier="" +[ -t 0 ] || dossier="$(cat)" +[ -n "$dossier" ] || dossier="$claim" +if [ -z "$dossier" ]; then + echo "agent-review: no --claim and empty stdin (run --help)" >&2 + exit 1 +fi + +printf '%s' "$dossier" | node "$helper" "${pass[@]}" +exit $? diff --git a/tools/merge-sweep-cov.sh b/tools/merge-sweep-cov.sh new file mode 100755 index 000000000..f8fb35da6 --- /dev/null +++ b/tools/merge-sweep-cov.sh @@ -0,0 +1,58 @@ +#!/usr/bin/env bash +# Coverage-aware merge sweep (server-side gh, robust polling). Per PR: +# - require reviewDecision == APPROVED; SKIP otherwise. +# - SKIP on any failing NON-Sonar required check (a real defect) or a merge conflict (DIRTY). +# - poll through BEHIND (update-branch) and the post-update re-CI window until the merge state +# is decidable (CLEAN/UNSTABLE), then: +# * Sonar SUCCESS/absent -> normal squash-merge. +# * Sonar FAILURE that is SOLELY new-code coverage (verified from the check-run summary, +# never a Bug/Vuln/Hotspot/Smell/Duplication/rating drop) -> admin squash-merge +# (coverage debt repaid in the Sonar burn-down; rubber-stamp tests are banned). +# * Sonar FAILURE on anything more -> SKIP (needs a real fix). +# Never touches the local working tree. Usage: merge-sweep-cov.sh +repo="$1"; shift +gate() { # prints MS \t REVIEW \t NONSONAR_FAILED \t SONARSTATE \t SHA + gh pr view "$1" --repo "$repo" --json mergeStateStatus,reviewDecision,statusCheckRollup,headRefOid 2>/dev/null | node -e " + let s='';process.stdin.on('data',d=>s+=d).on('end',()=>{ + try{ + const d=JSON.parse(s); + const bad=['FAILURE','ERROR','CANCELLED','TIMED_OUT','ACTION_REQUIRED','STARTUP_FAILURE']; + const rows=d.statusCheckRollup||[]; + const failed=rows.filter(c=>bad.includes((c.conclusion||c.state||'').toUpperCase())); + const nonSonar=failed.filter(c=>(c.name||c.context)!=='SonarCloud Code Analysis').map(c=>c.name||c.context); + const sonar=rows.find(c=>(c.name||c.context)==='SonarCloud Code Analysis')||{}; + const sonarState=(sonar.conclusion||sonar.state||'NONE').toUpperCase(); + process.stdout.write([(d.mergeStateStatus||'?'),(d.reviewDecision||'?'),(nonSonar.join(',')||'NONE'),sonarState,(d.headRefOid||'')].join('\t')); + }catch(e){process.stdout.write('ERR\tERR\tERR\tERR\t');} + })" +} +for n in "$@"; do + gh pr update-branch "$n" --repo "$repo" >/dev/null 2>&1 + done_pr="" + for i in $(seq 1 45); do # ~15 min per PR + IFS=$'\t' read -r ms rev nonsonar sonar sha < <(gate "$n") + if [ "$rev" != "APPROVED" ]; then echo "SKIP #$n review=$rev"; done_pr=1; break; fi + if [ "$nonsonar" != "NONE" ] && [ "$nonsonar" != "ERR" ]; then echo "SKIP #$n FAILED(non-sonar)=[$nonsonar]"; done_pr=1; break; fi + if [ "$ms" = "DIRTY" ]; then echo "SKIP #$n DIRTY (conflict)"; done_pr=1; break; fi + if [ "$ms" = "BEHIND" ]; then gh pr update-branch "$n" --repo "$repo" >/dev/null 2>&1; sleep 20; continue; fi + # Non-Sonar failures already ruled out above; a Sonar FAILURE here is the SOLE blocker + # (ms is typically BLOCKED, since SonarCloud Code Analysis is a REQUIRED check) — handle it + # regardless of ms so a coverage-only PR doesn't loop to timeout. + if [ "$sonar" = "FAILURE" ]; then + summary=$(gh api "repos/$repo/commits/$sha/check-runs" --jq '.check_runs[] | select(.name=="SonarCloud Code Analysis") | .output.summary' 2>/dev/null) + if printf '%s' "$summary" | grep -qi "Coverage on New Code" && ! printf '%s' "$summary" | grep -qiE "New Bugs|Bugs |Vulnerabilit|Security Hotspots|Security Rating|Code Smell|Duplicat|Maintainability Rating|Reliability Rating"; then + if gh pr merge "$n" --repo "$repo" --squash --admin --delete-branch >/dev/null 2>&1; then echo "MERGED #$n (admin: coverage-only override)"; else echo "FAIL-ADMIN #$n"; fi + else + echo "SKIP #$n Sonar fails on MORE than coverage — needs a real fix" + fi + done_pr=1; break + fi + if { [ "$ms" = "CLEAN" ] || [ "$ms" = "UNSTABLE" ]; } && { [ "$sonar" = "SUCCESS" ] || [ "$sonar" = "NONE" ]; }; then + if gh pr merge "$n" --repo "$repo" --squash --delete-branch >/dev/null 2>&1; then echo "MERGED #$n (clean)"; done_pr=1; break; fi + fi + # ms=BLOCKED/UNKNOWN with Sonar not-yet-failed → required checks still settling — wait + sleep 20 + done + [ -z "$done_pr" ] && echo "SKIP #$n (timeout waiting for a decidable merge state)" +done +echo "COV-SWEEP-DONE" diff --git a/tools/merge-sweep.sh b/tools/merge-sweep.sh new file mode 100755 index 000000000..d92a65f74 --- /dev/null +++ b/tools/merge-sweep.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Require-up-to-date merge sweep (server-side gh). Per PR: update-branch, then +# poll mergeStateStatus itself until CLEAN/UNSTABLE (mergeable) and merge — +# waiting THROUGH the transient UNKNOWN/BLOCKED window while post-update CI +# re-runs. Skips only on a genuinely FAILED required check or timeout. +repo="$1"; shift +PRS="$@" +mstate() { # prints "MS|REVIEW|FAILEDCHECKS" + gh pr view "$1" --repo "$repo" --json mergeStateStatus,reviewDecision,statusCheckRollup 2>/dev/null | node -e "let s='';process.stdin.on('data',d=>s+=d).on('end',()=>{try{const d=JSON.parse(s);const r=d.statusCheckRollup||[];const failed=r.filter(c=>['FAILURE','ERROR','CANCELLED','TIMED_OUT','ACTION_REQUIRED','STARTUP_FAILURE'].includes((c.conclusion||c.state||'').toUpperCase())).map(c=>c.name||c.context).join(',')||'none';process.stdout.write((d.mergeStateStatus||'?')+'|'+(d.reviewDecision||'?')+'|'+failed);}catch(e){process.stdout.write('ERR|ERR|err');}})" +} +for n in $PRS; do + gh pr update-branch "$n" --repo "$repo" >/dev/null 2>&1 + done_pr="" + for i in $(seq 1 50); do # up to ~17min per PR + IFS='|' read -r ms rev failed <<< "$(mstate "$n")" + if [ "$failed" != "none" ]; then echo "SKIP #$n ms=$ms FAILED=$failed"; done_pr=1; break; fi + if { [ "$ms" = "CLEAN" ] || [ "$ms" = "UNSTABLE" ]; } && [ "$rev" = "APPROVED" ]; then + if gh pr merge "$n" --repo "$repo" --squash --delete-branch >/dev/null 2>&1; then echo "MERGED #$n"; else echo "MERGE-REFUSED #$n ms=$ms rev=$rev"; fi + done_pr=1; break + fi + if [ "$ms" = "BEHIND" ]; then gh pr update-branch "$n" --repo "$repo" >/dev/null 2>&1; fi + if [ "$ms" = "DIRTY" ]; then echo "SKIP #$n ms=DIRTY (conflict)"; done_pr=1; break; fi + sleep 20 + done + [ -z "$done_pr" ] && echo "SKIP #$n (timeout waiting for CLEAN)" +done +echo "SWEEP-DONE"