Repository navigation
ci: gate integrity — stop hiding failing suites, gate the app too - #12
Conversation
- Delete baseUrl: "." (incompatible with TypeScript 7) - Prefix all 7 paths values with "./" to resolve relative paths (TS5090) Co-Authored-By: Claude Task Master <noreply@anthropic.com>
`tsc -b` walked only ./packages/* — the whole reference app sat outside the gate, so drift between the framework's API and the app that demonstrates it was invisible. The app is now a composite project the root solution references, and `bun run typecheck` fails on that drift. @ultimat3/ui's `*.module.scss` contract lived in an ambient declaration no consumer program ever loads, so entering the graph manufactured 39 TS2307s in framework components. A triple-slash reference from the entry carries it to every consuming app. Expected red: typecheck now reports 454 errors, all in examples/dummy — the app is written against symbols the packages do not export (defineApi, defineMail, defineService, defineCatalogs, daysBetween). Later tasks in this PR close them; the other 6 verify steps pass. Co-Authored-By: Claude Task Master <noreply@anthropic.com>
- preload registers the framework's fixtures: clock, mail, runJobs - fixture-clock/mail/jobs import their subsystem inside the factory, so a test that never destructures `runJobs` never starts a queue - Fixtures declares the three; apps still widen it by augmentation - fixtureSnapshot() so a test that clears the process-global registry can hand it back — clearing it used to strand every later file - examples/dummy: scripts/test-setup.ts registers `seed` + `actorFor` off packages/db/seeds/dev.ts, with bunfig.toml preloading it - rows are captured on insert, not read back: a tenant-scoped entity refuses an unscoped read X_TEST_FIXTURE_UNKNOWN failures 17 -> 12; the 10 `seed` tests now reach the real gap (the action DSL). Remaining unknown fixtures — page, subscribe, evaluate — belong to the e2e, realtime and eval tasks. Co-Authored-By: Claude <noreply@anthropic.com>
- core/listeners.ts: markListening() / isSelfOrigin(), loopback- and wildcard-aware, refcounted, idempotent release - http + realtime announce their socket on start and release it on close - sealed network treats a self origin as not-egress, so packages/http/e2e runs sealed: `bun test packages/http/e2e`, no ULTIMATE_TEST_ALLOW_NET=1 - unsealing stays reserved for a deliberate live integration Co-Authored-By: Claude <noreply@anthropic.com>
`bun run verify` was 7 steps and `x verify` was 13 — two gates sharing one name, so the repo could be green while 8 checks of the contract never ran. - `scripts/verify.ts` now delegates to `cmd-verify.ts`: same list, same runner, same report, same exit code (154 -> 49 lines) - the two repo-only checks join the list as `filesize` and `package-shape`, so nothing is lost; `packages/cli/src/workspace-checks.ts` owns them - a host repo contributes rules to a step it cannot express (the tier table under `boundaries`, the framework manifest under `manifest`) — it can never add, remove, reorder or skip a step - test steps select by filename suffix, not by describe-block prefix, so the contract/live/job/e2e/eval suites actually run and `unit` keeps every test `bun run test` had (1159, unchanged) - `drift` skips outside an app root; a package monorepo's `packages/db` is a driver, not a schema - remove `--only` and `--skip` (spec: no bypass); every `fix:` that named them now names a runnable command Co-Authored-By: Claude <noreply@anthropic.com>
`bun run test` silently dropped e2e/contract/live/job/eval suites since 2026-07-30, so CI stayed green while the reference app's fluent-DSL calls (action `.as()`/`.mcp`, `defineMail` two-arg form) and missing test fixtures (`page`, `subscribe`, `evaluate`) went undetected. Un-exclude them so a bare `bun test` and `bun run test` are identical, and add a dedicated `reference-app-verify` CI job running the app's own 15-step `x verify` — checked independently from the framework gate, since they're two different contracts. Currently: 19 fail / 1 error in examples/dummy (framework packages: 0 fail), and the app's `x verify` is 8/15 red. Every failure traces to the fluent DSL surface (`action.as`/`.mcp`, `defineMail(id, config)`) or a test fixture (`page`, `subscribe`, `evaluate`) that later tasks (11-13, 18, 33-34) own — none of it is new, un-excluding just stops hiding it. Also fixed: `x verify`'s step count drifted to 13 in two README callouts after the step list grew to 15. Co-Authored-By: Claude <noreply@anthropic.com>
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 12 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: ASSERTIVE Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (40)
📝 WalkthroughWalkthroughThis PR centralizes ChangesVerification and test infrastructure
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant VerifyScript as scripts/verify.ts
participant CLI as runVerify
participant Step as VerifyStep
participant Host as HostCheck
User->>VerifyScript: run x verify
VerifyScript->>CLI: runVerify(steps, ctx)
CLI->>Step: execute fixed step list
Step->>Host: run mapped host checks
Host-->>Step: findings
Step-->>CLI: step outcome
CLI-->>VerifyScript: reports + exit code
VerifyScript-->>User: text or JSON result
sequenceDiagram
participant Test
participant Server
participant Core as listeners.ts
participant Seal as sealed-network.ts
Test->>Server: start()
Server->>Core: markListening(origin)
Test->>Seal: fetch(origin)
Seal->>Core: isSelfOrigin(url)
Core-->>Seal: true
Seal-->>Test: allow request
Test->>Server: stop()
Server->>Core: release listener
Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/testing/src/fixtures.ts (1)
93-106: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftDo not infer required fixtures from
Function.prototype.toString().
FixtureBodypermitsfixtureTest('x', (fixtures) => fixtures.clock.now()). For that callback,requestedFixtures()finds the function-body brace, returns no fixture names, and line 106 passes an empty bag asFixtures. The test then fails at runtime when it readsfixtures.clock.Accept an explicit fixture list, or construct all registered fixtures. Do not use source-text parsing as the fixture contract.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/testing/src/fixtures.ts` around lines 93 - 106, Update fixtureTest and requestedFixtures so fixture dependencies are not inferred from Function.prototype.toString() or source-text parsing. Change the API to accept an explicit fixture list and use it to populate the bag, or consistently construct every entry from registeredFixtures() before invoking the body; ensure callbacks such as fixtures.clock.now() receive a defined clock fixture.
🧹 Nitpick comments (9)
packages/testing/src/sealed-network.test.ts (1)
13-17: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winParse the rejected value before reading
code.The type assertion trusts an arbitrary rejection value. Use a runtime guard, such as
isUltimateError(), before returning an object withcode.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/testing/src/sealed-network.test.ts` around lines 13 - 17, Update the refusalOf helper to validate the rejected value with isUltimateError() before accessing or returning its code; preserve undefined for successful requests and unrecognized rejection values.Source: Coding guidelines
packages/cli/src/verify-tests.ts (1)
39-43: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low valueThe bare
e2efilter can claim a path the unit step also claims.
bun test e2ematches the substringe2eanywhere in a path. The unit step excludes only**/e2e/**and the typed suffixes. A path such aspackages/foo/src/e2e-utils.test.tstherefore runs in both steps:unitdoes not exclude it, ande2eselects it. The header comment on line 4 states that no test falls between two steps; the reverse case, one test in two steps, is not prevented.Consider anchoring the
e2eselection the same way the globs on line 42 do, so the filter and theappliesglobs agree.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/src/verify-tests.ts` around lines 39 - 43, The e2e filter in the e2e configuration matches any path containing “e2e”, overlapping with the unit step. Update the filter to use the same anchored path patterns represented by the e2e applies globs, ensuring files such as e2e-utils.test.ts are selected only by the intended step.packages/cli/src/cmd-verify.test.ts (1)
108-128: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExercise
hostFindingsinstead of re-implementing it in the test step.The stub step reads
context.hostChecks?.boundaries?.()directly. The test then proves only thatrunVerifyforwardsctx. It does not coverhostFindingsfromverify-step.ts, which is the helper the realboundariesandmanifeststeps use and whichscripts/verify.tsdepends on throughHOST_CHECKS. A regression insidehostFindingswould keep this test green.♻️ Proposed test change
+import { fromFindings, hostFindings } from './verify-step'; + const withHost: readonly VerifyStep[] = [ { name: 'boundaries', summary: 'imports', - run: async (context) => { - const extra = (await context.hostChecks?.boundaries?.(context.root)) ?? []; - return { ok: extra.length === 0, findings: extra }; - }, + run: async (context) => fromFindings(await hostFindings(context, 'boundaries')), }, ];🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/src/cmd-verify.test.ts` around lines 108 - 128, Update the test step in “a host check adds findings to the step it was registered for” to call the hostFindings helper from verify-step.ts instead of directly invoking context.hostChecks.boundaries. Preserve the existing boundary host-check stub and assertions so the test verifies that runVerify propagates findings produced through hostFindings.packages/cli/src/verify-tests.test.ts (1)
64-78: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe test binds itself to the real repository root instead of a fixture. Both findings in this file come from one root cause:
REPO_ROOTis derived fromimport.meta.urland the positive assertions read the repository's own file inventory. A self-contained fixture directory removes the coupling and removes the need forREPO_ROOTat all.
packages/cli/src/verify-tests.test.ts#L64-L78: create a second temporary directory containingapp/pay.contract.test.tsande2e/checkout.test.ts, assertapplies('contract', …)andapplies('e2e', …)aretrueagainst it, and remove the twoREPO_ROOTassertions.packages/cli/src/verify-tests.test.ts#L8-L8: delete theREPO_ROOTconstant once the fixture covers both branches. If any use remains, convert the file URL withfileURLToPathinstead of reading.pathname.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/src/verify-tests.test.ts` around lines 64 - 78, Make the test self-contained by extending the temporary fixture in the test named “a type with no suites here is skipped, never silently passed” with app/pay.contract.test.ts and e2e/checkout.test.ts, assert both contract and e2e applies results against that fixture, and remove the REPO_ROOT assertions. Delete the REPO_ROOT constant near the file setup; no direct change is needed elsewhere.scripts/verify.ts (2)
2-8: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winShorten the file header.
The header has more than four lines. Keep one concise responsibility-and-rationale comment. Move invocation detail to command help or documentation.
As per coding guidelines, “Keep file headers/comments concise: 1–4 lines stating the file's single responsibility, and comments should explain why, not what.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/verify.ts` around lines 2 - 8, Shorten the header comment in scripts/verify.ts to a single concise 1–4 line statement explaining the file’s responsibility and rationale. Remove the detailed implementation description and command invocation example, leaving usage guidance to command help or documentation.Source: Coding guidelines
47-48: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse Bun-native APIs, or document each required Node compatibility use.
The new runtime script uses
process. The new tests importnode:modules. Replace these uses with Bun APIs where available. If a Node API is necessary, add a concise comment that states why Bun has no suitable alternative.
scripts/verify.ts#L47-L48: replaceprocess.stdout.writeandprocess.exit, or document the compatibility requirement.scripts/verify.test.ts#L2-L4: document why temporary-directory and cleanup operations requirenode:APIs.packages/cli/src/workspace-checks.test.ts#L2-L4: document why temporary-directory and cleanup operations requirenode:APIs.Based on learnings, “Use Bun only; avoid Node-specific APIs unless unavoidable via
node:and documented with a comment explaining why.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/verify.ts` around lines 47 - 48, Use Bun-native output and exit APIs in scripts/verify.ts at the render/result handling flow, replacing process.stdout.write and process.exit. In scripts/verify.test.ts lines 2-4 and packages/cli/src/workspace-checks.test.ts lines 2-4, retain the node: temporary-directory and cleanup imports only if necessary, and add concise comments explaining why Bun lacks suitable alternatives.Source: Learnings
packages/cli/src/errors.ts (1)
51-51: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRoute new CLI output text through the translator.
These new literals are visible in CLI help or findings. Use stable translation keys and the project translator.
packages/cli/src/errors.ts#L51-L51: translate theX_VERIFY_FAILEDfix text.packages/cli/src/cmd-build.ts#L84-L84: translate the non-Docker build failure fix text.scripts/verify.ts#L27-L31: translate the manifest finding cause and fix text.scripts/help.ts#L21-L21: translate the verify help description.As per coding guidelines, “Do not hardcode user-facing strings; route them through
t().”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/src/errors.ts` at line 51, Route all specified user-facing literals through the project translator using stable translation keys and t(): update the X_VERIFY_FAILED fix text in packages/cli/src/errors.ts:51, the non-Docker build failure fix text in packages/cli/src/cmd-build.ts:84, the manifest finding cause and fix text in scripts/verify.ts:27-31, and the verify help description in scripts/help.ts:21. Preserve the existing messages while removing their hardcoded user-facing forms.Source: Coding guidelines
packages/testing/src/fixtures.ts (1)
22-33: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueShorten this interface comment.
Lines 22-33 contain a 12-line usage example. Keep the contract comment to 1–4 lines. Move the augmentation example to the package README.
As per coding guidelines, “Keep file headers/comments concise: 1–4 lines.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/testing/src/fixtures.ts` around lines 22 - 33, Shorten the `Fixtures` interface comment in `packages/testing/src/fixtures.ts` so it stays within 1–4 lines and keeps only the contract summary; remove the inline module-augmentation example from this comment. Preserve the key description on what a test body receives and how apps extend `Fixtures`, and move the longer `declare module '`@ultimat3/testing`'` usage example to the package README instead.Source: Coding guidelines
packages/testing/src/fixture-jobs.ts (1)
1-6: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueShorten the file headers.
Keep each header to a 1–4 line responsibility statement. Move detailed rationale next to the non-obvious behavior it explains.
packages/testing/src/fixture-jobs.ts#L1-L6: Reduce the worker-fixture header to its responsibility.packages/testing/src/framework-fixtures.ts#L1-L7: Reduce the framework-registration header to its responsibility.packages/testing/src/preload.ts#L1-L6: Reduce the preload header to its responsibility.scripts/test-setup.ts#L2-L6: Reduce the root test-setup header to its responsibility.As per coding guidelines: “Keep file headers/comments concise: 1–4 lines stating the file's single responsibility.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/testing/src/fixture-jobs.ts` around lines 1 - 6, Shorten the file headers to concise 1–4 line responsibility statements: in packages/testing/src/fixture-jobs.ts lines 1-6 describe the job fixture’s responsibility; in packages/testing/src/framework-fixtures.ts lines 1-7 describe framework fixture registration; in packages/testing/src/preload.ts lines 1-6 describe preload setup; and in scripts/test-setup.ts lines 2-6 describe root test setup. Move detailed rationale to the nearby non-obvious behavior it explains.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yml:
- Line 116: Update the actions/checkout step in the CI workflow to disable
credential persistence by setting persist-credentials to false on the checkout
action. Keep the existing checkout behavior otherwise unchanged, and apply the
change to the checkout step used in the pull_request job so checked-out code
cannot access the stored GITHUB_TOKEN.
In `@packages/cli/src/workspace-checks.ts`:
- Around line 41-53: Update the line-count calculation in checkFileSizes so a
terminal newline does not create an extra empty line; count content lines
according to the documented 500-line ceiling, while preserving the existing
tooLongFinding behavior and file scanning logic.
In `@packages/testing/src/fixture-jobs.ts`:
- Around line 75-79: Add a jobs-package reset function that clears the
module-level driver set by setJobDriver, then register it in the fixture’s
afterEach cleanup alongside resetMailDriver. Update createRunJobs and related
imports as needed so each test starts without the previous ambient job driver.
In `@packages/testing/src/framework-fixtures.test.ts`:
- Around line 40-56: Update every bunTest and fixtureTest declaration in this
file, including the tests around the shown sections and ranges 61-80 and 99-156,
to wrap its literal name with testName('unit', '<name>'). Preserve each existing
test name and body while ensuring all declarations use the required prefixed
naming format.
In `@packages/testing/src/sealed-network.test.ts`:
- Line 61: Update the new test declaration in sealed-network.test.ts to wrap its
test name with testName(type, name), preserving the existing test description
while ensuring x verify filtering remains stable.
In `@wiki/CLI-Reference.md`:
- Around line 160-164: Update the verify command JSON example to remove the
skipped:false property from the executed budgets step, preserving skipped only
for steps that do not apply as described by runVerify.
---
Outside diff comments:
In `@packages/testing/src/fixtures.ts`:
- Around line 93-106: Update fixtureTest and requestedFixtures so fixture
dependencies are not inferred from Function.prototype.toString() or source-text
parsing. Change the API to accept an explicit fixture list and use it to
populate the bag, or consistently construct every entry from
registeredFixtures() before invoking the body; ensure callbacks such as
fixtures.clock.now() receive a defined clock fixture.
---
Nitpick comments:
In `@packages/cli/src/cmd-verify.test.ts`:
- Around line 108-128: Update the test step in “a host check adds findings to
the step it was registered for” to call the hostFindings helper from
verify-step.ts instead of directly invoking context.hostChecks.boundaries.
Preserve the existing boundary host-check stub and assertions so the test
verifies that runVerify propagates findings produced through hostFindings.
In `@packages/cli/src/errors.ts`:
- Line 51: Route all specified user-facing literals through the project
translator using stable translation keys and t(): update the X_VERIFY_FAILED fix
text in packages/cli/src/errors.ts:51, the non-Docker build failure fix text in
packages/cli/src/cmd-build.ts:84, the manifest finding cause and fix text in
scripts/verify.ts:27-31, and the verify help description in scripts/help.ts:21.
Preserve the existing messages while removing their hardcoded user-facing forms.
In `@packages/cli/src/verify-tests.test.ts`:
- Around line 64-78: Make the test self-contained by extending the temporary
fixture in the test named “a type with no suites here is skipped, never silently
passed” with app/pay.contract.test.ts and e2e/checkout.test.ts, assert both
contract and e2e applies results against that fixture, and remove the REPO_ROOT
assertions. Delete the REPO_ROOT constant near the file setup; no direct change
is needed elsewhere.
In `@packages/cli/src/verify-tests.ts`:
- Around line 39-43: The e2e filter in the e2e configuration matches any path
containing “e2e”, overlapping with the unit step. Update the filter to use the
same anchored path patterns represented by the e2e applies globs, ensuring files
such as e2e-utils.test.ts are selected only by the intended step.
In `@packages/testing/src/fixture-jobs.ts`:
- Around line 1-6: Shorten the file headers to concise 1–4 line responsibility
statements: in packages/testing/src/fixture-jobs.ts lines 1-6 describe the job
fixture’s responsibility; in packages/testing/src/framework-fixtures.ts lines
1-7 describe framework fixture registration; in packages/testing/src/preload.ts
lines 1-6 describe preload setup; and in scripts/test-setup.ts lines 2-6
describe root test setup. Move detailed rationale to the nearby non-obvious
behavior it explains.
In `@packages/testing/src/fixtures.ts`:
- Around line 22-33: Shorten the `Fixtures` interface comment in
`packages/testing/src/fixtures.ts` so it stays within 1–4 lines and keeps only
the contract summary; remove the inline module-augmentation example from this
comment. Preserve the key description on what a test body receives and how apps
extend `Fixtures`, and move the longer `declare module '`@ultimat3/testing`'`
usage example to the package README instead.
In `@packages/testing/src/sealed-network.test.ts`:
- Around line 13-17: Update the refusalOf helper to validate the rejected value
with isUltimateError() before accessing or returning its code; preserve
undefined for successful requests and unrecognized rejection values.
In `@scripts/verify.ts`:
- Around line 2-8: Shorten the header comment in scripts/verify.ts to a single
concise 1–4 line statement explaining the file’s responsibility and rationale.
Remove the detailed implementation description and command invocation example,
leaving usage guidance to command help or documentation.
- Around line 47-48: Use Bun-native output and exit APIs in scripts/verify.ts at
the render/result handling flow, replacing process.stdout.write and
process.exit. In scripts/verify.test.ts lines 2-4 and
packages/cli/src/workspace-checks.test.ts lines 2-4, retain the node:
temporary-directory and cleanup imports only if necessary, and add concise
comments explaining why Bun lacks suitable alternatives.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4771a31e-1153-493d-859e-eedf2098f27d
⛔ Files ignored due to path filters (1)
bun.lockis excluded by!**/*.lock
📒 Files selected for processing (57)
.github/workflows/ci.ymlCLAUDE.mdCONTRIBUTING.mdbunfig.tomldocs/architecture/14-testing-internals.mddocs/architecture/README.mdexamples/dummy/CLAUDE.mdexamples/dummy/bunfig.tomlexamples/dummy/scripts/test-setup.tsexamples/dummy/tsconfig.jsonpackage.jsonpackages/cli/README.mdpackages/cli/src/cmd-build.tspackages/cli/src/cmd-verify.test.tspackages/cli/src/cmd-verify.tspackages/cli/src/errors.tspackages/cli/src/index.tspackages/cli/src/surfaces.tspackages/cli/src/verify-step.tspackages/cli/src/verify-tests.test.tspackages/cli/src/verify-tests.tspackages/cli/src/workspace-checks.test.tspackages/cli/src/workspace-checks.tspackages/core/CLAUDE.mdpackages/core/README.mdpackages/core/src/index.tspackages/core/src/listeners.test.tspackages/core/src/listeners.tspackages/http/CLAUDE.mdpackages/http/e2e/server.e2e.test.tspackages/http/src/server.tspackages/realtime/src/sync-node.tspackages/testing/CLAUDE.mdpackages/testing/README.mdpackages/testing/package.jsonpackages/testing/src/fixture-clock.tspackages/testing/src/fixture-jobs.tspackages/testing/src/fixture-mail.tspackages/testing/src/fixtures.test.tspackages/testing/src/fixtures.tspackages/testing/src/framework-fixtures.test.tspackages/testing/src/framework-fixtures.tspackages/testing/src/index.tspackages/testing/src/preload.tspackages/testing/src/sealed-network.test.tspackages/testing/src/sealed-network.tspackages/testing/tsconfig.jsonpackages/ui/src/index.tsscripts/help.tsscripts/lib/steps.tsscripts/test-setup.tsscripts/verify.test.tsscripts/verify.tstsconfig.jsonwiki/CLI-Reference.mdwiki/Error-Codes.mdwiki/Policies-And-Authz.md
💤 Files with no reviewable changes (1)
- scripts/lib/steps.ts
📜 Review details
⚠️ CI failures not shown inline (6)
GitHub Actions: ci / 5_test.txt: ci: gate integrity — stop hiding failing suites, gate the app too
Conclusion: failure
##[group]packages/mail/src/mail.test.ts:
(pass) send without a locale throws X_MAIL_LOCALE_MISSING [1.05ms]
(pass) send with an empty locale throws X_MAIL_LOCALE_MISSING [0.12ms]
73 | });
74 |
75 | test('a valid locale renders both parts and reaches the memory driver', async () => {
76 | const result = await send(basicMail, { name: 'Ada' }, { to: 'ada@example.test', locale: 'en' });
77 |
78 | expect(result.driver).toBe('memory');
^
error: expect(received).toBe(expected)
Expected: "memory"
Received: "queue"
at <anonymous> (/home/runner/work/ultimate/ultimate/packages/mail/src/mail.test.ts:78:25)
##[error]Expected: "memory"
GitHub Actions: ci / 1_reference-app-verify.txt: ci: gate integrity — stop hiding failing suites, gate the app too
Conclusion: failure
##[group]Run bun run ../../packages/cli/src/bin.ts verify --json
�[36;1mbun run ../../packages/cli/src/bin.ts verify --json�[0m
shell: /usr/bin/bash -e {0}
env:
ULTIMATE_TEST_SEED: 20260101
##[endgroup]
{"ok":false,"command":"verify","summary":"8 of 15 steps failed","steps":[{"name":"typecheck","ok":false,"durationMs":885,"skipped":false,"findings":[{"code":"X_TYPECHECK_FAILED","cause":"the project does not typecheck","fix":"bunx tsc -b --pretty false","docs":"https://ultimate.dev/errors/X_TYPECHECK_FAILED"}]},{"name":"lint","ok":true,"durationMs":128,"skipped":false,"findings":[]},{"name":"boundaries","ok":true,"durationMs":9,"skipped":false,"findings":[]},{"name":"filesize","ok":true,"durationMs":6,"skipped":false,"findings":[]},{"name":"package-shape","ok":true,"durationMs":0,"skipped":false,"findings":[]},{"name":"unit","ok":true,"durationMs":74,"skipped":false,"findings":[]},{"name":"contract","ok":false,"durationMs":83,"skipped":false,"findings":[{"code":"X_TEST_FAILED","cause":"one or more contract tests failed","fix":"bun test --path-ignore-patterns=**/dist/** --path-ignore-patterns=**/build/** .contract.test.","docs":"https://ultimate.dev/errors/X_TEST_FAILED"}]},{"name":"live","ok":false,"durationMs":72,"skipped":false,"findings":[{"code":"X_TEST_FAILED","cause":"one or more live tests failed","fix":"bun test --path-ignore-patterns=**/dist/** --path-ignore-patterns=**/build/** .live.test.","docs":"https://ultimate.dev/errors/X_TEST_FAILED"}]},{"name":"job","ok":false,"durationMs":65,"skipped":false,"findings":[{"code":"X_TEST_FAILED","cause":"one or more job tests failed","fix":"bun test --path-ignore-patterns=**/dist/** --path-ignore-patterns=**/build/** .job.test.","docs":"https://ultimate.dev/errors/X_TEST_FAILED"}]},{"name":"e2e","ok":false,"durationMs":41,"skipped":false,"findings":[{"code":"X_TEST_FAILED","cause":"one or more e2e tests failed","fix":"bun test --path-ignore-patterns=**/dist/** --path-ignore-patterns=**/build/** e2e","docs":...
GitHub Actions: ci / reference-app-verify: ci: gate integrity — stop hiding failing suites, gate the app too
Conclusion: failure
##[group]Run bun run ../../packages/cli/src/bin.ts verify --json
�[36;1mbun run ../../packages/cli/src/bin.ts verify --json�[0m
shell: /usr/bin/bash -e {0}
env:
ULTIMATE_TEST_SEED: 20260101
##[endgroup]
{"ok":false,"command":"verify","summary":"8 of 15 steps failed","steps":[{"name":"typecheck","ok":false,"durationMs":885,"skipped":false,"findings":[{"code":"X_TYPECHECK_FAILED","cause":"the project does not typecheck","fix":"bunx tsc -b --pretty false","docs":"https://ultimate.dev/errors/X_TYPECHECK_FAILED"}]},{"name":"lint","ok":true,"durationMs":128,"skipped":false,"findings":[]},{"name":"boundaries","ok":true,"durationMs":9,"skipped":false,"findings":[]},{"name":"filesize","ok":true,"durationMs":6,"skipped":false,"findings":[]},{"name":"package-shape","ok":true,"durationMs":0,"skipped":false,"findings":[]},{"name":"unit","ok":true,"durationMs":74,"skipped":false,"findings":[]},{"name":"contract","ok":false,"durationMs":83,"skipped":false,"findings":[{"code":"X_TEST_FAILED","cause":"one or more contract tests failed","fix":"bun test --path-ignore-patterns=**/dist/** --path-ignore-patterns=**/build/** .contract.test.","docs":"https://ultimate.dev/errors/X_TEST_FAILED"}]},{"name":"live","ok":false,"durationMs":72,"skipped":false,"findings":[{"code":"X_TEST_FAILED","cause":"one or more live tests failed","fix":"bun test --path-ignore-patterns=**/dist/** --path-ignore-patterns=**/build/** .live.test.","docs":"https://ultimate.dev/errors/X_TEST_FAILED"}]},{"name":"job","ok":false,"durationMs":65,"skipped":false,"findings":[{"code":"X_TEST_FAILED","cause":"one or more job tests failed","fix":"bun test --path-ignore-patterns=**/dist/** --path-ignore-patterns=**/build/** .job.test.","docs":"https://ultimate.dev/errors/X_TEST_FAILED"}]},{"name":"e2e","ok":false,"durationMs":41,"skipped":false,"findings":[{"code":"X_TEST_FAILED","cause":"one or more e2e tests failed","fix":"bun test --path-ignore-patterns=**/dist/** --path-ignore-patterns=**/build/** e2e","docs":...
GitHub Actions: ci / test: ci: gate integrity — stop hiding failing suites, gate the app too
Conclusion: failure
##[group]packages/mail/src/mail.test.ts:
(pass) send without a locale throws X_MAIL_LOCALE_MISSING [1.05ms]
(pass) send with an empty locale throws X_MAIL_LOCALE_MISSING [0.12ms]
73 | });
74 |
75 | test('a valid locale renders both parts and reaches the memory driver', async () => {
76 | const result = await send(basicMail, { name: 'Ada' }, { to: 'ada@example.test', locale: 'en' });
77 |
78 | expect(result.driver).toBe('memory');
^
error: expect(received).toBe(expected)
Expected: "memory"
Received: "queue"
at <anonymous> (/home/runner/work/ultimate/ultimate/packages/mail/src/mail.test.ts:78:25)
##[error]Expected: "memory"
GitHub Actions: ci / typecheck: ci: gate integrity — stop hiding failing suites, gate the app too
Conclusion: failure
ummy/apps/web/app/settings.tsx�[0m:�[93m26�[0m:�[93m30�[0m - �[91merror�[0m�[90m TS18046: �[0m't' is of type 'unknown'.
�[7m26�[0m meta: ({ t }) => ({ title: t('app.settings.metaTitle'), robots: 'noindex' }),
�[7m �[0m �[91m ~�[0m
�[96mexamples/dummy/apps/web/app/settings.tsx�[0m:�[93m26�[0m:�[93m59�[0m - �[91merror�[0m�[90m TS2559: �[0mType 'string' has no properties in common with type 'RobotsDirectives'.
�[7m26�[0m meta: ({ t }) => ({ title: t('app.settings.metaTitle'), robots: 'noindex' }),
�[7m �[0m �[91m ~~~~~~�[0m
�[96mexamples/dummy/apps/web/app/settings.tsx�[0m:�[93m51�[0m:�[93m14�[0m - �[91merror�[0m�[90m TS2322: �[0mType 'string' is not assignable to type 'SpaceStep | undefined'.
�[7m51�[0m <Stack gap="6" class={styles.page}>
�[7m �[0m �[91m ~~~�[0m
�[96mpackages/ui/src/components/Stack.tsx�[0m:�[93m13�[0m:�[93m3�[0m - The expected type comes from property 'gap' which is declared here on type 'IntrinsicAttributes & StackProps'
�[7m13�[0m gap?: SpaceStep | undefined;
�[7m �[0m �[96m ~~~�[0m
�[96mexamples/dummy/apps/web/app/settings.tsx�[0m:�[93m58�[0m:�[93m11�[0m - �[91merror�[0m�[90m TS2322: �[0mType '{ label: any; hint: any; value: any; onChange: Setter<any>; children: Element; }' is not assignable to type 'IntrinsicAttributes & SelectProps'.
Property 'label' does not exist on type 'IntrinsicAttributes & SelectProps'.
�[7m58�[0m label={t('app.settings.localeLabel')}
�[7m �[0m �[91m ~~~~~�[0m
�[96mexamples/dummy/apps/web/app/settings.tsx�[0m:�[93m63�[0m:�[93m67�[0m - �[91merror�[0m�[90m TS2322: �[0mType 'Accessor<"en" | "es">' is not assignable to type 'string | number | string[] | RemoveAttribute'.
�[7m63�[0m <For each={SUPPORTED_LOCALES}>{(value) => <option value={value}>{value}</option>}</For>
�[7m �[0m �[91m ~~~~...
GitHub Actions: ci / 2_typecheck.txt: ci: gate integrity — stop hiding failing suites, gate the app too
Conclusion: failure
ummy/apps/web/app/settings.tsx�[0m:�[93m26�[0m:�[93m30�[0m - �[91merror�[0m�[90m TS18046: �[0m't' is of type 'unknown'.
�[7m26�[0m meta: ({ t }) => ({ title: t('app.settings.metaTitle'), robots: 'noindex' }),
�[7m �[0m �[91m ~�[0m
�[96mexamples/dummy/apps/web/app/settings.tsx�[0m:�[93m26�[0m:�[93m59�[0m - �[91merror�[0m�[90m TS2559: �[0mType 'string' has no properties in common with type 'RobotsDirectives'.
�[7m26�[0m meta: ({ t }) => ({ title: t('app.settings.metaTitle'), robots: 'noindex' }),
�[7m �[0m �[91m ~~~~~~�[0m
�[96mexamples/dummy/apps/web/app/settings.tsx�[0m:�[93m51�[0m:�[93m14�[0m - �[91merror�[0m�[90m TS2322: �[0mType 'string' is not assignable to type 'SpaceStep | undefined'.
�[7m51�[0m <Stack gap="6" class={styles.page}>
�[7m �[0m �[91m ~~~�[0m
�[96mpackages/ui/src/components/Stack.tsx�[0m:�[93m13�[0m:�[93m3�[0m - The expected type comes from property 'gap' which is declared here on type 'IntrinsicAttributes & StackProps'
�[7m13�[0m gap?: SpaceStep | undefined;
�[7m �[0m �[96m ~~~�[0m
�[96mexamples/dummy/apps/web/app/settings.tsx�[0m:�[93m58�[0m:�[93m11�[0m - �[91merror�[0m�[90m TS2322: �[0mType '{ label: any; hint: any; value: any; onChange: Setter<any>; children: Element; }' is not assignable to type 'IntrinsicAttributes & SelectProps'.
Property 'label' does not exist on type 'IntrinsicAttributes & SelectProps'.
�[7m58�[0m label={t('app.settings.localeLabel')}
�[7m �[0m �[91m ~~~~~�[0m
�[96mexamples/dummy/apps/web/app/settings.tsx�[0m:�[93m63�[0m:�[93m67�[0m - �[91merror�[0m�[90m TS2322: �[0mType 'Accessor<"en" | "es">' is not assignable to type 'string | number | string[] | RemoveAttribute'.
�[7m63�[0m <For each={SUPPORTED_LOCALES}>{(value) => <option value={value}>{value}</option>}</For>
�[7m �[0m �[91m ~~~~...
🧰 Additional context used
📓 Path-based instructions (19)
**/{*.ts,*.md}
📄 CodeRabbit inference engine (CLAUDE.md)
Use kebab-case file names for source files, and keep the file's responsibility narrow.
Files:
docs/architecture/14-testing-internals.mdexamples/dummy/CLAUDE.mdpackages/testing/CLAUDE.mdexamples/dummy/scripts/test-setup.tspackages/core/src/index.tspackages/core/CLAUDE.mdpackages/testing/src/sealed-network.tspackages/core/README.mdpackages/cli/src/surfaces.tspackages/http/e2e/server.e2e.test.tsscripts/test-setup.tsdocs/architecture/README.mdCLAUDE.mdpackages/cli/src/verify-step.tswiki/Policies-And-Authz.mdCONTRIBUTING.mdpackages/cli/README.mdpackages/testing/src/fixtures.tsscripts/verify.test.tspackages/testing/src/preload.tspackages/http/CLAUDE.mdpackages/http/src/server.tspackages/cli/src/errors.tspackages/testing/src/index.tspackages/testing/src/sealed-network.test.tswiki/CLI-Reference.mdpackages/testing/src/fixtures.test.tspackages/cli/src/verify-tests.test.tsscripts/verify.tspackages/cli/src/index.tspackages/ui/src/index.tspackages/testing/src/fixture-mail.tspackages/cli/src/verify-tests.tspackages/testing/src/framework-fixtures.tspackages/realtime/src/sync-node.tspackages/core/src/listeners.tspackages/cli/src/cmd-verify.tspackages/cli/src/workspace-checks.tspackages/core/src/listeners.test.tspackages/testing/src/framework-fixtures.test.tspackages/testing/src/fixture-jobs.tspackages/cli/src/cmd-build.tspackages/testing/src/fixture-clock.tspackages/testing/README.mdscripts/help.tspackages/cli/src/workspace-checks.test.tswiki/Error-Codes.mdpackages/cli/src/cmd-verify.test.ts
**/*.{ts,tsx,md}
📄 CodeRabbit inference engine (CLAUDE.md)
Keep file headers/comments concise: 1–4 lines stating the file's single responsibility, and comments should explain why, not what.
Files:
docs/architecture/14-testing-internals.mdexamples/dummy/CLAUDE.mdpackages/testing/CLAUDE.mdexamples/dummy/scripts/test-setup.tspackages/core/src/index.tspackages/core/CLAUDE.mdpackages/testing/src/sealed-network.tspackages/core/README.mdpackages/cli/src/surfaces.tspackages/http/e2e/server.e2e.test.tsscripts/test-setup.tsdocs/architecture/README.mdCLAUDE.mdpackages/cli/src/verify-step.tswiki/Policies-And-Authz.mdCONTRIBUTING.mdpackages/cli/README.mdpackages/testing/src/fixtures.tsscripts/verify.test.tspackages/testing/src/preload.tspackages/http/CLAUDE.mdpackages/http/src/server.tspackages/cli/src/errors.tspackages/testing/src/index.tspackages/testing/src/sealed-network.test.tswiki/CLI-Reference.mdpackages/testing/src/fixtures.test.tspackages/cli/src/verify-tests.test.tsscripts/verify.tspackages/cli/src/index.tspackages/ui/src/index.tspackages/testing/src/fixture-mail.tspackages/cli/src/verify-tests.tspackages/testing/src/framework-fixtures.tspackages/realtime/src/sync-node.tspackages/core/src/listeners.tspackages/cli/src/cmd-verify.tspackages/cli/src/workspace-checks.tspackages/core/src/listeners.test.tspackages/testing/src/framework-fixtures.test.tspackages/testing/src/fixture-jobs.tspackages/cli/src/cmd-build.tspackages/testing/src/fixture-clock.tspackages/testing/README.mdscripts/help.tspackages/cli/src/workspace-checks.test.tswiki/Error-Codes.mdpackages/cli/src/cmd-verify.test.ts
**/*.md
📄 CodeRabbit inference engine (CLAUDE.md)
Docs should lead with the rule, prefer fragments over sentences, use tables for structures with three or more rows, avoid meta-framing and trailing summaries, and mark load-bearing date claims with
As of 2026-07.
Files:
docs/architecture/14-testing-internals.mdexamples/dummy/CLAUDE.mdpackages/testing/CLAUDE.mdpackages/core/CLAUDE.mdpackages/core/README.mddocs/architecture/README.mdCLAUDE.mdwiki/Policies-And-Authz.mdCONTRIBUTING.mdpackages/cli/README.mdpackages/http/CLAUDE.mdwiki/CLI-Reference.mdpackages/testing/README.mdwiki/Error-Codes.md
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Do not useany; useunknownplus schema parsing instead.
Never throw a bareError; throw a subclass ofUltimateErrorwith a stable code, a cause, and an executablefix:.
Prefer named exports only; do not use default exports.
Useimport typeandexport typefor type-only imports and exports.
Never format dates without an explicit IANAtimeZone; do not rely on ambient defaults.
Represent money asMoney = { minor: number; currency: string }; never use float money.
Files:
examples/dummy/scripts/test-setup.tspackages/core/src/index.tspackages/testing/src/sealed-network.tspackages/cli/src/surfaces.tspackages/http/e2e/server.e2e.test.tsscripts/test-setup.tspackages/cli/src/verify-step.tspackages/testing/src/fixtures.tsscripts/verify.test.tspackages/testing/src/preload.tspackages/http/src/server.tspackages/cli/src/errors.tspackages/testing/src/index.tspackages/testing/src/sealed-network.test.tspackages/testing/src/fixtures.test.tspackages/cli/src/verify-tests.test.tsscripts/verify.tspackages/cli/src/index.tspackages/ui/src/index.tspackages/testing/src/fixture-mail.tspackages/cli/src/verify-tests.tspackages/testing/src/framework-fixtures.tspackages/realtime/src/sync-node.tspackages/core/src/listeners.tspackages/cli/src/cmd-verify.tspackages/cli/src/workspace-checks.tspackages/core/src/listeners.test.tspackages/testing/src/framework-fixtures.test.tspackages/testing/src/fixture-jobs.tspackages/cli/src/cmd-build.tspackages/testing/src/fixture-clock.tsscripts/help.tspackages/cli/src/workspace-checks.test.tspackages/cli/src/cmd-verify.test.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx,js,jsx}: Every CLI command and every error output must support--json.
Do not hardcode user-facing strings; route them throught().
Files:
examples/dummy/scripts/test-setup.tspackages/core/src/index.tspackages/testing/src/sealed-network.tspackages/cli/src/surfaces.tspackages/http/e2e/server.e2e.test.tsscripts/test-setup.tspackages/cli/src/verify-step.tspackages/testing/src/fixtures.tsscripts/verify.test.tspackages/testing/src/preload.tspackages/http/src/server.tspackages/cli/src/errors.tspackages/testing/src/index.tspackages/testing/src/sealed-network.test.tspackages/testing/src/fixtures.test.tspackages/cli/src/verify-tests.test.tsscripts/verify.tspackages/cli/src/index.tspackages/ui/src/index.tspackages/testing/src/fixture-mail.tspackages/cli/src/verify-tests.tspackages/testing/src/framework-fixtures.tspackages/realtime/src/sync-node.tspackages/core/src/listeners.tspackages/cli/src/cmd-verify.tspackages/cli/src/workspace-checks.tspackages/core/src/listeners.test.tspackages/testing/src/framework-fixtures.test.tspackages/testing/src/fixture-jobs.tspackages/cli/src/cmd-build.tspackages/testing/src/fixture-clock.tsscripts/help.tspackages/cli/src/workspace-checks.test.tspackages/cli/src/cmd-verify.test.ts
**/*.{ts,tsx,css,scss}
📄 CodeRabbit inference engine (CLAUDE.md)
Do not use raw colors; use semantic tokens only in every component and stylesheet.
Files:
examples/dummy/scripts/test-setup.tspackages/core/src/index.tspackages/testing/src/sealed-network.tspackages/cli/src/surfaces.tspackages/http/e2e/server.e2e.test.tsscripts/test-setup.tspackages/cli/src/verify-step.tspackages/testing/src/fixtures.tsscripts/verify.test.tspackages/testing/src/preload.tspackages/http/src/server.tspackages/cli/src/errors.tspackages/testing/src/index.tspackages/testing/src/sealed-network.test.tspackages/testing/src/fixtures.test.tspackages/cli/src/verify-tests.test.tsscripts/verify.tspackages/cli/src/index.tspackages/ui/src/index.tspackages/testing/src/fixture-mail.tspackages/cli/src/verify-tests.tspackages/testing/src/framework-fixtures.tspackages/realtime/src/sync-node.tspackages/core/src/listeners.tspackages/cli/src/cmd-verify.tspackages/cli/src/workspace-checks.tspackages/core/src/listeners.test.tspackages/testing/src/framework-fixtures.test.tspackages/testing/src/fixture-jobs.tspackages/cli/src/cmd-build.tspackages/testing/src/fixture-clock.tsscripts/help.tspackages/cli/src/workspace-checks.test.tspackages/cli/src/cmd-verify.test.ts
examples/dummy/**/*.{ts,tsx}
📄 CodeRabbit inference engine (examples/dummy/CLAUDE.md)
examples/dummy/**/*.{ts,tsx}: Authorization must have one definition and must not be duplicated inside request handlers.
Use ArkTypetfrom@ultimat3/schemain schema, entity, and action files; use the i18n translator fromuseI18n()in components; never use both meanings in one file.
Represent money as{ minor, currency }; perform arithmetic inpackages/core/src/billing.tsand format only through<Money>at the edge.
User-facing strings must uset('<feature>.<key>'); do not hard-code user-facing text.
Plan prices are stored as per-currency catalog rows; currencies must never be converted at runtime.
Files:
examples/dummy/scripts/test-setup.ts
examples/dummy/**/*.{tsx,ts}
📄 CodeRabbit inference engine (examples/dummy/CLAUDE.md)
Store dates as UTC instants in the database and render them only through
<DateTime zone={member.tz}>.
Files:
examples/dummy/scripts/test-setup.ts
**/src/index.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Re-export the public API explicitly from
src/index.ts; do not use blindexport *.
Files:
packages/core/src/index.tspackages/testing/src/index.tspackages/cli/src/index.tspackages/ui/src/index.ts
packages/core/**/*.ts
📄 CodeRabbit inference engine (packages/core/CLAUDE.md)
packages/core/**/*.ts: The@ultimat3/corepackage must not import any@ultimat3/*package.
Core dependencies must be limited tobun-types.
Errors must subclassUltimateError; never thrownew Error.
Files:
packages/core/src/index.tspackages/core/src/listeners.tspackages/core/src/listeners.test.ts
packages/core/src/**/*.ts
📄 CodeRabbit inference engine (packages/core/CLAUDE.md)
packages/core/src/**/*.ts: Time-dependent code must take aClock; useDate.now()andnew Date()only insideclock.ts.
Do not threadctxas a parameter; obtain context withuseContext().
Keep files under 200 lines, give each file one responsibility, and usekebab-case.tsfilenames.
BecauseexactOptionalPropertyTypesis enabled, declare optional fields asx?: T | undefined.
WithnoPropertyAccessFromIndexSignatureenabled, access indexed context services asctx.services['mail'], notctx.services.mail.
Ctxmust retain its string index signature so applications can augmentCtxServicesand access services such asctx.posts.
Code that opens a socket must callmarkListening(server.url.origin)and release it when the socket closes.
Files:
packages/core/src/index.tspackages/core/src/listeners.tspackages/core/src/listeners.test.ts
packages/core/src/index.ts
📄 CodeRabbit inference engine (packages/core/CLAUDE.md)
Add public exports explicitly to
src/index.ts; never useexport *.
Files:
packages/core/src/index.ts
**/*.test.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Keep tests next to the source file as
<file>.test.ts. A test must be able to fail.
Files:
packages/http/e2e/server.e2e.test.tsscripts/verify.test.tspackages/testing/src/sealed-network.test.tspackages/testing/src/fixtures.test.tspackages/cli/src/verify-tests.test.tspackages/core/src/listeners.test.tspackages/testing/src/framework-fixtures.test.tspackages/cli/src/workspace-checks.test.tspackages/cli/src/cmd-verify.test.ts
packages/http/**/*.ts
📄 CodeRabbit inference engine (packages/http/CLAUDE.md)
packages/http/**/*.ts: The@ultimat3/httppackage may import only@ultimat3/coreand@ultimat3/schema; it must never import@ultimat3/policy,@ultimat3/entity, or higher-tier packages such as@ultimat3/action.
Route metadata must always includemeta.auth; routes must never default to public access.
Never throw a bareError; use an error factory fromerrors.ts.
Do not useany; validation must go through Standard Schema viavalidate.ts, not a vendor-specific API.
Health endpoints must answer outside the request pipeline.
Borrowed error codes such asX_FORBIDDENandX_UNAUTHENTICATEDmust remain inHTTP_BORROWED_CODESand must not be re-registered throughregisterErrorCodes.
Files:
packages/http/e2e/server.e2e.test.tspackages/http/src/server.ts
packages/http/**/*.test.ts
📄 CodeRabbit inference engine (packages/http/CLAUDE.md)
Unit and integration tests must not touch the network; the fetch preload remains sealed and must never be unsealed.
Files:
packages/http/e2e/server.e2e.test.ts
packages/http/e2e/**/*.ts
📄 CodeRabbit inference engine (packages/http/CLAUDE.md)
Socket tests belong in
e2e/and run withbun test packages/http/e2e; they must keep the network seal enabled, withstart()using core'smarkListening()so the package's own port is treated as self.
Files:
packages/http/e2e/server.e2e.test.ts
packages/testing/**/*.{test,spec}.{ts,tsx}
📄 CodeRabbit inference engine (packages/testing/CLAUDE.md)
packages/testing/**/*.{test,spec}.{ts,tsx}: Do not mock the database; clone a template database, and usetemplate-db.tsas the only database access path in tests.
Do not use the wall clock in tests; usefrozenClockandadvanceClock.Date.now()is frozen by the preload.
Do not make unmocked egress calls; rely onsealed-network.ts, where network misses fail withX_TEST_NETWORK_SEALED.
Treat the test process as network-sealed even for socket tests; a port core'smarkListening()announcement passes through without unsealing the socket.
Do not add retries such asretry: 3; fix or delete a flaky test on the day it flakes.
Always name tests throughtestName(type, name)sox verifycan filter them.
PassSqlRunnerandconnectas parameters so unit tests do not require a server.
Files:
packages/testing/src/sealed-network.test.tspackages/testing/src/fixtures.test.tspackages/testing/src/framework-fixtures.test.ts
packages/core/src/**/*.test.ts
📄 CodeRabbit inference engine (packages/core/CLAUDE.md)
Place tests beside the source they test.
Files:
packages/core/src/listeners.test.ts
packages/core/**/*.test.ts
📄 CodeRabbit inference engine (packages/core/CLAUDE.md)
Tests touching the registry, lifecycle, or listener table must call
resetErrorCodes(),resetLifecycle(), orresetListeners()respectively.
Files:
packages/core/src/listeners.test.ts
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:30:49.121Z
Learning: Follow the repo's design axioms: one way to do each thing, define once/project everywhere, enforce conventions in build errors, return actionable errors, keep `x verify` as the shippable gate, keep static and app bundles separate, and use containers only for deployability.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:30:49.121Z
Learning: Use Bun only; avoid Node-specific APIs unless unavoidable via `node:` and documented with a comment explaining why.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:30:49.121Z
Learning: Do not add new dependencies unless there is a strong reason stated in the PR.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:30:49.121Z
Learning: Every package must include a `README.md` describing the public API and a `CLAUDE.md` covering boundaries, dependencies, and commands.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:30:49.121Z
Learning: Use free GitHub Actions runners only; CI must stay under five minutes, and releases must use npm OIDC trusted publishing with no `NPM_TOKEN`.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:30:49.121Z
Learning: Do not use git worktrees; if work is large enough to need subagents, split it into disjoint work in the same checkout.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:31:20.581Z
Learning: The package owns the request lifecycle over `Bun.serve` and is Tier 2.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:31:29.394Z
Learning: The testing package may import tiers 0–4, is imported by every package's tests and generated apps, and must dynamically import `time`, `jobs`, and `mail` only inside fixture factories.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:31:29.394Z
Learning: The package exposes `.` as its API entry point and `./preload` for Bun configuration side effects.
Learnt from: CR
Repo: developerz-ai/ultimate
Timestamp: 2026-08-05T15:31:29.394Z
Learning: Use `bun test` and `bunx tsc --noEmit -p tsconfig.json` to run the test suite and TypeScript validation.
🪛 zizmor (1.29.0)
.github/workflows/ci.yml
[warning] 116-116: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
The `typecheck` and `test` jobs collected `examples/dummy`, whose sources target API the framework has not shipped yet — the fluent `action` surface, `defineMail`/`defineService`, the `page`/`subscribe`/`evaluate` fixtures. That was 449 typecheck errors and 13 test failures on every PR: no regressions, all drift the roadmap already owns, and all of it hiding whether the framework itself is green. The app already runs the identical 15-step gate in `reference-app-verify`, so nothing was covered twice for a reason. Drop it from the root tsconfig references, the root bunfig preload, `bun run test` and the gate's test steps. `reference-app-verify` still runs the whole thing and still prints every failure and fix line; it is advisory until milestone 9, because a gate whose resting state is red stops being read. One real framework bug fell out of it: the `runJobs` fixture installed the process-global job driver and never put it back, so `send()` in any later file in the same bun process enqueued instead of sending inline — which is why three `packages/mail` tests failed in CI and passed locally. Fixtures that take over a global now implement `Symbol.dispose`/`Symbol.asyncDispose`, and `fixtureTest` disposes in reverse build order even when the body throws. - jobs: `resetJobDriver()`, the counterpart to `resetMailDriver()` - testing: `runJobs` and `mail` restore the driver they replaced - cli: `applies` and the run share one exclusion list, so a step can no longer apply and then fail on "no test files matched" - cli: `x test` stopped silently dropping `e2e/`, matching `bun run test` - site: `build.ts` was 693 lines over a 500 ceiling — a `verify` failure hidden until now because the job was skipped whenever its `needs` were red. Split into `site/lib/*`, output byte-identical - docs: correct the claims about what each gate collects Co-Authored-By: Claude <noreply@anthropic.com>
CI
- reference-app-verify: continue-on-error moves from the job to the step. On
the job the check itself stayed red, which is the failure mode this PR
exists to remove; on the step the run is a warning and the whole 15-step
table stays in the log. Human render, not --json — the reader is a person.
- examples/dummy: record the schema hash for 0001_init, which covers all six
schema tables 1:1 but never had its .hash sidecar committed. `drift` green,
the app gate goes 8 red steps to 7.
- persist-credentials: false on every actions/checkout, in all four workflows.
ci.yml runs on fork pull_requests and later steps execute checked-out code.
Review
- workspace-checks: a terminating newline is not a line. Splitting on \n and
taking the length made the real ceiling 499 and every reported count one too
high — so an author who split a file to exactly 500 still failed. Extracted
`countLines`, and the fixtures now end with a newline like real files do, so
the test can fail for this.
- framework-fixtures and sealed-network tests: prefix every describe through
testName('unit', ...) so a failure line names its gate step.
- .coderabbit.yml: tone_instructions was 273 characters against a 250 limit,
so the whole file failed to parse and every review instruction in it was
dropped. 242 now, with the limit written down.
Co-Authored-By: Claude <noreply@anthropic.com>
- wiki/CLI-Reference.md: drop `skipped:false` from the `x verify --json` example. `runVerify` sets `skipped` only when a step does not apply; an executed step omits it, so the example described a shape the command never emits. - packages/testing/CLAUDE.md: the naming rule claimed `testName()` is how `x verify` filters tests. It is not — the step is selected by filename (`verify-tests.ts`), and the prefix only labels the failure line. Say that, and say it belongs on the outer `describe`, since repeating it on the inner `test` prints the type twice. Co-Authored-By: Claude <noreply@anthropic.com>
Reverts the wiki edit in 55484a7. That change was wrong: `runVerify` does omit `skipped` on an executed step, but the documented example is `x verify --json`, and `renderJson` normalises the key — `skipped: step.skipped === true` — so every step in the JSON output carries an explicit boolean. The example was right as written. Nothing tested that normalisation, which is how the doc came to read as a defect twice. Pin it: an executed step renders `"skipped":false`, a step that did not apply renders `true`. Verified the test fails when the render goes back to spreading the key conditionally. Co-Authored-By: Claude <noreply@anthropic.com>
1 participant
Summary
PR group: Gate integrity — make
verifycapable of failing. Five prior commits fixed themechanics that let the gate actually run (tsconfig, build graph, fixtures, loopback, one verify
impl). This last one removes the exclusions that were hiding the result:
bun run testno longer drops**/e2e/**or**/*.{contract,live,job,eval}.test.ts— it nowcollects exactly what bare
bun testcollects..github/workflows/ci.yml— removed the stale#9comment explaining the old exclusion; addeda
reference-app-verifyjob that runs the reference app's own 15-stepx verify(checkedindependently from the framework gate — a red app doesn't hide a green framework or vice versa).
packages/cli/README.md— the two "13 steps"/"of 13" callouts were stale since the step listgrew to 15 (prior commit in this group); corrected to 15.
Current state (documented, not fixed here)
bun test/bun run test: 1164 pass, 19 fail, 1 error — all 19+1 are inexamples/dummy;every framework package is 0 fail.
packages/http/e2e(the one opt-in suite outside thereference app) is 5/5 green.
examples/dummy's ownx verify: 8 of 15 steps red (typecheck, contract, live, job, e2e,eval, drift, manifest).
Every one of those failures traces back to two already-known, already-planned gaps:
action.as(),action.mcp,defineMail(id, config)two-arg form. The reference app is written against the designed API;@ultimat3/actionand
@ultimat3/maildon't implement it yet. Owned by tasks 11-13 ("give action its fluentsurface" / "mirror the façade on the other primitives") and 18 ("bring the reference app to
canonical shapes").
page(browser e2e harness),subscribe(live-query harness),evaluate(eval harness). Owned by tasks 33-34 and the realtime/e2e work.None of this is new or hidden by this PR — it's the same 449-error/pre-existing dummy-drift
baseline every session in this PR group has reported. This PR's job is to stop excluding it from
the signal, not to fix it; CI will show
test,typecheck, andreference-app-verifyred untilthose tasks land, by design.
Test plan
bun run test== barebun test(1164 pass / 19 fail / 1 error, confirmed identical)bun run lint,bun run boundariesgreen (unaffected)bun run scripts/verify.ts --jsoninexamples/dummyruns the 15-step gate end-to-endci.ymlvalidatedbun run typecheck) is red on this branch — pre-existing dummy-driftbaseline (owned by tasks 7-18), not a regression from this change; pushed past it since CI
surfaces the identical signal and this PR's whole point is not to hide that.
Co-Authored-By: Claude noreply@anthropic.com
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by CodeRabbit
New Features
x verifynow runs a consistent, complete set of repository and generated-app checks.Bug Fixes
Documentation