From 557dd7dbd43d366faa5f12e52e9346f41b5796bd Mon Sep 17 00:00:00 2001 From: sebi Date: Mon, 10 Aug 2026 08:41:46 -0500 Subject: [PATCH 1/4] feat(testing): register the whole framework fixture bag MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - `network` fixture: offline()/drop()/online() over an offline gate in the sealed network, ahead of the mocks so the app's own offline path runs. New X_TEST_NETWORK_OFFLINE. - `page`, `budget`, `signIn`, `deploy`, `subscribe` declared in fixture-drivers.ts: the name resolves and asking for it without a browser or a replicator fails as X_TEST_FIXTURE_UNAVAILABLE naming the driver — not X_TEST_FIXTURE_UNKNOWN, whose fix would have each app inventing its own idea of what a page is. A driver arrives through defineFixtures; no second seam. - Fixtures type widened to the full bag; PageLike grown to the members the reference app actually calls. - examples/dummy keeps only seed + actorFor. Its 11 X_TEST_FIXTURE_UNKNOWN failures are now 11 X_TEST_FIXTURE_UNAVAILABLE (6 page, 5 subscribe). Co-Authored-By: Claude --- examples/dummy/CLAUDE.md | 9 +- examples/dummy/scripts/test-setup.ts | 8 +- packages/testing/CLAUDE.md | 5 +- packages/testing/README.md | 23 +++- packages/testing/src/errors.ts | 41 +++++++ packages/testing/src/fixture-drivers.test.ts | 71 ++++++++++++ packages/testing/src/fixture-drivers.ts | 92 ++++++++++++++++ packages/testing/src/fixture-network.test.ts | 103 ++++++++++++++++++ packages/testing/src/fixture-network.ts | 58 ++++++++++ packages/testing/src/fixtures.ts | 17 ++- .../testing/src/framework-fixtures.test.ts | 19 +++- packages/testing/src/framework-fixtures.ts | 32 ++++-- packages/testing/src/index.ts | 28 ++++- packages/testing/src/sealed-network.ts | 38 ++++++- packages/testing/src/test-types.ts | 25 +++++ wiki/Error-Codes.md | 2 + wiki/Testing.md | 28 +++++ 17 files changed, 571 insertions(+), 28 deletions(-) create mode 100644 packages/testing/src/fixture-drivers.test.ts create mode 100644 packages/testing/src/fixture-drivers.ts create mode 100644 packages/testing/src/fixture-network.test.ts create mode 100644 packages/testing/src/fixture-network.ts diff --git a/examples/dummy/CLAUDE.md b/examples/dummy/CLAUDE.md index ee1da1283..c92bcf934 100644 --- a/examples/dummy/CLAUDE.md +++ b/examples/dummy/CLAUDE.md @@ -79,9 +79,12 @@ Feature slice: `apps/web/app//{entity,repo,service,actions,mutator,live - Every prompt carries `.evals.ts` (the cases) and `.vN.baseline.json` (the recorded scores). A prompt with no eval fails `x verify` with `X_EVAL_MISSING`; the gate is the drop from the baseline, so re-record with `ULTIMATE_EVAL_RECORD=1 x test eval` and commit the diff. -- Test fixtures come from `scripts/test-setup.ts`, the one preload in `bunfig.toml`. `clock`, - `mail` and `runJobs` are the framework's; `seed` and `actorFor` are Postly's. A test that - destructures anything else fails with `X_TEST_FIXTURE_UNKNOWN` — register it there, once. +- Test fixtures come from `scripts/test-setup.ts`, the one preload in `bunfig.toml`. `seed` and + `actorFor` are Postly's; everything else is the framework's — `clock`, `mail`, `network`, + `runJobs` built in-process, and `page`, `budget`, `signIn`, `deploy`, `subscribe` waiting on a + driver (`X_TEST_FIXTURE_UNAVAILABLE` until one is installed). A test that destructures a name + nobody registered fails with `X_TEST_FIXTURE_UNKNOWN` — register it there, once. Never register + a framework name here: two apps would then disagree about what a `page` is. ## Boundaries (build errors, not lint warnings) diff --git a/examples/dummy/scripts/test-setup.ts b/examples/dummy/scripts/test-setup.ts index f82af7e72..3768a3b2d 100644 --- a/examples/dummy/scripts/test-setup.ts +++ b/examples/dummy/scripts/test-setup.ts @@ -1,6 +1,8 @@ -// Postly's test preload: the fixtures the APP owns. `clock`, `mail` and `runJobs` arrive with -// the framework's own preload, imported below — an app registers only what the framework cannot -// know, which here is the seed graph and how a member becomes an actor. +// Postly's test preload: the fixtures the APP owns, which is two of them. Everything else in the +// bag — `clock`, `mail`, `network`, `runJobs`, and the driver-backed `page`, `budget`, `signIn`, +// `deploy`, `subscribe` — arrives with the framework's own preload, imported below. An app +// registers only what the framework cannot know, which here is the seed graph and how a member +// becomes an actor. Registering `page` here would be Postly deciding for itself what a page is. // // [test] // preload = ["./scripts/test-setup.ts"] diff --git a/packages/testing/CLAUDE.md b/packages/testing/CLAUDE.md index 2af8e033d..b1921b59f 100644 --- a/packages/testing/CLAUDE.md +++ b/packages/testing/CLAUDE.md @@ -11,10 +11,13 @@ factories only**, so a test that never destructures `mail` never loads the mail | No wall clock | `frozenClock` / `advanceClock`; `Date.now()` is frozen by the preload | | No unmocked egress | `sealed-network.ts` patches fetch; a miss is `X_TEST_NETWORK_SEALED` | | Self is not egress | a port core's `markListening()` announced passes through — a socket test never unseals | +| Offline is a state, not a mock | `network.offline()` / `.drop()` fail every request as `X_TEST_NETWORK_OFFLINE`, ahead of the mocks — the app's own offline path runs | | No retries | a flake is fixed or deleted the day it flakes; there is no `retry: 3` | | Test names | the filename picks the step; `testName(type, name)` on the outer `describe` puts that type on every failure line under it. Never on the inner `test` too — the prefix would print twice | | Injection | `SqlRunner` and `connect` are parameters, so unit tests need no server | -| Fixtures | the preload registers `clock`, `mail`, `runJobs` — an app registers the rest | +| Fixtures | the preload registers the whole framework bag — an app registers only what the framework cannot know (`seed`, `actorFor`) | +| Built vs declared | `clock` `mail` `network` `runJobs` are built in-process; `page` `budget` `signIn` `deploy` `subscribe` are declared and wait for a driver (`X_TEST_FIXTURE_UNAVAILABLE`) | +| One seam for drivers | a driver registers over a declaration with `defineFixtures` — merges, last wins. Never a second registration mechanism | | Registry hygiene | the fixture registry is process-global; a test that clears it snapshots with `fixtureSnapshot()` and hands it back in `afterAll` | | Fixture teardown | a fixture that installs process-global state (the ambient job or mail driver) implements `Symbol.dispose` / `Symbol.asyncDispose` and restores what was there; `fixtureTest` disposes in reverse build order even when the body throws | | Building one by hand | `createRunJobs()` outside `fixtureTest` is not disposed for you — reset the driver in `afterEach`, or the next file in the process inherits your queue | diff --git a/packages/testing/README.md b/packages/testing/README.md index 7a5263325..e160cd626 100644 --- a/packages/testing/README.md +++ b/packages/testing/README.md @@ -15,7 +15,9 @@ frozen clock. Never let a test reach the network unmocked — it fails by design | `test-types.ts` | the six test types and their helpers | | `matchers.ts` | `toBeUltimateError` `toDenyPolicy` `toEmitSteps` `toMatchOpenApi` `toBeWithinBudget` `toRejectInput` | | `fixtures.ts` | the registry + `test('…', ({ clock }) => …)` injection | -| `fixture-{clock,mail,jobs}.ts` | the three fixtures the framework owns | +| `fixture-{clock,mail,jobs,network}.ts` | the four fixtures the framework builds in-process | +| `fixture-drivers.ts` | the five it declares but a driver must build — `page` `budget` `signIn` `deploy` `subscribe` | +| `framework-fixtures.ts` | registers both sets; the app registers only what it owns | | `preload.ts` | the bunfig preload that installs all of the above | ## Install @@ -42,14 +44,25 @@ test('the three-day sleep releases the worker', async ({ clock, runJobs }) => { }); ``` -| Fixture | Is | Registered by | +| Fixture | Is | Built by | |---|---|---| | `clock` | `now()` · `advance('3d')` · `set(instant)` on the frozen clock | the preload | | `mail` | `outbox()` · `lastTo(address)` · `failOnce(mail)` over an in-memory transport | the preload | +| `network` | `offline()` · `drop()` · `online()` · `state()` over the sealed network | the preload | | `runJobs` | a worker: call it to enqueue+drain, then `drain()` `due()` `inFlight()` `depth()` | the preload | +| `page` | the browser: `goto` `gotoStreamed` `getByRole` `evaluate` `waitForServiceWorker` | a browser driver | +| `budget` | `jsBytes(route)` measured off the built output | a browser driver | +| `signIn` | put the browser session in a member's shoes | a browser driver | +| `deploy` | `newBuild()` — same app, new build id, page still open | a browser driver | +| `subscribe` | one subscriber's `rows()` `patches()` `settled()` `lsn()` | a replicator | | anything else | whatever the app registers | the app's `scripts/test-setup.ts` | -`mail` and `runJobs` install a process-global driver for the length of one test and hand the previous one back afterwards. A fixture that takes over a global does the same: implement `Symbol.dispose` or `Symbol.asyncDispose` on what the factory returns, and `fixtureTest` calls it in reverse build order — including when the test body throws. +The last five are **declared but not built**: the name resolves, and destructuring one in a process +with no driver fails as `X_TEST_FIXTURE_UNAVAILABLE`, naming the driver rather than telling you to +register a fixture that is not yours to define. A driver arrives through the same registry — +`defineFixtures` merges, last registration wins — so there is no second seam to learn. + +`mail`, `network` and `runJobs` install a process-global driver for the length of one test and hand the previous one back afterwards. A fixture that takes over a global does the same: implement `Symbol.dispose` or `Symbol.asyncDispose` on what the factory returns, and `fixtureTest` calls it in reverse build order — including when the test body throws. An app adds its own with `defineFixtures` and widens the type by augmenting `Fixtures`: @@ -64,7 +77,9 @@ declare module '@ultimat3/testing' { ``` Destructuring a name nobody registered fails with `X_TEST_FIXTURE_UNKNOWN`, which names the set -that *is* registered — never `undefined is not an object` from inside the body. +that *is* registered — never `undefined is not an object` from inside the body. A name that is +registered but has no driver fails with `X_TEST_FIXTURE_UNAVAILABLE` instead; the two are different +instructions, so they are different codes. ## The six test types diff --git a/packages/testing/src/errors.ts b/packages/testing/src/errors.ts index 7fee4c5a2..f780a967a 100644 --- a/packages/testing/src/errors.ts +++ b/packages/testing/src/errors.ts @@ -4,18 +4,22 @@ import { registerErrorCodes, UltimateError } from '@ultimat3/core'; export const TESTING_ERROR_CODES = [ 'X_TEST_NETWORK_SEALED', + 'X_TEST_NETWORK_OFFLINE', 'X_TEST_DB_UNAVAILABLE', 'X_TEST_NONDETERMINISTIC', 'X_TEST_FIXTURE_UNKNOWN', + 'X_TEST_FIXTURE_UNAVAILABLE', ] as const; export type TestingErrorCode = (typeof TESTING_ERROR_CODES)[number]; export const TESTING_ERROR_TITLES: Readonly> = { X_TEST_NETWORK_SEALED: 'a test tried to reach the network', + X_TEST_NETWORK_OFFLINE: 'the test network is offline', X_TEST_DB_UNAVAILABLE: 'no Postgres for the test template', X_TEST_NONDETERMINISTIC: 'a test read wall-clock time or unseeded randomness', X_TEST_FIXTURE_UNKNOWN: 'a test requested a fixture nobody registered', + X_TEST_FIXTURE_UNAVAILABLE: 'a declared fixture has no driver in this process', }; // Titles must be registered for `format()` to render the contract's first line. Every code above is @@ -96,3 +100,40 @@ export class FixtureUnknownError extends UltimateError { export const fixtureUnknown = (name: string, registered: readonly string[]): UltimateError => new FixtureUnknownError({ name, registered }); + +/** + * Different failure from `X_TEST_FIXTURE_UNKNOWN`, and the distinction is the whole point: the + * name IS registered, so "register it" is the wrong instruction. What is missing is the driver + * underneath — a browser, a replicator — which the framework declares but deliberately does not + * bundle. Naming what it needs turns "undefined is not an object" into a decision the reader can + * make: install the driver, or stop asking for the fixture. + */ +export class FixtureUnavailableError extends UltimateError { + constructor(input: { name: string; needs: string }) { + super({ + code: 'X_TEST_FIXTURE_UNAVAILABLE', + cause: `fixture "${input.name}" is declared but nothing in this process drives it — it needs ${input.needs}`, + fix: `install one in the test preload: defineFixtures({ ${input.name}: () => yourDriver() })`, + docs: docsFor('X_TEST_FIXTURE_UNAVAILABLE'), + }); + } +} + +export const fixtureUnavailable = (name: string, needs: string): UltimateError => + new FixtureUnavailableError({ name, needs }); + +/** + * A request made while `network.offline()` (or `network.drop()`) is in force. Coded rather than a + * bare `TypeError` because a test that lands here uncaught needs to know which of the two it was: + * the app's offline path not running, or a fixture left offline by the test before it. + */ +export class NetworkOfflineError extends UltimateError { + constructor(input: { url: string; method: string; mode: 'offline' | 'dropped' }) { + super({ + code: 'X_TEST_NETWORK_OFFLINE', + cause: `${input.method} ${input.url} while the test network is ${input.mode}`, + fix: 'network.online() before the call — or assert the offline path instead of the request', + docs: docsFor('X_TEST_NETWORK_OFFLINE'), + }); + } +} diff --git a/packages/testing/src/fixture-drivers.test.ts b/packages/testing/src/fixture-drivers.test.ts new file mode 100644 index 000000000..b75a5e290 --- /dev/null +++ b/packages/testing/src/fixture-drivers.test.ts @@ -0,0 +1,71 @@ +import { afterAll, test as bunTest, describe, expect } from 'bun:test'; +import { + DRIVER_FIXTURE_NAMES, + DRIVER_FIXTURE_NEEDS, + driverFixtures, + unavailableFixture, +} from './fixture-drivers'; +import { defineFixtures, fixtureSnapshot, registeredFixtures, runWithFixtures } from './fixtures'; +import { registerFrameworkFixtures } from './framework-fixtures'; + +// The registry is process-global. This file registers a driver over `page`, so it hands the +// framework's own declaration back — otherwise every later file inherits this file's stub. +const BEFORE = fixtureSnapshot(); +afterAll(() => { + defineFixtures(BEFORE); +}); + +describe('unit · the declared-but-driverless fixtures', () => { + bunTest('every declared name has a driver requirement written down', () => { + expect([...DRIVER_FIXTURE_NAMES]).toEqual(['budget', 'deploy', 'page', 'signIn', 'subscribe']); + for (const name of DRIVER_FIXTURE_NAMES) { + expect(DRIVER_FIXTURE_NEEDS[name].length).toBeGreaterThan(10); + } + }); + + bunTest('the bag registers one factory per declared name', () => { + expect(Object.keys(driverFixtures()).sort()).toEqual([...DRIVER_FIXTURE_NAMES]); + }); + + // The distinction this whole file exists for: `page` IS registered, so "register it" — the fix + // X_TEST_FIXTURE_UNKNOWN prints — would be the wrong instruction. The failure has to say driver. + bunTest('a declared name resolves, and fails as UNAVAILABLE rather than UNKNOWN', async () => { + registerFrameworkFixtures(); + expect(registeredFixtures()).toContain('page'); + + await expect(runWithFixtures(async ({ page }) => void page)).rejects.toBeUltimateError( + 'X_TEST_FIXTURE_UNAVAILABLE', + ); + }); + + bunTest('the failure names the fixture and what it is waiting on', () => { + expect(() => unavailableFixture('subscribe')()).toThrow(/subscribe/); + expect(() => unavailableFixture('subscribe')()).toThrow(/replicator/); + }); + + // Throwing when built, not when used: a fixture that returned a proxy would surface three + // awaits later as a missing method, pointing at the assertion instead of the missing driver. + bunTest('it throws at build time, before the body runs', async () => { + let bodyRan = false; + await expect( + runWithFixtures(async ({ budget }) => { + bodyRan = true; + void budget; + }), + ).rejects.toBeUltimateError('X_TEST_FIXTURE_UNAVAILABLE'); + expect(bodyRan).toBe(false); + }); + + // How a browser driver arrives: the ordinary registry merge, no second seam to learn. + bunTest('a registered driver replaces the declaration', async () => { + registerFrameworkFixtures(); + defineFixtures({ page: () => ({ url: () => '/feed' }) }); + + let seen = ''; + await runWithFixtures(async ({ page }) => { + seen = page.url(); + }); + + expect(seen).toBe('/feed'); + }); +}); diff --git a/packages/testing/src/fixture-drivers.ts b/packages/testing/src/fixture-drivers.ts new file mode 100644 index 000000000..93283d80b --- /dev/null +++ b/packages/testing/src/fixture-drivers.ts @@ -0,0 +1,92 @@ +// The fixtures the framework DECLARES but cannot build in this process: a browser for `page`, +// `budget`, `signIn` and `deploy`; a replicator feeding a live-query registry for `subscribe`. +// +// Declared here rather than left out, because the name is the contract. An app that registered its +// own `page` would be deciding for itself what a page is, and two apps would then disagree — the +// same reason `clock` is not an app fixture. And an unregistered name fails as +// X_TEST_FIXTURE_UNKNOWN, whose fix ("register it") is the wrong instruction: what is missing is a +// driver, not a registration. So the name resolves, and asking for it without a driver says so. +// +// A driver overrides these the ordinary way — `defineFixtures` merges, last registration wins. + +import { fixtureUnavailable } from './errors'; +import type { FixtureFactory } from './fixtures'; + +/** Per-route byte budgets, measured off the built output rather than declared. */ +export interface TestBudget { + jsBytes(route: string): Promise; +} + +/** Put the browser session in this member's shoes. A row, because the app owns what a member is. */ +export type SignIn = (member: Readonly>) => Promise; + +/** Version skew: same app, new immutable build id, while the page stays open. */ +export interface TestDeploy { + newBuild(): Promise; +} + +/** + * What `query.live(input, { actor })` resolves to, named structurally so `@ultimat3/testing` does + * not take a dependency on `@ultimat3/query` for one type. The real `LiveQuery` satisfies it. + */ +export interface LiveTarget { + readonly name: string; + readonly queryHash: string; +} + +export interface LiveFeedPatch { + readonly op: 'insert' | 'update' | 'delete'; + readonly row: R; +} + +/** One subscriber's view: what it holds, what it was sent, and how it got there. */ +export interface LiveFeed { + rows(): readonly R[]; + row(id: string): R | undefined; + /** The optimistic twin a mutator applied locally, before the server confirmed it. */ + local(id: string): R | undefined; + patches(): readonly LiveFeedPatch[]; + /** Resolves when every patch in flight has been applied — never a sleep. */ + settled(): Promise; + lsn(): string; + /** Set when a reconnect resumed from a cursor; undefined when it resnapshotted. */ + resubscribedFrom(): string | undefined; + /** How many snapshots this subscriber received. A resume that refetched shows up here. */ + snapshots(): number; +} + +export type Subscribe = ( + target: LiveTarget | Promise, +) => Promise>; + +export const DRIVER_FIXTURE_NAMES = ['budget', 'deploy', 'page', 'signIn', 'subscribe'] as const; + +export type DriverFixtureName = (typeof DRIVER_FIXTURE_NAMES)[number]; + +/** What each one is waiting on. `Record` over the name union, so the two lists cannot drift. */ +export const DRIVER_FIXTURE_NEEDS: Readonly> = { + budget: 'the byte counts a browser run measures off the built output', + deploy: 'a second build to switch the running app to', + page: 'a browser driving the built app', + signIn: 'a browser session against the app’s own sign-in route', + subscribe: 'an in-process replicator feeding the live-query registry', +}; + +/** + * Throws when built, not when used. Building is where the test is still on its own first line, so + * the failure names the fixture instead of surfacing three awaits later as a missing method. + */ +export const unavailableFixture = + (name: DriverFixtureName): FixtureFactory => + () => { + throw fixtureUnavailable(name, DRIVER_FIXTURE_NEEDS[name]); + }; + +/** The declared bag, as `defineFixtures` takes it. */ +export const driverFixtures = (): Readonly> => ({ + budget: unavailableFixture('budget'), + deploy: unavailableFixture('deploy'), + page: unavailableFixture('page'), + signIn: unavailableFixture('signIn'), + subscribe: unavailableFixture('subscribe'), +}); diff --git a/packages/testing/src/fixture-network.test.ts b/packages/testing/src/fixture-network.test.ts new file mode 100644 index 000000000..971cf3566 --- /dev/null +++ b/packages/testing/src/fixture-network.test.ts @@ -0,0 +1,103 @@ +import { afterEach, test as bunTest, describe, expect } from 'bun:test'; +import { createTestNetwork } from './fixture-network'; +import { + isNetworkSealed, + mockJson, + networkState, + resetNetwork, + sealNetwork, + unsealNetwork, +} from './sealed-network'; + +// The gate is process-global and bun shares one process across files: a test that leaves the +// process offline takes every later file's fetch down with it. +afterEach(() => { + resetNetwork(); + unsealNetwork(); +}); + +const URL_UNDER_TEST = 'https://api.stripe.test/v1/charges'; + +describe('unit · the network fixture', () => { + bunTest('offline fails the request the app’s offline path is written for', async () => { + sealNetwork(); + const network = createTestNetwork(); + + network.offline(); + + await expect(fetch(URL_UNDER_TEST)).rejects.toBeUltimateError('X_TEST_NETWORK_OFFLINE'); + expect(network.state()).toBe('offline'); + }); + + bunTest('online puts it back, and a mock answers again', async () => { + sealNetwork(); + mockJson(URL_UNDER_TEST, { ok: true }); + const network = createTestNetwork(); + + network.offline(); + await expect(fetch(URL_UNDER_TEST)).rejects.toBeUltimateError('X_TEST_NETWORK_OFFLINE'); + network.online(); + + expect(await (await fetch(URL_UNDER_TEST)).json()).toEqual({ ok: true }); + }); + + // The rule the offline gate sits ahead of the mocks for: a mock that still answered would be + // the one thing the offline path never sees, and the test would pass without exercising it. + bunTest('a mocked route is offline too — offline beats the mock', async () => { + sealNetwork(); + mockJson(URL_UNDER_TEST, { ok: true }); + const network = createTestNetwork(); + + network.offline(); + + await expect(fetch(URL_UNDER_TEST)).rejects.toBeUltimateError('X_TEST_NETWORK_OFFLINE'); + }); + + bunTest('drop is offline that names itself, so a resume is not a resubscribe', async () => { + sealNetwork(); + const network = createTestNetwork(); + + network.drop(); + + expect(network.state()).toBe('dropped'); + await expect(fetch(URL_UNDER_TEST)).rejects.toBeUltimateError('X_TEST_NETWORK_OFFLINE'); + }); + + // ULTIMATE_TEST_ALLOW_NET=1 unseals the process on purpose. `offline()` still has to bite there, + // or a deliberate-integration file silently tests nothing when it goes offline. + bunTest('offline works in an unsealed process, and hands the process back unsealed', () => { + unsealNetwork(); + const network = createTestNetwork(); + + network.offline(); + expect(isNetworkSealed()).toBe(true); + + network[Symbol.dispose](); + + expect(isNetworkSealed()).toBe(false); + expect(networkState()).toBe('online'); + }); + + bunTest('a sealed process stays sealed after disposal', () => { + sealNetwork(); + const network = createTestNetwork(); + + network.offline(); + network[Symbol.dispose](); + + expect(isNetworkSealed()).toBe(true); + expect(networkState()).toBe('online'); + }); + + // The leak this fixture would otherwise cause: bun shares one process across files, so an + // offline left behind fails every later file at its first fetch, somewhere else entirely. + bunTest('disposal puts the process back online even after drop', () => { + sealNetwork(); + const network = createTestNetwork(); + + network.drop(); + network[Symbol.dispose](); + + expect(networkState()).toBe('online'); + }); +}); diff --git a/packages/testing/src/fixture-network.ts b/packages/testing/src/fixture-network.ts new file mode 100644 index 000000000..3535873d0 --- /dev/null +++ b/packages/testing/src/fixture-network.ts @@ -0,0 +1,58 @@ +// The `network` fixture: pull the cable, put it back. What an offline test needs is not a mock +// that answers differently — it is a request that fails the way a real one fails, so the app's own +// offline path runs instead of a branch written for the test. +// +// `drop()` exists next to `offline()` because the two are different bugs. A clean offline is what a +// service worker answers; a dropped connection is what a live subscription must resume from. A +// fixture with only one of them cannot tell a resubscribe apart from a resume. + +import { + isNetworkSealed, + type NetworkState, + networkState, + sealNetwork, + setNetworkState, + unsealNetwork, +} from './sealed-network'; + +/** `Disposable`: the gate is process-global, so the fixture puts the process back online after. */ +export interface TestNetwork extends Disposable { + /** Every request fails as it would with no route to the host. */ + offline(): void; + /** Offline, and the connection was cut rather than closed — a subscriber must reconnect. */ + drop(): void; + online(): void; + state(): NetworkState; +} + +/** + * Synchronous, unlike `mail` and `runJobs`: the gate it drives lives in this package, so there is + * no subsystem to import on demand. The test bodies rely on it — `network.offline()` is followed + * on the next line by the mutation that has to observe it. + */ +export function createTestNetwork(): TestNetwork { + // A process that unsealed on purpose (ULTIMATE_TEST_ALLOW_NET=1) still gets a working + // `offline()`, and gets its unsealed fetch back on disposal rather than keeping ours. + const sealedBefore = isNetworkSealed(); + let sealedByUs = false; + + const goto = (next: NetworkState): void => { + if (next !== 'online' && !isNetworkSealed()) { + sealNetwork(); + sealedByUs = true; + } + setNetworkState(next); + }; + + return { + offline: () => goto('offline'), + drop: () => goto('dropped'), + online: () => goto('online'), + state: networkState, + [Symbol.dispose]: (): void => { + setNetworkState('online'); + if (sealedByUs && !sealedBefore) unsealNetwork(); + sealedByUs = false; + }, + }; +} diff --git a/packages/testing/src/fixtures.ts b/packages/testing/src/fixtures.ts index 8a32b4027..0dac9588e 100644 --- a/packages/testing/src/fixtures.ts +++ b/packages/testing/src/fixtures.ts @@ -11,8 +11,11 @@ import { test as bunTest } from 'bun:test'; import { fixtureUnknown } from './errors'; import type { TestClock } from './fixture-clock'; +import type { SignIn, Subscribe, TestBudget, TestDeploy } from './fixture-drivers'; import type { RunJobs } from './fixture-jobs'; import type { TestMail } from './fixture-mail'; +import type { TestNetwork } from './fixture-network'; +import type { PageLike } from './test-types'; /** Built once per test, on first use. */ export type FixtureFactory = () => T | Promise; @@ -20,8 +23,8 @@ export type FixtureFactory = () => T | Promise; export type FixtureMap = Readonly>; /** - * What a test body receives. The three the framework owns are declared here and registered by - * the preload; apps widen it by augmenting `Fixtures`: + * What a test body receives. Everything the framework owns is declared here and registered by the + * preload; apps widen it by augmenting `Fixtures`: * * ```ts * declare module '@ultimat3/testing' { @@ -30,11 +33,21 @@ export type FixtureMap = Readonly>; * } * } * ``` + * + * The last five are declared but driver-backed: destructuring one in a process with no driver + * fails as `X_TEST_FIXTURE_UNAVAILABLE`, naming what is missing. Typed here anyway, because the + * type is the contract a driver implements — see `fixture-drivers.ts`. */ export interface Fixtures { readonly clock: TestClock; readonly mail: TestMail; + readonly network: TestNetwork; readonly runJobs: RunJobs; + readonly budget: TestBudget; + readonly deploy: TestDeploy; + readonly page: PageLike; + readonly signIn: SignIn; + readonly subscribe: Subscribe; } const registry = new Map(); diff --git a/packages/testing/src/framework-fixtures.test.ts b/packages/testing/src/framework-fixtures.test.ts index f6465dee3..d3c45bd9b 100644 --- a/packages/testing/src/framework-fixtures.test.ts +++ b/packages/testing/src/framework-fixtures.test.ts @@ -6,8 +6,12 @@ import { mailDriver, resetMailDriver, tryMailDriver } from '@ultimat3/mail'; import { frozenNow, setFrozenClock } from './determinism'; import { createRunJobs } from './fixture-jobs'; import { createTestMail } from './fixture-mail'; -import { fixtureTest } from './fixtures'; -import { FRAMEWORK_FIXTURE_NAMES, registerFrameworkFixtures } from './framework-fixtures'; +import { fixtureTest, registeredFixtures } from './fixtures'; +import { + ALL_FIXTURE_NAMES, + FRAMEWORK_FIXTURE_NAMES, + registerFrameworkFixtures, +} from './framework-fixtures'; import { testName } from './test-types'; // Every global these fixtures touch is process-wide and bun shares one process across files. @@ -42,9 +46,16 @@ const message = (mailId: string) => ({ }); describe(testName('unit', 'the framework fixture bag'), () => { - bunTest('owns exactly clock, mail and runJobs', () => { + bunTest('builds exactly clock, mail, network and runJobs in-process', () => { registerFrameworkFixtures(); - expect([...FRAMEWORK_FIXTURE_NAMES]).toEqual(['clock', 'mail', 'runJobs']); + expect([...FRAMEWORK_FIXTURE_NAMES]).toEqual(['clock', 'mail', 'network', 'runJobs']); + }); + + // The bag is the contract: a name the reference app destructures and the framework never + // registers fails as X_TEST_FIXTURE_UNKNOWN, whose fix tells the app to invent its own `page`. + bunTest('registers every name it declares, driver-backed ones included', () => { + registerFrameworkFixtures(); + expect(registeredFixtures()).toEqual(expect.arrayContaining([...ALL_FIXTURE_NAMES])); }); // The regression the registration exists for: before it, every body destructuring `clock` diff --git a/packages/testing/src/framework-fixtures.ts b/packages/testing/src/framework-fixtures.ts index 3943b3926..937498105 100644 --- a/packages/testing/src/framework-fixtures.ts +++ b/packages/testing/src/framework-fixtures.ts @@ -1,22 +1,40 @@ -// Registers the fixtures the FRAMEWORK owns — `clock`, `mail`, `runJobs` — so an app registers -// only what it owns (`seed`, `actorFor`, …). Called by the preload, which is why an app never -// writes `defineFixtures({ clock })` and why two apps cannot disagree about what `clock` means. +// Registers the fixtures the FRAMEWORK owns, so an app registers only what it owns (`seed`, +// `actorFor`, …). Called by the preload, which is why an app never writes `defineFixtures({ clock })` +// and why two apps cannot disagree about what `clock` means. // -// Each factory imports its subsystem on demand, for the reason the whole registry is lazy: a -// test that never touches jobs must not pay for the queue, and a `packages/core` test must not -// boot mail. `defineFixtures` merges, so registering twice is idempotent. +// Two lists, because the bag has two kinds of member. The first four are built here and always +// work. The rest are declared here and built by a driver the process installs — see +// `fixture-drivers.ts` for why a declared-and-unavailable fixture beats an unregistered name. +// +// Each factory imports its subsystem on demand, for the reason the whole registry is lazy: a test +// that never touches jobs must not pay for the queue, and a `packages/core` test must not boot +// mail. `defineFixtures` merges, so registering twice is idempotent — and so a driver registered +// afterwards replaces the declaration it was waiting on. import { createTestClock } from './fixture-clock'; +import { DRIVER_FIXTURE_NAMES, driverFixtures } from './fixture-drivers'; import { createRunJobs } from './fixture-jobs'; import { createTestMail } from './fixture-mail'; +import { createTestNetwork } from './fixture-network'; import { defineFixtures } from './fixtures'; -export const FRAMEWORK_FIXTURE_NAMES = ['clock', 'mail', 'runJobs'] as const; +/** Built in-process. Always available, in every test type. */ +export const FRAMEWORK_FIXTURE_NAMES = ['clock', 'mail', 'network', 'runJobs'] as const; + +export { DRIVER_FIXTURE_NAMES }; + +/** Every name the framework puts in the bag, in the order `registeredFixtures()` reports them. */ +export const ALL_FIXTURE_NAMES: readonly string[] = [ + ...FRAMEWORK_FIXTURE_NAMES, + ...DRIVER_FIXTURE_NAMES, +].sort(); export function registerFrameworkFixtures(): void { defineFixtures({ + ...driverFixtures(), clock: createTestClock, mail: createTestMail, + network: createTestNetwork, runJobs: createRunJobs, }); } diff --git a/packages/testing/src/index.ts b/packages/testing/src/index.ts index c15b139eb..11f208297 100644 --- a/packages/testing/src/index.ts +++ b/packages/testing/src/index.ts @@ -34,6 +34,8 @@ export { } from './determinism'; export type { TestingErrorCode } from './errors'; export { + FixtureUnavailableError, + NetworkOfflineError, NetworkSealedError, NondeterministicError, TESTING_ERROR_CODES, @@ -44,24 +46,45 @@ export type { EntityLike, EntityRegistry, Factory, FactoryOptions } from './fact export { defineFactory, factoriesFor } from './factories'; export type { TestClock, TestDuration } from './fixture-clock'; export { createTestClock } from './fixture-clock'; +export type { + DriverFixtureName, + LiveFeed, + LiveFeedPatch, + LiveTarget, + SignIn, + Subscribe, + TestBudget, + TestDeploy, +} from './fixture-drivers'; +export { DRIVER_FIXTURE_NEEDS, driverFixtures, unavailableFixture } from './fixture-drivers'; export type { JobRunTrace, RunJobs, StepTally } from './fixture-jobs'; export { createRunJobs } from './fixture-jobs'; export type { MailRef, TestMail } from './fixture-mail'; export { createTestMail } from './fixture-mail'; +export type { TestNetwork } from './fixture-network'; +export { createTestNetwork } from './fixture-network'; export { fixtureTest as test } from './fixtures'; -export { FRAMEWORK_FIXTURE_NAMES, registerFrameworkFixtures } from './framework-fixtures'; +export { + ALL_FIXTURE_NAMES, + DRIVER_FIXTURE_NAMES, + FRAMEWORK_FIXTURE_NAMES, + registerFrameworkFixtures, +} from './framework-fixtures'; export type { AppHandle, AppOptions, BootedApp } from './harness'; export { describeApp, testApp } from './harness'; export type { MatcherResult } from './matchers'; export { matchersInstalled, recordSteps } from './matchers'; -export type { MockRoute } from './sealed-network'; +export type { MockRoute, NetworkState } from './sealed-network'; export { allowHost, + isNetworkSealed, mockFetch, mockJson, + networkState, requestedUrls, resetNetwork, sealNetwork, + setNetworkState, unsealNetwork, } from './sealed-network'; export type { SqlRunner, TemplateDbConfig, WorkerDatabase } from './template-db'; @@ -82,6 +105,7 @@ export type { E2eFixtures, EvalCase, EvalOptions, + LocatorLike, OpenApiLike, PageLike, TestType, diff --git a/packages/testing/src/sealed-network.ts b/packages/testing/src/sealed-network.ts index 554009180..54d3eec41 100644 --- a/packages/testing/src/sealed-network.ts +++ b/packages/testing/src/sealed-network.ts @@ -3,10 +3,16 @@ // for reasons nobody can reproduce — so the default is "nothing gets out". import { isSelfOrigin } from '@ultimat3/core'; -import { NetworkSealedError } from './errors'; +import { NetworkOfflineError, NetworkSealedError } from './errors'; export type FetchLike = typeof globalThis.fetch; +/** + * `offline` is the cable pulled; `dropped` is the same for a request, but tells a transport its + * connection was cut rather than closed — so it reconnects and resumes instead of resubscribing. + */ +export type NetworkState = 'online' | 'offline' | 'dropped'; + export interface MockRoute { /** Exact URL, or a prefix ending in `*`, or a RegExp. */ readonly match: string | RegExp; @@ -18,9 +24,16 @@ interface SealState { readonly mocks: MockRoute[]; readonly seen: string[]; original: FetchLike | undefined; + network: NetworkState; } -const state: SealState = { allowed: new Set(), mocks: [], seen: [], original: undefined }; +const state: SealState = { + allowed: new Set(), + mocks: [], + seen: [], + original: undefined, + network: 'online', +}; const matches = (route: MockRoute, url: string): boolean => { if (route.match instanceof RegExp) return route.match.test(url); @@ -47,6 +60,15 @@ export function sealNetwork(): void { globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit): Promise => { const url = urlOf(input); state.seen.push(url); + // Ahead of the mocks on purpose: a test that mocked Stripe and then went offline is asserting + // the offline path, and a mock that still answered would be the one thing that path never sees. + if (state.network !== 'online') { + throw new NetworkOfflineError({ + url, + method: methodOf(input, init), + mode: state.network, + }); + } const mock = state.mocks.find((route) => matches(route, url)); if (mock !== undefined) { return mock.handler(input instanceof Request ? input : new Request(url, init)); @@ -74,6 +96,17 @@ export function unsealNetwork(): void { state.original = undefined; } +/** Whether the patch is installed. The `network` fixture seals before going offline, so that + * `offline()` has teeth in a process that deliberately unsealed (`ULTIMATE_TEST_ALLOW_NET=1`). */ +export const isNetworkSealed = (): boolean => state.original !== undefined; + +/** The one writer of the offline gate — `createTestNetwork()`. Never call it from a test body. */ +export function setNetworkState(next: NetworkState): void { + state.network = next; +} + +export const networkState = (): NetworkState => state.network; + function safeHost(url: string): string | undefined { try { return new URL(url).host; @@ -113,4 +146,5 @@ export function resetNetwork(): void { state.allowed.clear(); state.mocks.length = 0; state.seen.length = 0; + state.network = 'online'; } diff --git a/packages/testing/src/test-types.ts b/packages/testing/src/test-types.ts index 639377eb6..97c0f9c3c 100644 --- a/packages/testing/src/test-types.ts +++ b/packages/testing/src/test-types.ts @@ -34,11 +34,36 @@ export const jobTest = (name: string, body: TestBody): void => { test(testName('job', name), body); }; +/** One element selection, resolved when it is used rather than when it is built. */ +export interface LocatorLike { + count(): Promise; + click(): Promise; + first(): LocatorLike; + isVisible(): Promise; +} + +/** + * The browser surface an e2e test drives. Every member is one the reference app's e2e suite + * already calls — this is the observed contract, not a wish list, and the driver that implements + * it (a browser, at milestone 11) is the only thing that may add to it. + */ export interface PageLike { goto(url: string): Promise; + /** The first flush of a streamed response — the shell, before the holes resolve. */ + gotoStreamed(url: string): Promise<{ readonly html: string }>; reload(): Promise; + /** Resolves once the service worker controls the page, so the offline assertions are not racy. */ + waitForServiceWorker(): Promise; title(): Promise; content(): Promise; + url(): string; + evaluate(fn: () => T): Promise; + locator(selector: string): LocatorLike; + getByRole( + role: string, + options?: { readonly name?: string; readonly level?: number }, + ): LocatorLike; + getByText(text: string): LocatorLike; } export interface E2eFixtures { diff --git a/wiki/Error-Codes.md b/wiki/Error-Codes.md index 1feac0cc8..109043148 100644 --- a/wiki/Error-Codes.md +++ b/wiki/Error-Codes.md @@ -319,7 +319,9 @@ One pipeline in `@ultimat3/core` serves `storage`, `seo` and `pwa`. It **decodes | Code | Means | Typical cause | Fix | |---|---|---|---| | `X_TEST_DB_UNAVAILABLE` | no Postgres for the test template | nothing listening | run `x dev` (embedded Postgres), or set `TEST_DATABASE_URL` | +| `X_TEST_FIXTURE_UNAVAILABLE` | a declared fixture has no driver in this process | destructuring `page`, `budget`, `signIn`, `deploy` or `subscribe` with no browser or replicator installed | `defineFixtures({ : () => yourDriver() })` in the test preload — `cause` names what the fixture needs | | `X_TEST_FIXTURE_UNKNOWN` | a test requested a fixture nobody registered | a destructured fixture name no `defineFixtures` call declares | `defineFixtures({ : () => buildIt() })` at test setup — `cause` lists the registered names | +| `X_TEST_NETWORK_OFFLINE` | the test network is offline | a request made after `network.offline()` or `network.drop()` | `network.online()` before the call — or assert the offline path instead of the request | | `X_TEST_NETWORK_SEALED` | a test tried to reach the network | an unmocked external call | `mockFetch('', …)`, or `allowHost('')` if it must be real | | `X_TEST_NONDETERMINISTIC` | a test read wall-clock time or unseeded randomness | `Date.now()` in the code under test | wrap in `frozenClock()` / `seededRandom()`, or remove the read | diff --git a/wiki/Testing.md b/wiki/Testing.md index 0d3dd98d9..5d9dfe416 100644 --- a/wiki/Testing.md +++ b/wiki/Testing.md @@ -116,6 +116,34 @@ test('onboardOrg retries only the failed step', async ({ seed, clock, mail }) => The `job` example is the one that matters: it asserts the durability guarantee, not that mail was sent. See [Jobs and workflows](Jobs-And-Workflows) and [Policies and authz](Policies-And-Authz). +## The fixture bag + +`test` passes a bag as the first argument and builds only what the body destructures — a test that never names `runJobs` never starts a queue. The framework owns the whole bag; an app registers only what the framework cannot know. + +| Fixture | Is | Built by | +|---|---|---| +| `clock` | `now()` · `advance('3d')` · `set(instant)` | the preload | +| `mail` | `outbox()` · `lastTo(address)` · `failOnce(mail)` | the preload | +| `network` | `offline()` · `drop()` · `online()` · `state()` | the preload | +| `runJobs` | enqueue+drain, then `drain()` `due()` `inFlight()` `depth()` | the preload | +| `page` | `goto` · `gotoStreamed` · `getByRole` · `evaluate` · `waitForServiceWorker` | a browser driver | +| `budget` | `jsBytes(route)` off the built output | a browser driver | +| `signIn` | put the browser session in a member's shoes | a browser driver | +| `deploy` | `newBuild()` — same app, new build id, page still open | a browser driver | +| `subscribe` | one subscriber's `rows()` `patches()` `settled()` `lsn()` | a replicator | +| `seed`, and anything else | the app's own graph | the app's `scripts/test-setup.ts` | + +The last five are **declared, not built**. The framework does not bundle a browser, so the name resolves and asking for it without a driver fails as `X_TEST_FIXTURE_UNAVAILABLE` naming what is missing — different from `X_TEST_FIXTURE_UNKNOWN`, whose fix ("register it") would have the app inventing its own idea of what a page is. A driver installs through the same registry: + +```ts +// scripts/test-setup.ts +defineFixtures({ page: () => openBrowserPage(), seed: () => loadSeed }); +``` + +`defineFixtures` merges and the last registration wins, so a driver replaces the declaration it was waiting on. There is no second seam. + +`network.offline()` fails every request ahead of the mocks, so the app's own offline path runs instead of a branch written for the test; `drop()` is the same for a request but tells a subscriber its connection was cut rather than closed, which is what separates a resume from a resubscribe. + ## Generated scaffolds Every primitive emits a test scaffold that fails until filled in — an untested action is a red build, not a backlog item. From 5bc42ed9a2152dddfc0318c9aeba504f1a8b0b6c Mon Sep 17 00:00:00 2001 From: sebi Date: Mon, 10 Aug 2026 08:55:18 -0500 Subject: [PATCH 2/4] fix(verify): put site/ under the gate's typecheck scope MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - new site/tsconfig.json, referenced from the root — `tsc -b` never walked site/, so build.ts and lib/* shipped untypechecked - empty `paths` there: site/ is its own bundle graph (axiom 6), so an @ultimat3/* import is now a resolution error, not a convention - declare the frontmatter vocabulary as `PageMeta`, clearing the 24 TS4111s a bare Record produced; `meta.nav` is checked, `meta.navv` is not a property - typecheck step summary said "every workspace"; site/ is not one - parse.test.ts fixture gave `verify` an `--only` flag it does not and must not have — moved the string-flag case onto `db --name` Co-Authored-By: Claude --- packages/cli/src/cmd-verify.ts | 2 +- packages/cli/src/parse.test.ts | 26 +++++++++++++------------- scripts/verify.test.ts | 33 +++++++++++++++++++++++++++++++++ site/README.md | 6 ++++++ site/lib/config.ts | 26 +++++++++++++++++++++++++- site/tsconfig.json | 12 ++++++++++++ tsconfig.json | 3 +++ 7 files changed, 93 insertions(+), 15 deletions(-) create mode 100644 site/tsconfig.json diff --git a/packages/cli/src/cmd-verify.ts b/packages/cli/src/cmd-verify.ts index 513c2dceb..7e6c5be83 100644 --- a/packages/cli/src/cmd-verify.ts +++ b/packages/cli/src/cmd-verify.ts @@ -27,7 +27,7 @@ import { checkFileSizes, checkPackageShape, hasWorkspacePackages } from './works export const VERIFY_STEPS: readonly VerifyStep[] = [ { name: 'typecheck', - summary: 'tsc across every workspace', + summary: 'tsc -b across every project the root references', async run(ctx) { const result = await ctx.runner(['bunx', 'tsc', '-b', '--pretty', 'false'], { cwd: ctx.root, diff --git a/packages/cli/src/parse.test.ts b/packages/cli/src/parse.test.ts index 73eee2da6..58fc241d9 100644 --- a/packages/cli/src/parse.test.ts +++ b/packages/cli/src/parse.test.ts @@ -5,17 +5,15 @@ import { flagBool, flagString, parseArgs } from './parse'; import { thrownBy } from './thrown-by'; const SPECS: readonly CommandSpec[] = [ - { - name: 'verify', - summary: 'the gate', - usage: 'x verify', - flags: [{ name: 'only', type: 'string', summary: 'steps' }], - }, + // `verify` really does declare no flags — narrowing the gate would make "green" mean whatever + // the caller chose (axiom 5), so the fixture carries no `--only`/`--skip` either. + { name: 'verify', summary: 'the gate', usage: 'x verify', flags: [] }, { name: 'db', summary: 'database', usage: 'x db ', subcommands: ['gen', 'migrate', 'branch'], + flags: [{ name: 'name', type: 'string', summary: 'migration or branch name' }], }, { name: 'g', aliases: ['generate'], summary: 'scaffold', usage: 'x g ' }, { name: 'help', summary: 'help', usage: 'x help' }, @@ -43,9 +41,11 @@ describe('unit · parseArgs', () => { }); test('reads string flags in both --flag value and --flag=value form', () => { - expect(flagString(parseArgs(['verify', '--only', 'lint'], SPECS), 'only')).toBe('lint'); - expect(flagString(parseArgs(['verify', '--only=lint,drift'], SPECS), 'only')).toBe( - 'lint,drift', + expect(flagString(parseArgs(['db', 'branch', '--name', 'feat-billing'], SPECS), 'name')).toBe( + 'feat-billing', + ); + expect(flagString(parseArgs(['db', 'branch', '--name=feat-billing'], SPECS), 'name')).toBe( + 'feat-billing', ); }); @@ -59,9 +59,9 @@ describe('unit · parseArgs', () => { }); test('everything after -- is passthrough, not a flag', () => { - const args = parseArgs(['verify', '--', '--only', 'nonsense'], SPECS); - expect(args.passthrough).toEqual(['--only', 'nonsense']); - expect(flagString(args, 'only')).toBeUndefined(); + const args = parseArgs(['db', 'branch', '--', '--name', 'nonsense'], SPECS); + expect(args.passthrough).toEqual(['--name', 'nonsense']); + expect(flagString(args, 'name')).toBeUndefined(); }); test('an unknown command throws X_CLI_UNKNOWN_COMMAND with a suggestion', () => { @@ -84,7 +84,7 @@ describe('unit · parseArgs', () => { }); test('a string flag with no value is an error, not a silent empty string', () => { - expect(() => parseArgs(['verify', '--only'], SPECS)).toThrow(); + expect(() => parseArgs(['db', 'branch', '--name'], SPECS)).toThrow(); }); test('bare argv and --help both route to the help command', () => { diff --git a/scripts/verify.test.ts b/scripts/verify.test.ts index f5984823c..7bafbf256 100644 --- a/scripts/verify.test.ts +++ b/scripts/verify.test.ts @@ -15,6 +15,24 @@ import { tierBoundaries, } from './verify'; +const readJson = async (path: string): Promise => Bun.file(path).json(); + +const isRecord = (value: unknown): value is Record => + typeof value === 'object' && value !== null && !Array.isArray(value); + +/** One key off a parsed tsconfig, checked rather than cast — the file is data, not a type. */ +const field = (value: unknown, key: string): unknown => (isRecord(value) ? value[key] : undefined); + +/** The `path` of every project a tsconfig references, in declaration order. */ +async function tsconfigReferences(path: string): Promise { + const references = field(await readJson(path), 'references'); + if (!Array.isArray(references)) return []; + return references.flatMap((reference: unknown) => { + const target = field(reference, 'path'); + return typeof target === 'string' ? [target] : []; + }); +} + describe('unit · the repo gate is the CLI gate', () => { test('the repo adds rules to steps, never steps of its own', () => { const names: readonly string[] = verifyStepNames(); @@ -55,6 +73,21 @@ describe('unit · the repo gate is the CLI gate', () => { } }); + // The `typecheck` step is `tsc -b` at the repo root, so a directory the root project does not + // reference is a directory the gate never reads. `site/` shipped for a year outside it. + test('the typecheck step reaches site/, not only packages/', async () => { + const references = await tsconfigReferences(join(repoRoot(), 'tsconfig.json')); + expect(references).toContain('./site'); + }); + + test('site/ is its own bundle graph — no @ultimat3/* path resolves inside it', async () => { + const config = await readJson(join(repoRoot(), 'site/tsconfig.json')); + // Axiom 6: the static path never pays for the app path. Emptying the inherited `paths` makes + // that a build error rather than a comment — `import '@ultimat3/core'` here simply cannot + // resolve. + expect(field(field(config, 'compilerOptions'), 'paths')).toEqual({}); + }); + test('this repo has no tier violations and its manifest still generates', async () => { const root = repoRoot(); expect(await tierBoundaries(root)).toEqual([]); diff --git a/site/README.md b/site/README.md index 218feab4c..8c5dcc68a 100644 --- a/site/README.md +++ b/site/README.md @@ -61,9 +61,15 @@ scripts/theme.js # the only JS: before-paint theme apply + delegated to pages/*.md # content, with frontmatter assets/logo.svg # mark + favicon, theme-aware via its own internal stylesheet assets/og.svg # 1200x630 social card +tsconfig.json # this directory is a TS project, referenced by the root tsconfig CNAME # ultimate.developerz.ai ``` +`site/` is a project the root `tsconfig.json` references, so `bun run verify`'s `typecheck` step +covers it: a type error here is a red gate, exactly as it is in `packages/`. Its `paths` are empty +on purpose — the site is its own bundle graph (axiom 6), so no `@ultimat3/*` import can resolve +inside it. The build itself stays dependency-free and runs straight off the TypeScript sources. + ## Authoring Frontmatter, all required unless noted: diff --git a/site/lib/config.ts b/site/lib/config.ts index 344813642..b20e2e36e 100644 --- a/site/lib/config.ts +++ b/site/lib/config.ts @@ -69,10 +69,34 @@ export const PAGE_ORDER = [ 'changelog', ] as const; +/** + * The frontmatter vocabulary every page draws from. Declared instead of left to a bare + * `Record` so a key the build reads is a name the compiler knows: `meta.nav` is + * checked, `meta.navv` is not a property. A key outside the list still parses — it just has to be + * read by bracket, which is the compiler saying it is not part of the contract. + */ +export interface PageMeta { + /** `

`, `` and the JSON-LD headline. `lib/seo.ts` fails the build without it. */ + readonly title?: string; + /** `<meta name="description">` and the JSON-LD description. 50–160 characters, enforced. */ + readonly description?: string; + /** Home only: the hero heading, when it should differ from `title`. */ + readonly headline?: string; + /** The standfirst under the heading, when it should differ from `description`. */ + readonly lede?: string; + /** The short label the breadcrumb, the header and the pager use. */ + readonly nav?: string; + /** `'true'` opts the page into the header menu; every page is reachable regardless. */ + readonly menu?: string; + /** `YYYY-MM-DD`, published as `sitemap.xml`'s `lastmod` and JSON-LD's `dateModified`. */ + readonly updated?: string; + readonly [key: string]: string | undefined; +} + export interface Page { readonly slug: string; readonly url: string; readonly file: string; - readonly meta: Record<string, string>; + readonly meta: PageMeta; readonly body: string; } diff --git a/site/tsconfig.json b/site/tsconfig.json new file mode 100644 index 000000000..c3a616532 --- /dev/null +++ b/site/tsconfig.json @@ -0,0 +1,12 @@ +{ + "extends": "../tsconfig.base.json", + "compilerOptions": { + "rootDir": ".", + "noEmit": true, + "types": ["bun"], + "lib": ["ES2023"], + "paths": {} + }, + "include": ["**/*.ts"], + "exclude": ["dist"] +} diff --git a/tsconfig.json b/tsconfig.json index 41239c333..273263875 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -84,6 +84,9 @@ }, { "path": "./packages/ui" + }, + { + "path": "./site" } ] } From 906d9df2c42c7513a5fcab3f2cdf5feb93ccda69 Mon Sep 17 00:00:00 2001 From: sebi <gore.sebyx@yahoo.com> Date: Mon, 10 Aug 2026 09:28:40 -0500 Subject: [PATCH 3/4] =?UTF-8?q?fix:=20address=20PR=20#25=20review=20?= =?UTF-8?q?=E2=80=94=20fixture=20typing,=20network=20restore?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - defineFixtures holds every declared name to its declared type, so a half-built `page` driver is a compile error at the registration rather than a missing method three awaits into a later test - the network fixture restores the state it found on disposal instead of forcing 'online' — an outer fixture already offline stayed offline - setNetworkState is no longer exported: the gate's one writer is the `network` fixture, and a direct call skipped its disposal - parse.test.ts asserts X_CLI_BAD_FLAG + "expects a value", not any throw - verify.test.ts compiles a probe under site/ and asserts @ultimat3/core cannot resolve — the effect, not just the `paths` knob - file headers cut to the 1-4 line convention; the rationale moved onto the declarations it explains, not deleted - testName('unit', …) on both new outer describes - site/README.md dates its verification claims As of 2026-08 Co-Authored-By: Claude <noreply@anthropic.com> --- examples/dummy/scripts/test-setup.ts | 32 ++++++++------- packages/cli/src/parse.test.ts | 4 +- packages/testing/CLAUDE.md | 2 + packages/testing/README.md | 6 ++- packages/testing/src/fixture-drivers.test.ts | 42 +++++++++++++++++++- packages/testing/src/fixture-drivers.ts | 29 ++++++++------ packages/testing/src/fixture-network.test.ts | 20 +++++++++- packages/testing/src/fixture-network.ts | 23 ++++++----- packages/testing/src/fixtures.ts | 19 ++++++++- packages/testing/src/framework-fixtures.ts | 26 ++++++------ packages/testing/src/index.ts | 13 +++++- scripts/verify.test.ts | 41 ++++++++++++++++++- site/README.md | 9 +++-- 13 files changed, 204 insertions(+), 62 deletions(-) diff --git a/examples/dummy/scripts/test-setup.ts b/examples/dummy/scripts/test-setup.ts index 3768a3b2d..d12fa7536 100644 --- a/examples/dummy/scripts/test-setup.ts +++ b/examples/dummy/scripts/test-setup.ts @@ -1,8 +1,8 @@ -// Postly's test preload: the fixtures the APP owns, which is two of them. Everything else in the -// bag — `clock`, `mail`, `network`, `runJobs`, and the driver-backed `page`, `budget`, `signIn`, -// `deploy`, `subscribe` — arrives with the framework's own preload, imported below. An app -// registers only what the framework cannot know, which here is the seed graph and how a member -// becomes an actor. Registering `page` here would be Postly deciding for itself what a page is. +// Postly's test preload: the fixtures the APP owns, which is two of them — the seed graph, and +// how a member becomes an actor. Everything else in the bag arrives with the framework's own +// preload, imported below, so an app registers only what the framework cannot know. + +// The suite's one preload, named in `bunfig.toml`: // // [test] // preload = ["./scripts/test-setup.ts"] @@ -12,16 +12,14 @@ // depend on workspace symlinks. A generated app writes `@ultimat3/testing` here. For the same // reason `scripts/` is not in tsconfig's `include` — a composite project cannot reach across // into another one's sources. Both go away when the app joins the workspace. -// -// Booting `apps/web/api` is the other job, and it belongs HERE rather than in a test file. It is -// the registration pass: it stamps each export name onto its declaration, which is what gives a -// projection a stable name to project under. A test in `app/` that imported it would be `app/` -// reaching into `api/` at runtime — the boundary `x verify` rejects with `X_BOUNDARY_VIOLATION`, -// because it is the edge along which a page could call a handler instead of the typed client. -// The preload is outside both, runs once for the whole suite, and is already where the app says -// what its tests need. Same relative-path convention, same reason. - import '../../../packages/testing/src/preload'; +// The registration pass, and it belongs HERE rather than in a test file: importing the API stamps +// each export name onto its declaration, which is what gives a projection a stable name to project +// under. A test in `app/` that imported it would be `app/` reaching into `api/` at runtime — the +// boundary `x verify` rejects with `X_BOUNDARY_VIOLATION`, because it is the edge along which a +// page could call a handler instead of the typed client. The preload is outside both, runs once +// for the whole suite, and is already where the app says what its tests need. Same relative-path +// convention, same reason. import '../apps/web/api'; import { assert, userActor } from '../../../packages/core/src/index'; import type { Driver, EntityCore, Repo, Seed } from '../../../packages/entity/src/index'; @@ -115,6 +113,12 @@ const actorFor = (member: SeedRow) => roles: [String(member['role'])], }); +/** + * Two names, and deliberately no more. `clock`, `mail`, `network`, `runJobs` and the driver-backed + * `page`, `budget`, `signIn`, `deploy` and `subscribe` all arrive with the framework's preload: + * registering `page` here would be Postly deciding for itself what a page is, and two apps would + * then disagree about it. + */ defineFixtures({ seed: createSeed, actorFor: () => actorFor, diff --git a/packages/cli/src/parse.test.ts b/packages/cli/src/parse.test.ts index 58fc241d9..3fbb2a25c 100644 --- a/packages/cli/src/parse.test.ts +++ b/packages/cli/src/parse.test.ts @@ -84,7 +84,9 @@ describe('unit · parseArgs', () => { }); test('a string flag with no value is an error, not a silent empty string', () => { - expect(() => parseArgs(['db', 'branch', '--name'], SPECS)).toThrow(); + const failure = thrownBy(() => parseArgs(['db', 'branch', '--name'], SPECS)); + expect(failure.code).toBe('X_CLI_BAD_FLAG'); + expect(String(failure.cause)).toContain('expects a value'); }); test('bare argv and --help both route to the help command', () => { diff --git a/packages/testing/CLAUDE.md b/packages/testing/CLAUDE.md index b1921b59f..75a483158 100644 --- a/packages/testing/CLAUDE.md +++ b/packages/testing/CLAUDE.md @@ -12,12 +12,14 @@ factories only**, so a test that never destructures `mail` never loads the mail | No unmocked egress | `sealed-network.ts` patches fetch; a miss is `X_TEST_NETWORK_SEALED` | | Self is not egress | a port core's `markListening()` announced passes through — a socket test never unseals | | Offline is a state, not a mock | `network.offline()` / `.drop()` fail every request as `X_TEST_NETWORK_OFFLINE`, ahead of the mocks — the app's own offline path runs | +| One way offline | the `network` fixture. `setNetworkState` is the gate's only writer and is not exported — setting it from a test body skips the fixture's disposal and leaves every later file offline | | No retries | a flake is fixed or deleted the day it flakes; there is no `retry: 3` | | Test names | the filename picks the step; `testName(type, name)` on the outer `describe` puts that type on every failure line under it. Never on the inner `test` too — the prefix would print twice | | Injection | `SqlRunner` and `connect` are parameters, so unit tests need no server | | Fixtures | the preload registers the whole framework bag — an app registers only what the framework cannot know (`seed`, `actorFor`) | | Built vs declared | `clock` `mail` `network` `runJobs` are built in-process; `page` `budget` `signIn` `deploy` `subscribe` are declared and wait for a driver (`X_TEST_FIXTURE_UNAVAILABLE`) | | One seam for drivers | a driver registers over a declaration with `defineFixtures` — merges, last wins. Never a second registration mechanism | +| A driver arrives whole | `defineFixtures` holds every name `Fixtures` declares to its declared type, so a half-built `page` is a compile error at the registration, not a missing method three awaits later | | Registry hygiene | the fixture registry is process-global; a test that clears it snapshots with `fixtureSnapshot()` and hands it back in `afterAll` | | Fixture teardown | a fixture that installs process-global state (the ambient job or mail driver) implements `Symbol.dispose` / `Symbol.asyncDispose` and restores what was there; `fixtureTest` disposes in reverse build order even when the body throws | | Building one by hand | `createRunJobs()` outside `fixtureTest` is not disposed for you — reset the driver in `afterEach`, or the next file in the process inherits your queue | diff --git a/packages/testing/README.md b/packages/testing/README.md index e160cd626..7b538ded8 100644 --- a/packages/testing/README.md +++ b/packages/testing/README.md @@ -62,7 +62,11 @@ with no driver fails as `X_TEST_FIXTURE_UNAVAILABLE`, naming the driver rather t register a fixture that is not yours to define. A driver arrives through the same registry — `defineFixtures` merges, last registration wins — so there is no second seam to learn. -`mail`, `network` and `runJobs` install a process-global driver for the length of one test and hand the previous one back afterwards. A fixture that takes over a global does the same: implement `Symbol.dispose` or `Symbol.asyncDispose` on what the factory returns, and `fixtureTest` calls it in reverse build order — including when the test body throws. +The declaration is also the driver's type: `defineFixtures` holds every name `Fixtures` declares to +the type it was declared with, so a half-built `page` is a compile error at the registration rather +than a missing method three awaits into a later test. + +`mail`, `network` and `runJobs` install a process-global driver for the length of one test and hand the previous one back afterwards — the state they *found*, not a fixed default, so an outer fixture already offline stays offline when an inner one disposes. A fixture that takes over a global does the same: implement `Symbol.dispose` or `Symbol.asyncDispose` on what the factory returns, and `fixtureTest` calls it in reverse build order — including when the test body throws. Going offline is the `network` fixture's job and only its job; the gate's writer is not exported, because a test that set it directly would skip that disposal and take every later file down with it. An app adds its own with `defineFixtures` and widens the type by augmenting `Fixtures`: diff --git a/packages/testing/src/fixture-drivers.test.ts b/packages/testing/src/fixture-drivers.test.ts index b75a5e290..bdb41e837 100644 --- a/packages/testing/src/fixture-drivers.test.ts +++ b/packages/testing/src/fixture-drivers.test.ts @@ -1,3 +1,7 @@ +// The names the framework declares but cannot build: that asking for one without a driver fails as +// UNAVAILABLE rather than UNKNOWN, that it fails when built rather than when used, and that a +// driver arrives through the ordinary registry merge — whole, or not at all. + import { afterAll, test as bunTest, describe, expect } from 'bun:test'; import { DRIVER_FIXTURE_NAMES, @@ -7,6 +11,8 @@ import { } from './fixture-drivers'; import { defineFixtures, fixtureSnapshot, registeredFixtures, runWithFixtures } from './fixtures'; import { registerFrameworkFixtures } from './framework-fixtures'; +import type { LocatorLike, PageLike } from './test-types'; +import { testName } from './test-types'; // The registry is process-global. This file registers a driver over `page`, so it hands the // framework's own declaration back — otherwise every later file inherits this file's stub. @@ -15,7 +21,29 @@ afterAll(() => { defineFixtures(BEFORE); }); -describe('unit · the declared-but-driverless fixtures', () => { +const stubLocator = (): LocatorLike => ({ + count: async () => 0, + click: async () => undefined, + first: () => stubLocator(), + isVisible: async () => false, +}); + +/** A driver is a whole `PageLike` or it is not a driver — see the rejection case below. */ +const stubPage = (url: string): PageLike => ({ + goto: async () => undefined, + gotoStreamed: async () => ({ html: '' }), + reload: async () => undefined, + waitForServiceWorker: async () => undefined, + title: async () => '', + content: async () => '', + url: () => url, + evaluate: async <T>(fn: () => T) => fn(), + locator: stubLocator, + getByRole: stubLocator, + getByText: stubLocator, +}); + +describe(testName('unit', 'the declared-but-driverless fixtures'), () => { bunTest('every declared name has a driver requirement written down', () => { expect([...DRIVER_FIXTURE_NAMES]).toEqual(['budget', 'deploy', 'page', 'signIn', 'subscribe']); for (const name of DRIVER_FIXTURE_NAMES) { @@ -59,7 +87,7 @@ describe('unit · the declared-but-driverless fixtures', () => { // How a browser driver arrives: the ordinary registry merge, no second seam to learn. bunTest('a registered driver replaces the declaration', async () => { registerFrameworkFixtures(); - defineFixtures({ page: () => ({ url: () => '/feed' }) }); + defineFixtures({ page: () => stubPage('/feed') }); let seen = ''; await runWithFixtures(async ({ page }) => { @@ -68,4 +96,14 @@ describe('unit · the declared-but-driverless fixtures', () => { expect(seen).toBe('/feed'); }); + + // The declared type is the contract a driver signs, so a half-built one is a compile error at + // the registration — not a missing method three awaits into some later test. `@ts-expect-error` + // fails the compile if this ever starts type-checking, which is the assertion. + bunTest('a half-built driver does not type-check', () => { + registerFrameworkFixtures(); + // @ts-expect-error `url()` alone is not a PageLike — no goto, no locator, no service worker. + expect(() => defineFixtures({ page: () => ({ url: () => '/feed' }) })).not.toThrow(); + defineFixtures(BEFORE); + }); }); diff --git a/packages/testing/src/fixture-drivers.ts b/packages/testing/src/fixture-drivers.ts index 93283d80b..93eee41e7 100644 --- a/packages/testing/src/fixture-drivers.ts +++ b/packages/testing/src/fixture-drivers.ts @@ -1,16 +1,9 @@ // The fixtures the framework DECLARES but cannot build in this process: a browser for `page`, // `budget`, `signIn` and `deploy`; a replicator feeding a live-query registry for `subscribe`. -// -// Declared here rather than left out, because the name is the contract. An app that registered its -// own `page` would be deciding for itself what a page is, and two apps would then disagree — the -// same reason `clock` is not an app fixture. And an unregistered name fails as -// X_TEST_FIXTURE_UNKNOWN, whose fix ("register it") is the wrong instruction: what is missing is a -// driver, not a registration. So the name resolves, and asking for it without a driver says so. -// -// A driver overrides these the ordinary way — `defineFixtures` merges, last registration wins. +// Each is a type a driver implements, plus a factory that says what is missing until one does. import { fixtureUnavailable } from './errors'; -import type { FixtureFactory } from './fixtures'; +import type { FixtureFactory, Fixtures } from './fixtures'; /** Per-route byte budgets, measured off the built output rather than declared. */ export interface TestBudget { @@ -73,17 +66,29 @@ export const DRIVER_FIXTURE_NEEDS: Readonly<Record<DriverFixtureName, string>> = }; /** + * Declared rather than left out, because the name is the contract. An app that registered its own + * `page` would be deciding for itself what a page is, and two apps would then disagree — the same + * reason `clock` is not an app fixture. And an unregistered name fails as X_TEST_FIXTURE_UNKNOWN, + * whose fix ("register it") is the wrong instruction: what is missing is a driver, not a + * registration. So the name resolves, and asking for it without a driver says so. + * * Throws when built, not when used. Building is where the test is still on its own first line, so * the failure names the fixture instead of surfacing three awaits later as a missing method. */ export const unavailableFixture = - (name: DriverFixtureName): FixtureFactory => + <K extends DriverFixtureName>(name: K): FixtureFactory<Fixtures[K]> => () => { throw fixtureUnavailable(name, DRIVER_FIXTURE_NEEDS[name]); }; -/** The declared bag, as `defineFixtures` takes it. */ -export const driverFixtures = (): Readonly<Record<DriverFixtureName, FixtureFactory>> => ({ +/** Each declaration carries the type its driver must satisfy — `defineFixtures` holds it to that. */ +export type DriverFixtures = { readonly [K in DriverFixtureName]: FixtureFactory<Fixtures[K]> }; + +/** + * The declared bag, as `defineFixtures` takes it. A driver overrides these the ordinary way — + * `defineFixtures` merges, last registration wins. + */ +export const driverFixtures = (): DriverFixtures => ({ budget: unavailableFixture('budget'), deploy: unavailableFixture('deploy'), page: unavailableFixture('page'), diff --git a/packages/testing/src/fixture-network.test.ts b/packages/testing/src/fixture-network.test.ts index 971cf3566..8c1c05043 100644 --- a/packages/testing/src/fixture-network.test.ts +++ b/packages/testing/src/fixture-network.test.ts @@ -8,6 +8,7 @@ import { sealNetwork, unsealNetwork, } from './sealed-network'; +import { testName } from './test-types'; // The gate is process-global and bun shares one process across files: a test that leaves the // process offline takes every later file's fetch down with it. @@ -18,7 +19,7 @@ afterEach(() => { const URL_UNDER_TEST = 'https://api.stripe.test/v1/charges'; -describe('unit · the network fixture', () => { +describe(testName('unit', 'the network fixture'), () => { bunTest('offline fails the request the app’s offline path is written for', async () => { sealNetwork(); const network = createTestNetwork(); @@ -100,4 +101,21 @@ describe('unit · the network fixture', () => { expect(networkState()).toBe('online'); }); + + // Restore, not reset: an outer fixture that is already offline is the state this one found, and + // forcing 'online' on the way out would let an inner fixture's disposal reconnect the outer test. + bunTest('disposal restores the state it found, not online', () => { + sealNetwork(); + const outer = createTestNetwork(); + outer.drop(); + + const inner = createTestNetwork(); + inner.online(); + inner[Symbol.dispose](); + + expect(networkState()).toBe('dropped'); + + outer[Symbol.dispose](); + expect(networkState()).toBe('online'); + }); }); diff --git a/packages/testing/src/fixture-network.ts b/packages/testing/src/fixture-network.ts index 3535873d0..a65bb6193 100644 --- a/packages/testing/src/fixture-network.ts +++ b/packages/testing/src/fixture-network.ts @@ -1,10 +1,6 @@ -// The `network` fixture: pull the cable, put it back. What an offline test needs is not a mock -// that answers differently — it is a request that fails the way a real one fails, so the app's own -// offline path runs instead of a branch written for the test. -// -// `drop()` exists next to `offline()` because the two are different bugs. A clean offline is what a -// service worker answers; a dropped connection is what a live subscription must resume from. A -// fixture with only one of them cannot tell a resubscribe apart from a resume. +// The `network` fixture: pull the cable, put back exactly what was there. What an offline test +// needs is not a mock that answers differently — it is a request that fails the way a real one +// fails, so the app's own offline path runs instead of a branch written for the test. import { isNetworkSealed, @@ -15,11 +11,15 @@ import { unsealNetwork, } from './sealed-network'; -/** `Disposable`: the gate is process-global, so the fixture puts the process back online after. */ +/** `Disposable`: the gate is process-global, so the fixture puts back the state it found. */ export interface TestNetwork extends Disposable { /** Every request fails as it would with no route to the host. */ offline(): void; - /** Offline, and the connection was cut rather than closed — a subscriber must reconnect. */ + /** + * Offline, and the connection was cut rather than closed — a subscriber must reconnect. + * Next to `offline()` because the two are different bugs: a clean offline is what a service + * worker answers, and a fixture with only one of them cannot tell a resume from a resubscribe. + */ drop(): void; online(): void; state(): NetworkState; @@ -34,6 +34,9 @@ export function createTestNetwork(): TestNetwork { // A process that unsealed on purpose (ULTIMATE_TEST_ALLOW_NET=1) still gets a working // `offline()`, and gets its unsealed fetch back on disposal rather than keeping ours. const sealedBefore = isNetworkSealed(); + // Both halves of what was here, because disposal restores rather than assumes. An outer fixture + // already offline must not come back online because an inner one finished. + const stateBefore = networkState(); let sealedByUs = false; const goto = (next: NetworkState): void => { @@ -50,7 +53,7 @@ export function createTestNetwork(): TestNetwork { online: () => goto('online'), state: networkState, [Symbol.dispose]: (): void => { - setNetworkState('online'); + setNetworkState(stateBefore); if (sealedByUs && !sealedBefore) unsealNetwork(); sealedByUs = false; }, diff --git a/packages/testing/src/fixtures.ts b/packages/testing/src/fixtures.ts index 0dac9588e..9c342d3d9 100644 --- a/packages/testing/src/fixtures.ts +++ b/packages/testing/src/fixtures.ts @@ -20,6 +20,7 @@ import type { PageLike } from './test-types'; /** Built once per test, on first use. */ export type FixtureFactory<T = unknown> = () => T | Promise<T>; +/** The registry's own shape, where the built type is erased — what `fixtureSnapshot()` hands back. */ export type FixtureMap = Readonly<Record<string, FixtureFactory>>; /** @@ -50,10 +51,24 @@ export interface Fixtures { readonly subscribe: Subscribe; } +/** + * A registration bag, with every name the framework declares held to the type it was declared + * with. A driver that registers a half-built `page` is a compile error at the registration, rather + * than a missing method three awaits into some later test — the same reason the name is declared + * at all. A key `Fixtures` does not name is the app's, and takes any factory: the framework has + * nothing to check it against until the app augments `Fixtures`. + * + * Written over the argument's own keys rather than as an intersection, so a value typed only as + * `FixtureMap` — a snapshot on its way back into the registry — still satisfies it. + */ +export type FixtureRegistration<M> = { + readonly [K in keyof M]: K extends keyof Fixtures ? FixtureFactory<Fixtures[K]> : FixtureFactory; +}; + const registry = new Map<string, FixtureFactory>(); -export function defineFixtures(map: FixtureMap): void { - for (const [name, factory] of Object.entries(map)) registry.set(name, factory); +export function defineFixtures<M extends FixtureRegistration<M>>(map: M): void { + for (const [name, factory] of Object.entries(map as FixtureMap)) registry.set(name, factory); } export function clearFixtures(): void { diff --git a/packages/testing/src/framework-fixtures.ts b/packages/testing/src/framework-fixtures.ts index 937498105..1ad4eb675 100644 --- a/packages/testing/src/framework-fixtures.ts +++ b/packages/testing/src/framework-fixtures.ts @@ -1,15 +1,6 @@ // Registers the fixtures the FRAMEWORK owns, so an app registers only what it owns (`seed`, -// `actorFor`, …). Called by the preload, which is why an app never writes `defineFixtures({ clock })` -// and why two apps cannot disagree about what `clock` means. -// -// Two lists, because the bag has two kinds of member. The first four are built here and always -// work. The rest are declared here and built by a driver the process installs — see -// `fixture-drivers.ts` for why a declared-and-unavailable fixture beats an unregistered name. -// -// Each factory imports its subsystem on demand, for the reason the whole registry is lazy: a test -// that never touches jobs must not pay for the queue, and a `packages/core` test must not boot -// mail. `defineFixtures` merges, so registering twice is idempotent — and so a driver registered -// afterwards replaces the declaration it was waiting on. +// `actorFor`, …). Called by the preload, which is why an app never writes +// `defineFixtures({ clock })` and why two apps cannot disagree about what `clock` means. import { createTestClock } from './fixture-clock'; import { DRIVER_FIXTURE_NAMES, driverFixtures } from './fixture-drivers'; @@ -18,7 +9,12 @@ import { createTestMail } from './fixture-mail'; import { createTestNetwork } from './fixture-network'; import { defineFixtures } from './fixtures'; -/** Built in-process. Always available, in every test type. */ +/** + * Built in-process. Always available, in every test type — the first of the bag's two kinds of + * member. The other is `DRIVER_FIXTURE_NAMES`: declared here, built by a driver the process + * installs. See `fixture-drivers.ts` for why a declared-and-unavailable fixture beats an + * unregistered name. + */ export const FRAMEWORK_FIXTURE_NAMES = ['clock', 'mail', 'network', 'runJobs'] as const; export { DRIVER_FIXTURE_NAMES }; @@ -29,6 +25,12 @@ export const ALL_FIXTURE_NAMES: readonly string[] = [ ...DRIVER_FIXTURE_NAMES, ].sort(); +/** + * Each factory imports its subsystem on demand, for the reason the whole registry is lazy: a test + * that never touches jobs must not pay for the queue, and a `packages/core` test must not boot + * mail. `defineFixtures` merges, so registering twice is idempotent — and so a driver registered + * afterwards replaces the declaration it was waiting on. + */ export function registerFrameworkFixtures(): void { defineFixtures({ ...driverFixtures(), diff --git a/packages/testing/src/index.ts b/packages/testing/src/index.ts index 11f208297..5b9b13f93 100644 --- a/packages/testing/src/index.ts +++ b/packages/testing/src/index.ts @@ -1,4 +1,10 @@ -export type { FixtureBody, FixtureFactory, FixtureMap, Fixtures } from './fixtures'; +export type { + FixtureBody, + FixtureFactory, + FixtureMap, + FixtureRegistration, + Fixtures, +} from './fixtures'; export { clearFixtures, defineFixtures, @@ -48,6 +54,7 @@ export type { TestClock, TestDuration } from './fixture-clock'; export { createTestClock } from './fixture-clock'; export type { DriverFixtureName, + DriverFixtures, LiveFeed, LiveFeedPatch, LiveTarget, @@ -75,6 +82,9 @@ export { describeApp, testApp } from './harness'; export type { MatcherResult } from './matchers'; export { matchersInstalled, recordSteps } from './matchers'; export type { MockRoute, NetworkState } from './sealed-network'; +// `setNetworkState` is deliberately not here: it is the offline gate's one writer, and a test that +// called it directly would bypass the `network` fixture's disposal and leave the whole process +// offline for every file after it. The fixture is the way to go offline — there is no second one. export { allowHost, isNetworkSealed, @@ -84,7 +94,6 @@ export { requestedUrls, resetNetwork, sealNetwork, - setNetworkState, unsealNetwork, } from './sealed-network'; export type { SqlRunner, TemplateDbConfig, WorkerDatabase } from './template-db'; diff --git a/scripts/verify.test.ts b/scripts/verify.test.ts index 7bafbf256..ad0693642 100644 --- a/scripts/verify.test.ts +++ b/scripts/verify.test.ts @@ -3,7 +3,7 @@ import { mkdtemp, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { verifyStepNames } from '@ultimat3/cli'; -import { repoRoot } from './lib/run'; +import { repoRoot, run } from './lib/run'; // Only the integration assertion below lives here — `checkRoadmap`'s own cases are in // `scripts/roadmap.test.ts`, next to their source. import { checkRoadmap } from './roadmap'; @@ -88,6 +88,45 @@ describe('unit · the repo gate is the CLI gate', () => { expect(field(field(config, 'compilerOptions'), 'paths')).toEqual({}); }); + // Both tests exist because either alone lies. The one above pins the mechanism — `paths` is the + // knob, and an edit that refills it should be caught by name. This one pins the effect: emptying + // `paths` does not by itself make `@ultimat3/core` unresolvable, since node resolution would + // still walk up to a linked workspace package in `node_modules`. Only a real compile answers. + test('site/ is its own bundle graph — a real tsc cannot resolve @ultimat3/core inside it', async () => { + // The probe sits under `site/` so resolution walks the same ancestor directories a real site + // source does. The `.` prefix keeps it out of site/tsconfig.json's own `**/*.ts` include + // while it exists — TypeScript's wildcards skip dot-directories. + const dir = await mkdtemp(join(repoRoot(), 'site', '.probe-')); + try { + await Bun.write(join(dir, 'probe.ts'), "import '@ultimat3/core';\n"); + await Bun.write( + join(dir, 'tsconfig.json'), + `${JSON.stringify( + { + extends: '../tsconfig.json', + compilerOptions: { noEmit: true }, + files: ['./probe.ts'], + include: [], + }, + null, + 2, + )}\n`, + ); + const config = join(dir, 'tsconfig.json'); + const result = await run(['bunx', 'tsc', '-p', config, '--pretty', 'false'], { + cwd: repoRoot(), + }); + expect(result.ok).toBe(false); + // Asserted in two fragments rather than one sentence: tsc words a side-effect import's + // failure (TS2882) differently from a named one (TS2307), and which sentence this tsc + // build prints is not what the test is about — that the specifier does not resolve is. + expect(result.output).toContain('Cannot find module'); + expect(result.output).toContain("'@ultimat3/core'"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }, 60_000); + test('this repo has no tier violations and its manifest still generates', async () => { const root = repoRoot(); expect(await tierBoundaries(root)).toEqual([]); diff --git a/site/README.md b/site/README.md index 8c5dcc68a..06156aac4 100644 --- a/site/README.md +++ b/site/README.md @@ -65,10 +65,11 @@ tsconfig.json # this directory is a TS project, referenced by the ro CNAME # ultimate.developerz.ai ``` -`site/` is a project the root `tsconfig.json` references, so `bun run verify`'s `typecheck` step -covers it: a type error here is a red gate, exactly as it is in `packages/`. Its `paths` are empty -on purpose — the site is its own bundle graph (axiom 6), so no `@ultimat3/*` import can resolve -inside it. The build itself stays dependency-free and runs straight off the TypeScript sources. +As of 2026-08, `site/` is a project the root `tsconfig.json` references, so `bun run verify`'s +`typecheck` step covers it: a type error here is a red gate, exactly as it is in `packages/`. Its +`paths` are empty on purpose — the site is its own bundle graph (axiom 6), so no `@ultimat3/*` +import can resolve inside it. The build itself stays dependency-free and runs straight off the +TypeScript sources. ## Authoring From b8e621c451de06373fcd1928e5192066f057ee7d Mon Sep 17 00:00:00 2001 From: sebi <gore.sebyx@yahoo.com> Date: Mon, 10 Aug 2026 09:37:51 -0500 Subject: [PATCH 4/4] =?UTF-8?q?docs:=20address=20PR=20#25=20review=20?= =?UTF-8?q?=E2=80=94=20header=20length,=20dated=20claim?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - test-setup.ts: drop the `bunfig.toml` preload snippet the header restated; bunfig.toml already states it, and a fact in two places drifts (axiom 2). Leaves an unambiguous 3-line header. - site/README.md: the tree row duplicated the dated claim below it; trimmed to what the file is, so `As of 2026-08` states it once. Co-Authored-By: Claude <noreply@anthropic.com> --- examples/dummy/scripts/test-setup.ts | 5 ----- site/README.md | 2 +- 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/examples/dummy/scripts/test-setup.ts b/examples/dummy/scripts/test-setup.ts index d12fa7536..91833a2b8 100644 --- a/examples/dummy/scripts/test-setup.ts +++ b/examples/dummy/scripts/test-setup.ts @@ -2,11 +2,6 @@ // how a member becomes an actor. Everything else in the bag arrives with the framework's own // preload, imported below, so an app registers only what the framework cannot know. -// The suite's one preload, named in `bunfig.toml`: -// -// [test] -// preload = ["./scripts/test-setup.ts"] -// // The framework is imported by relative path rather than by `@ultimat3/*`: this directory is not // a workspace member yet (issue #9), and a preload runs before anything else, so it must not // depend on workspace symlinks. A generated app writes `@ultimat3/testing` here. For the same diff --git a/site/README.md b/site/README.md index 06156aac4..9c2dfa733 100644 --- a/site/README.md +++ b/site/README.md @@ -61,7 +61,7 @@ scripts/theme.js # the only JS: before-paint theme apply + delegated to pages/*.md # content, with frontmatter assets/logo.svg # mark + favicon, theme-aware via its own internal stylesheet assets/og.svg # 1200x630 social card -tsconfig.json # this directory is a TS project, referenced by the root tsconfig +tsconfig.json # the site's own TypeScript project — the note below the tree owns why CNAME # ultimate.developerz.ai ```