Skip to content

ci: gate integrity — foundations - #25

Merged
sebyx07 merged 4 commits into
mainfrom
fix/register-fixture-bag
Aug 10, 2026
Merged

sebyx07 merged 4 commits into
mainfrom
fix/register-fixture-bag

Conversation

@sebyx07

@sebyx07 sebyx07 commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Fix reference-app tsconfig for TypeScript 7 (baseUrl/paths, already merged upstream)
  • Register the fixture bag so app tests can run
  • Let a booted test server reach its own loopback (already merged upstream)
  • Put site/ under the gate's typecheck scope
  • Un-exclude e2e/ suites in CI: verified already done — package.json's test script and
    .github/workflows/ci.yml's test job carry no opt-in exclusions at all (commit 1131e4e,
    already on main); bun run test runs e2e (and contract/live/job/eval) every time. No code
    change needed for this task; confirmed via a full local run (3481 pass/0 fail) and bun run verify (17/17 green).

Test plan

  • bun run verify — 17/17 green
  • bun run test — 3481 pass / 0 fail / 23 skip

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Summary by CodeRabbit

  • New Features

    • Added controllable test network states: online, offline, and dropped.
    • Added framework and driver-backed testing fixtures for browser access, budgets, deployment, sign-in, and subscriptions.
    • Added clearer errors for unavailable fixtures and blocked network requests.
    • Expanded page and locator testing capabilities.
  • Documentation

    • Updated testing, fixture, error-code, and project verification documentation.
  • Bug Fixes

    • Root verification now typechecks all referenced projects, including the site project.
    • Improved test coverage for fixture availability and network cleanup.

sebyx07 and others added 2 commits August 10, 2026 08:41
- `network` fixture: offline()/drop()/online() over an offline gate in the
  sealed network, ahead of the mocks so the app's own offline path runs.
  New X_TEST_NETWORK_OFFLINE.
- `page`, `budget`, `signIn`, `deploy`, `subscribe` declared in
  fixture-drivers.ts: the name resolves and asking for it without a browser
  or a replicator fails as X_TEST_FIXTURE_UNAVAILABLE naming the driver —
  not X_TEST_FIXTURE_UNKNOWN, whose fix would have each app inventing its
  own idea of what a page is. A driver arrives through defineFixtures; no
  second seam.
- Fixtures type widened to the full bag; PageLike grown to the members the
  reference app actually calls.
- examples/dummy keeps only seed + actorFor. Its 11 X_TEST_FIXTURE_UNKNOWN
  failures are now 11 X_TEST_FIXTURE_UNAVAILABLE (6 page, 5 subscribe).

Co-Authored-By: Claude <noreply@anthropic.com>
- new site/tsconfig.json, referenced from the root — `tsc -b` never
  walked site/, so build.ts and lib/* shipped untypechecked
- empty `paths` there: site/ is its own bundle graph (axiom 6), so an
  @ultimat3/* import is now a resolution error, not a convention
- declare the frontmatter vocabulary as `PageMeta`, clearing the 24
  TS4111s a bare Record produced; `meta.nav` is checked, `meta.navv`
  is not a property
- typecheck step summary said "every workspace"; site/ is not one
- parse.test.ts fixture gave `verify` an `--only` flag it does not and
  must not have — moved the string-flag case onto `db --name`

Co-Authored-By: Claude <noreply@anthropic.com>
@sebyx07 sebyx07 added the claudetm Created by Claude Task Master label Aug 10, 2026
@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 30 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6d9a25c5-0d7e-4c52-949b-de1b9d6dfa11

📥 Commits

Reviewing files that changed from the base of the PR and between 5bc42ed and 906d9df.

📒 Files selected for processing (13)
  • examples/dummy/scripts/test-setup.ts
  • packages/cli/src/parse.test.ts
  • packages/testing/CLAUDE.md
  • packages/testing/README.md
  • packages/testing/src/fixture-drivers.test.ts
  • packages/testing/src/fixture-drivers.ts
  • packages/testing/src/fixture-network.test.ts
  • packages/testing/src/fixture-network.ts
  • packages/testing/src/fixtures.ts
  • packages/testing/src/framework-fixtures.ts
  • packages/testing/src/index.ts
  • scripts/verify.test.ts
  • site/README.md
📝 Walkthrough

Walkthrough

The testing package adds framework and driver-backed fixtures, unavailable-driver errors, and process-global network controls. The site becomes a referenced TypeScript project with typed page metadata. CLI parser tests now cover db branch --name.

Changes

Testing fixtures and network controls

Layer / File(s) Summary
Fixture contracts and availability errors
packages/testing/src/fixture-drivers.ts, packages/testing/src/errors.ts, packages/testing/src/fixtures.ts, packages/testing/src/test-types.ts
The testing API defines driver-backed fixtures, live-query contracts, locator operations, and structured unavailable or offline errors.
Framework fixture registration
packages/testing/src/framework-fixtures.ts, packages/testing/src/index.ts, packages/testing/src/fixture-drivers.test.ts, packages/testing/src/framework-fixtures.test.ts, packages/testing/README.md, wiki/Testing.md, wiki/Error-Codes.md, examples/dummy/...
The registry declares all framework fixtures and allows drivers to replace unavailable factories. Tests and documentation cover fixture ownership, lookup errors, registration, and disposal.
Network fixture state and cleanup
packages/testing/src/sealed-network.ts, packages/testing/src/fixture-network.ts, packages/testing/src/fixture-network.test.ts
The network fixture supports online, offline, and dropped states. Requests fail with structured details when the state is not online. Disposal restores state and sealing.

Site project verification

Layer / File(s) Summary
Site TypeScript project
site/tsconfig.json, tsconfig.json, packages/cli/src/cmd-verify.ts, scripts/verify.test.ts, site/README.md
Root verification now checks referenced projects, including the isolated site project. Tests validate the project reference and empty site path mappings.
Page metadata contract
site/lib/config.ts
PageMeta defines known optional frontmatter fields and permits additional metadata keys. Page.meta uses this contract.

CLI parser fixtures

Layer / File(s) Summary
Database branch flag parsing
packages/cli/src/parse.test.ts
Parser tests cover db branch --name in separated and equals forms, passthrough handling, and missing values.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Test
  participant TestNetwork
  participant SealedNetwork
  participant Fetch
  participant NetworkOfflineError
  Test->>TestNetwork: offline() or drop()
  TestNetwork->>SealedNetwork: setNetworkState(mode)
  Fetch->>SealedNetwork: request URL and method
  SealedNetwork->>NetworkOfflineError: reject non-online request
  Test->>TestNetwork: dispose()
  TestNetwork->>SealedNetwork: restore online state and sealing
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately identifies foundational CI gate and test-infrastructure changes, although it does not name each implementation area.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/register-fixture-bag

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/dummy/scripts/test-setup.ts`:
- Around line 1-5: Reduce the header comment above the test preload setup to no
more than four lines while preserving its statement that the file registers only
Postly-owned fixtures: the seed graph and member-to-actor setup.

In `@packages/cli/src/parse.test.ts`:
- Line 87: Update the missing-value test around parseArgs to capture the thrown
error with thrownBy, then assert its stable X_CLI_BAD_FLAG code and the “expects
a value” cause instead of only checking that any exception occurs. Preserve the
existing db branch --name input and SPECS setup.

In `@packages/testing/src/fixture-drivers.test.ts`:
- Line 18: Update the outer describe suite in the declared-but-driverless
fixtures test to import and use testName('unit', 'the declared-but-driverless
fixtures') as its suite name, preserving the existing suite body.

In `@packages/testing/src/fixture-drivers.ts`:
- Around line 1-10: Shorten the header in
packages/testing/src/fixture-drivers.ts (lines 1-10) to 1–4 lines stating the
file’s responsibility and rationale. Apply the same reduction to
packages/testing/src/framework-fixtures.ts (lines 1-12), and add a 1–4 line
responsibility-and-rationale header before imports in
packages/testing/src/fixture-drivers.test.ts (lines 1-12).

In `@packages/testing/src/fixture-network.test.ts`:
- Line 21: Update the outer describe suite in the network fixture test to call
testName('unit', 'the network fixture') instead of using a literal suite name,
while leaving the nested test structure unchanged.

In `@packages/testing/src/fixture-network.ts`:
- Around line 36-55: Capture the existing NetworkState before any mutation in
the fixture setup, then restore that saved value in the returned object’s
[Symbol.dispose] instead of always setting the state to 'online'. Keep the
sealedByUs and sealedBefore cleanup behavior unchanged.

In `@packages/testing/src/fixtures.ts`:
- Around line 20-23: Update defineFixtures and its FixtureMap typing to preserve
each declared fixture’s type by checking factories for every keyof Fixtures
against Fixtures[K], while continuing to allow custom application fixture keys.
Revise the fixture-drivers test registration for page to provide a complete
PageLike-compatible driver, or add a compile-time assertion that rejects the
incomplete driver.

In `@packages/testing/src/index.ts`:
- Around line 77-89: Remove setNetworkState from the public exports in
packages/testing/src/index.ts lines 77-89. Keep the setNetworkState
implementation in packages/testing/src/sealed-network.ts lines 103-106 internal
for fixture implementation use, with no direct change required there beyond
preserving its non-public status.

In `@scripts/verify.test.ts`:
- Around line 83-90: Update the test around the existing site tsconfig assertion
to validate actual module resolution, not just compilerOptions.paths. In the
test module, create and compile a temporary probe importing `@ultimat3/core` with
site/tsconfig.json, then assert compilation fails with “Cannot find module
'`@ultimat3/core`'”; retain cleanup and the existing paths assertion as
appropriate.

In `@site/README.md`:
- Around line 64-72: Add “As of 2026-07” to the documentation paragraph
describing the site TypeScript project and bun run verify typecheck coverage,
while preserving the existing claims and wording.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4e8b0fd4-fb3e-4fcc-bff9-810a24887dfe

📥 Commits

Reviewing files that changed from the base of the PR and between 68172c0 and 5bc42ed.

📒 Files selected for processing (24)
  • examples/dummy/CLAUDE.md
  • examples/dummy/scripts/test-setup.ts
  • packages/cli/src/cmd-verify.ts
  • packages/cli/src/parse.test.ts
  • packages/testing/CLAUDE.md
  • packages/testing/README.md
  • packages/testing/src/errors.ts
  • packages/testing/src/fixture-drivers.test.ts
  • packages/testing/src/fixture-drivers.ts
  • packages/testing/src/fixture-network.test.ts
  • packages/testing/src/fixture-network.ts
  • packages/testing/src/fixtures.ts
  • packages/testing/src/framework-fixtures.test.ts
  • packages/testing/src/framework-fixtures.ts
  • packages/testing/src/index.ts
  • packages/testing/src/sealed-network.ts
  • packages/testing/src/test-types.ts
  • scripts/verify.test.ts
  • site/README.md
  • site/lib/config.ts
  • site/tsconfig.json
  • tsconfig.json
  • wiki/Error-Codes.md
  • wiki/Testing.md

Comment thread examples/dummy/scripts/test-setup.ts Outdated
Comment thread packages/cli/src/parse.test.ts Outdated
Comment thread packages/testing/src/fixture-drivers.test.ts Outdated
Comment thread packages/testing/src/fixture-drivers.ts Outdated
Comment thread packages/testing/src/fixture-network.test.ts Outdated
Comment thread packages/testing/src/fixture-network.ts
Comment thread packages/testing/src/fixtures.ts
Comment thread packages/testing/src/index.ts
Comment thread scripts/verify.test.ts
Comment thread site/README.md Outdated
sebyx07 and others added 2 commits August 10, 2026 09:28
- defineFixtures holds every declared name to its declared type, so a
  half-built `page` driver is a compile error at the registration rather
  than a missing method three awaits into a later test
- the network fixture restores the state it found on disposal instead of
  forcing 'online' — an outer fixture already offline stayed offline
- setNetworkState is no longer exported: the gate's one writer is the
  `network` fixture, and a direct call skipped its disposal
- parse.test.ts asserts X_CLI_BAD_FLAG + "expects a value", not any throw
- verify.test.ts compiles a probe under site/ and asserts @ultimat3/core
  cannot resolve — the effect, not just the `paths` knob
- file headers cut to the 1-4 line convention; the rationale moved onto
  the declarations it explains, not deleted
- testName('unit', …) on both new outer describes
- site/README.md dates its verification claims As of 2026-08

Co-Authored-By: Claude <noreply@anthropic.com>
- test-setup.ts: drop the `bunfig.toml` preload snippet the header
  restated; bunfig.toml already states it, and a fact in two places
  drifts (axiom 2). Leaves an unambiguous 3-line header.
- site/README.md: the tree row duplicated the dated claim below it;
  trimmed to what the file is, so `As of 2026-08` states it once.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claudetm Created by Claude Task Master

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant