Skip to content

test: remove quick-tier batch barriers and fixture waits - #12760

Closed
backryun wants to merge 14 commits into
diegosouzapw:release/v3.8.51from
backryun:codex/test-quick-tier-12589
Closed

backryun wants to merge 14 commits into
diegosouzapw:release/v3.8.51from
backryun:codex/test-quick-tier-12589

Conversation

@backryun

@backryun backryun commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

The quick tier previously stopped at nine sequential command-line batches. This continuation gives each phase one scheduler, preserves process isolation, and removes measured timer and host-environment dependencies from fixtures. The earlier local comparison of the same 3,903 selected files at concurrency 8 improved from 391.26s to 310.36s (20.7%); that benchmark predates this base refresh and has not been repeated.

Depends on #12588, then #12606. Restacked onto release/v3.8.51 at d6f3150. The release-relative diff includes both parents because their branches live on a fork. Review this continuation's four commits / 24 files in the incremental parent comparison.

Changes:

  • Derive the opt-in quick tier from canonical collectors, retaining the six measured exclusions, dashboard loader, serial concurrency, and file isolation. Pass paths through stdin to remove command-line limits and batch barriers.
  • Clear inherited application DB overrides before the per-file isolation preload. Record JSONL timings, phase results, and worker exits; drain reporter output and preserve failure, skip, TODO, and leaked-handle behavior.
  • Replace timer waits in Batch, compression, provider/combo recovery, and JobRegistry fixtures with existing helpers and controlled clocks. Repair Redis command framing and calendar, CLI, janitor, binary-path, and mocked Adobe fixtures without dropping assertions.
  • Record the base refresh, CI shard correction in the parent, and current validation in docs/architecture/TEST_EXECUTION_TIERS.md.

Current validation on Node 24.20.0:

  • All 14 quick runner/worker regressions passed in the full Node run. The plan selects 3,927 of 4,985 canonical Node files.
  • Full Node main: 37,675 passed, 34 failed, 23 skipped. Every failure name also appeared in the release-base run (37,698 passed, 43 failed, 23 skipped). These are separate runs with different partitions and scheduling, not a controlled speed or assertion-count comparison.
  • Dashboard separately: 163 passed, one failure with the same redaction assertion as the base. Serial: 23 passed.
  • Vitest: 465 passed in 51 files after initializing the isolated temporary DB. The initial attempt had a concurrent migration collision and skipped five tests in one failed suite.
  • Parent relocation/conflict tests passed 366 and 37 tests; partition/serial guards passed 15. Discovery passed with 5,576 files, 34 collectors, and nine frozen orphans.
  • Integration shard execution found shared-harness import drift, now fixed in Test tiers: split slow e2e and upstream-gated live out of default integration glob #12606. The remaining six combo assertion failures reproduced on the release base. Live collection passed one mocked test and skipped 17 opt-in tests. Resilience HTTP failed and hung, requiring termination.
  • Runner-API and pre-commit checks passed. Strict mutation registration retains two base omissions. check:docs-all retains six base count drifts (171 migrations versus 169; 20 routing strategies versus 19).
  • Full coverage was not rerun; the 60/60/60/60 floor is unchanged. CI must be checked on the newly pushed heads.

⚠️ base-red inherited: #12732. The separate production repair candidate remains #12333. Full Node, integration, mutation, and documentation gates are still red; this does not establish a green combined release base. Auto-merge is not enabled.

Refs #12539 and #12589. Landing order remains #12588 → #12606 → this continuation.

@backryun
backryun force-pushed the codex/test-quick-tier-12589 branch 2 times, most recently from 35240e5 to 779c57c Compare September 6, 2026 23:18
@backryun backryun changed the title test: resume tiered execution and eliminate fixture waits test: remove quick-tier batch barriers and fixture waits Sep 7, 2026
@backryun
backryun force-pushed the codex/test-quick-tier-12589 branch 2 times, most recently from d3c01e2 to c9a56fd Compare September 7, 2026 22:34
backryun and others added 14 commits September 9, 2026 05:20
Moves all ~4.3k tests/unit/*.test.* files into per-prefix subdirectories
(e.g. chatcore/, combo/, translator/, cli/, db/, provider/) plus misc/ for
the unclassifiable tail. No test logic changed except one-level-deeper
relative import/path fixes.

Syncs the 4 runner-glob owners (package.json test:unit scripts,
check-test-discovery.mjs, build-test-impact-map.mjs, merge-train.sh),
renames quality-baseline keys (file-size, eslint-suppressions,
test-masking-allowlist), and updates docs/ test-path references so the
fabricated-docs, test-discovery, file-size and masking gates stay green.

Refs diegosouzapw#12539
- stryker.conf.json: remap 338 tap.testFiles entries from flat
  tests/unit/*.test.ts paths to their new per-prefix subdirs
  (fixes check:mutation-test-coverage --strict).
- tests/unit/serial/: evict refresh-serializer*.test.ts misclassified
  by substring match into the concurrency quarantine; they belong to
  oauth/. Drop serial from the parallel runner brace so the
  quarantine is not double-executed (fixes test-serial-quarantine).
claude-fable, embedding-clova, obscura, provider-cost-data,
translator-clova-v3 -> per-prefix subdirs. flat = 0 again.
… tier split

The suite relocation left tests/unit/db/db-pre-migration-backup-retention-10421.test.ts importing ../../src (resolves to tests/src), and merge-train.sh / TIA omitted the moonshot and seekai prefixes that test:unit actually runs.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…tegration glob

Moves 8 slow hermetic suites (batch-e2e 150s, live-ws-heartbeat 54s,
resilience-http 36s, +5) to tests/integration/e2e/ with a dedicated
test:integration:e2e script, and 7 upstream-gated suites (live-gemini*,
gemini-live-429, codex-chat-reasoning, wire-capture) to
tests/integration/live/ with test:integration:live. Shared helpers used
by both tiers stay at tests/integration/ root; tier-private helpers move
along. CI runs both tiers after the default suite so no assertions stop
running anywhere.

Default test:integration: 115 -> 100 files. Pre-existing failures
(combo-failover/routing assertions, resilience-http-e2e 1 case) unchanged
and out of scope.

Refs diegosouzapw#12539
@backryun
backryun force-pushed the codex/test-quick-tier-12589 branch from c9a56fd to 68b7baa Compare September 8, 2026 20:23
@backryun

Copy link
Copy Markdown
Contributor Author

Superseded by #13163, which rebuilds the useful deterministic fixture changes as one focused commit on the current release base. The replacement preserves existing test paths and CI collection. This branch is retained for reference.

@backryun backryun closed this Sep 10, 2026
@backryun
backryun deleted the codex/test-quick-tier-12589 branch September 10, 2026 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant