feat(ci): wire up JS/TS tests in CI, fix all failures, ban source-regex tests - #60707
Conversation
5e11c60 to
1fdf323
Compare
ee19bb5 to
441be86
Compare
ReviewOverall this is a strong, well-motivated PR — wiring the JS/TS packages into CI, killing the "regex-scan the source in a test" antipattern, and doing the extractions as pure/DI modules is exactly the right call. I spot-checked the refactors and the behavior parity looks faithful ( A few findings, roughly in priority order: 1. The extracted electron unit tests don't actually run in CI
|
|
Thanks for putting this together — the overall direction is good, and the new JS/TS matrix is green across all five workspaces. I also verified the desktop UI suite (146 files / 1,180 tests), TUI suite (107 files / 1,115 passed), web suite (9 files / 59 tests), desktop build, and native host build locally. There are a few blockers before this is ready:
The extracted desktop/TUI helpers themselves look sound, the current production wiring preserves behavior, and the CI workflow rename is wired consistently. Once the issues above are addressed, this should be in much better shape for another pass. |
903c560 to
1aec395
Compare
f411fff to
0b2b25b
Compare
it's implied by ts and mjs files already
Add vitest.setup.ts with IS_REACT_ACT_ENVIRONMENT=true + auto-cleanup, and wrap render()/fireEvent() calls in act() across 8 test files: - provider-config-panel.test.tsx: wrap renderPanel + fireEvent in act - providers-settings.test.tsx: wrap renderProvidersSettings + fireEvent in act - use-prompt-actions/index.test.tsx: add actRender helper, wrap all 38 render calls, wrap Harness handle methods (submitText/cancelRun/steerPrompt/ restoreToMessage) in act at the onReady callback level - attachments.test.tsx: make renderWithI18n async + wrap in act - skills/index.test.tsx: make renderSkills async + wrap fireEvent in act - messaging/index.test.tsx: make renderMessaging async + wrap fireEvent in act - gateway-connecting-overlay.test.tsx: wrap all render/rerender in act - preview-pane.test.tsx: wrap render calls in act, make tests async Reduces act warnings from 44 to 4 (remaining are fake-timer + pre-render store mutation edge cases). All 146 test files / 1180 tests still pass.
…o native binary exists When neither a prebuild nor a compiled build/Release/*.node is found for the target platform-arch, stage-native-deps.mjs now runs electron-rebuild -f -w node-pty to compile one from source before re-copying build/Release into the staged dist. This makes the staging script self-sufficient — it always produces a working native binary dir regardless of whether npm ci --ignore-scripts skipped postinstall or whether node-pty publishes prebuilds for the target (e.g. linux-x64 has no prebuild).
b5f005d to
c8b2624
Compare
Extracted zoomWiringForWindowKind() + ZOOM_WINDOW_CONFIG into zoom.ts so
the pet-overlay-
opts-out / chat-windows-keep-zoom contract is tested via the pure
config,
not by reading source. Callers in main.ts now use
zoomWiringForWindowKind()
instead of inline { zoom: false } / default { zoom: true }.
…rlay Move setGatewayState + rerender calls inside act() blocks and make the synchronous soft-switch test async so all state updates are wrapped. Eliminates the last 4 act() warnings (44 → 0).
this is useful for debugging actions where a job's pass/fail isn't the same as it is in the gha ui (e.g. neutral status jobs)
c8b2624 to
2af3df0
Compare
…s targets
stageNodePty received electron-builder's { platform, arch } but unconditionally
copied host build/Release, staging a host binary (e.g. macOS Mach-O) for a
foreign target (e.g. linux-arm64). The fallback rebuild also didn't pass the
target arch to electron-rebuild.
Now:
- build/Release is only staged when target platform+arch match the host
- cross-platform targets with no matching prebuild fail closed
- same-platform different-arch rebuild passes --arch to electron-rebuild
- post-staging validation reads .node magic bytes (ELF/Mach-O/PE) and rejects
any binary whose platform doesn't match the target
Adds stageNodePtyInto() (testable core) and classifyNativeBinary() (pure),
plus 11 regression tests covering the cross-target scenarios.
The set-matrix step wrote the npm workspace query result directly to $GITHUB_OUTPUT. If discovery ever produced [], the matrix would expand to zero check jobs, leaving the reusable workflow green without running any JS/TS checks. Now the step validates the result is a non-empty array before emitting it, and exits 1 with a GitHub annotation if it's empty or jq failed.
8ea1126 to
08b8044
Compare
OutThisLife
left a comment
There was a problem hiding this comment.
Re-reviewed the latest head (08b8044) in a worktree off the PR branch. The blockers from the prior two rounds are all resolved — nice work:
- Electron production code is typechecked again (
tsc -p .andtsc -p tsconfig.electron.json). - Cross-OS native staging now fails closed (requesting
linux-arm64from a darwin host throws instead of staging a host binary). ui-tuidevcallsbuild:ink(no moreMissing script: build-ink).previewtypo fixed; the Electron suite runs under the vitestelectronproject and is now part ofcheck; the nested@hermes/inktypecheck is in the workspace matrix; empty-matrix discovery fails closed.
Locally: electron project = 37 files / 404 passed / 1 skipped, electron typecheck clean, and it merges cleanly onto current main.
One new blocker, though:
1. classifyNativeBinary rejects real Darwin .node files → macOS build/check fails ⚠️
The Mach-O magic bytes are byte-reversed. Real node-pty Darwin thin binaries are stored little-endian on disk:
$ xxd -l 4 node_modules/node-pty/prebuilds/darwin-arm64/pty.node
00000000: cffa edfe # MH_CIGAM_64 = 0xcffaedfe (arm64 + x64 both)
but the classifier checks the big-endian form:
// Mach-O 64-bit: feedfacf
if (buf[0] === 0xfe && buf[1] === 0xed && buf[2] === 0xfa && buf[3] === 0xcf) return 'darwin'so both Darwin prebuilds classify as null:
darwin-arm64/pty.node => null
darwin-x64/pty.node => null
win32-x64/pty.node => win32
On a Darwin host, stageNodePtyInto(..., { platform: 'darwin' }) then hits validateStagedBinaries and throws:
$ node apps/desktop/scripts/stage-native-deps.mjs # host = darwin
Error: [stage-native-deps] native binary platform mismatch (target=darwin):
prebuilds/darwin-arm64/pty.node: expected darwin, got unknown
$ echo $?
1
Since build ends with stage-native-deps.mjs and check ends with build, npm run check is broken for every macOS contributor. CI doesn't catch it because the runners are Linux (the ELF path is correct), and the new tests only plant big-endian fake headers — the exact false-confidence class this PR is trying to eliminate. The same command succeeds on main.
Fix: recognize the on-disk (little-endian) forms MH_CIGAM/MH_CIGAM_64 (ce fa ed fe / cf fa ed fe), and add a regression test that classifies a real Darwin prebuild (or writes CIGAM bytes). Worth handling FAT_CIGAM (be ba fe ca) too.
2. (carryover, low priority) electron-rebuild invoked via node_modules/.bin/electron-rebuild
Fine on POSIX, but on Windows the .bin entry is a shim rather than a Node script, so this fallback would fail exactly where node-pty prebuild gaps happen. Prefer resolving @electron/rebuild/lib/cli.js (or require.resolve). Only fires on the no-prebuild rebuild path.
Nit
validateStagedBinaries checks platform but not arch, so a same-platform wrong-arch binary could still slip through. Secondary to #1.
Overall the direction and the extraction work are solid — happy to approve once #1 is fixed and re-verified against a real Darwin prebuild (or a CIGAM fixture) in stage-native-deps.test.mjs.
…lassifier classifyNativeBinary only checked big-endian Mach-O/Fat magic bytes (feedfacf, feedfac, cafebabe). Real Darwin .node files from node-pty prebuilds are stored little-endian on disk (cffaedfe = MH_CIGAM_64), so every Darwin prebuild classified as null, and validateStagedBinaries threw a platform mismatch on macOS — breaking npm run check for every macOS contributor. CI didn't catch it because runners are Linux (ELF path was correct) and tests only planted big-endian fake headers. Add recognition for all six Mach-O/Fat byte orderings: - MH_CIGAM (cefaedfe) — LE 32-bit - MH_CIGAM_64 (cffaedfe) — LE 64-bit [the one real prebuilds use] - FAT_CIGAM (bebafeca) — LE universal Update makeFakeNode to write LE CIGAM_64 bytes for the darwin fixture (matching real on-disk format) and add regression tests for all new magic forms.
OutThisLife
left a comment
There was a problem hiding this comment.
Re-reviewed at ede5024 — the classifier fix is correct and complete.
classifyNativeBinary now recognizes the little-endian on-disk forms (MH_CIGAM ce fa ed fe, MH_CIGAM_64 cf fa ed fe, FAT_CIGAM be ba fe ca) alongside the big-endian constants. Re-ran the exact repro that failed before:
node apps/desktop/scripts/stage-native-deps.mjson a Darwin host → exit 0 (was exit 1); stageddarwin-arm64/pty.nodevalidates as a realMach-O 64-bit bundle arm64.classifyNativeBinaryon the real node-pty prebuilds →darwin-arm64 => darwin,darwin-x64 => darwin,win32-* => win32(all previouslynull).- The test fixture now writes the real LE
CIGAM_64bytes instead of the big-endian form, somakeFakeNode('darwin')exercises the true on-disk path, plus dedicated regression tests for both LE forms.stage-native-deps.test.mjs: 15 passed.
Full electron project: 37 files / 408 passed / 1 skipped. Merges cleanly onto current main.
The two remaining items are non-blocking and can be follow-ups if you want:
electron-rebuildinvoked vianode_modules/.bin/electron-rebuild(Windows.binshim isn't node-runnable) — only fires on the no-prebuild rebuild path.validateStagedBinarieschecks platform but not arch.
LGTM — approving. Nice work landing the JS/TS CI gate and the source-regex ban.
What does this PR do?
Wires up JS/TS tests (ui-tui + apps/desktop) into CI with
npm run checkat the workspace root, and fixes all pre-existing test failures across both packages so the CI gate is green from the start. Also bans the "regex-scan source code in tests" antipattern in AGENTS.md and converts all offending tests to real behavior-testing via small extracted pure/DI-testable modules.Related Issue
Fixes #
Type of Change
Changes Made
CI & infrastructure
feat(ci): run JS tests in CI, add npm run check in ws root— adds ajs-tests.ymlworkflow that runsnpm run checkacross all npm workspaces.feat(ci): load npm workspaces from package.json— the CI matrix dynamically discovers workspaces vianpm query .workspaceinstead of a hardcoded list, so new packages are picked up automatically. Addedui-tui/packages/*to the rootworkspacesarray so@hermes/inkgets its own typecheck job.change(desktop): add vitest config for the desktop app— addsvitest.config.ts+vitest.setup.tswithIS_REACT_ACT_ENVIRONMENT=trueand auto-cleanup. Two vitest projects:ui(jsdom,src/**/*.test.{ts,tsx}) andelectron(node,electron/**/*.test.ts).change(ci/desktop): move desktop app build into check job— folds the desktop build into the check script sonpm run checkcovers typecheck + test + test:desktop:all + build end-to-end.cleanup(ci): make all tsbuildinfo gitignored— adds*.tsbuildinfoto.gitignore.AGENTS.md: ban source-regex tests
feat(agent): ban regex-scanning source code in tests— new AGENTS.md antipattern section: tests must notreadFileSynca source file andassert.match/toMatcha regex against it. Prescribes the extraction pattern (pure functions, injected deps, no Electron import) withbackend-probes.ts/backend-command.tsas canonical examples. Carves out a build-artifact exception (readingdist/entry.jsis legit).Desktop: extract source-regex tests into real unit tests
test(desktop): extract hiddenWindowsChildOptions + stopBackendChild— the test file with a literal// TODO FIXME this is an antipatterncomment. ExtractedhiddenWindowsChildOptions()intowindows-child-options.tsandstopBackendChild()intobackend-child.ts, both pure/DI-testable. Consolidated the duplicate definition betweenmain.tsandbootstrap-runner.ts.test(desktop): extract Windows hermes-resolution helpers— extractedbuildPathExtCandidates(),chooseUpdaterArgs(),resolveVenvHermesCommand()intowindows-hermes-path.tswith deps injected (fileExists, canImportHermesCli, etc).test(desktop): extract profile-delete routing decision— extractedprofileNameFromDeleteRequest(),decideProfileDeleteAction(),resolveRouteProfile()intoprofile-delete-routing.ts.test(desktop): fix relative import in oauth-net-request.test.ts— the oauth behavior was already covered by real unit tests; just needed the.tsextension fix and removal of the dead source-regex test.test(desktop): replace windows-child-process.test.ts regex with real tests— removed the tracked source-regex test file entirely; its call-site contracts are now covered by the extracted behavior/DI-level tests above.test(tui): extract cursor-layout + fast-echo helpers— extractedresolveCursorLayout(),fastBackspaceEffect(),fastAppendEffect()fromtextInput.tsxso the cursor-drift regression is tested by calling the pure function with a deliberately stalecurvs freshcurRefCurrent.Desktop: fix broken/stale tests
test(desktop): fix a handful of broken tests—model-options.test.ts(explicit_only/explicitOnlydefaults),model-settings.test.tsx(setApiRequestProfilemock +QueryClientProviderwrapper),panes.test.ts(widthOverridepersisted),pane-shell.test.tsx(resizableprop),onboarding.test.ts(startsWithfor query-param paths),use-prompt-actions/index.test.tsx(source: 'desktop'on recovery retry path),model-menu-panel.test.tsx(document.bodyfor portal content).test(desktop): fix React act() warnings— wrappedrender()/fireEvent()inawait act(async () => { ... })across 8 test files. AddedactRenderhelper in use-prompt-actions. Reduced warnings from 44 → 0.test(desktop): move node tests to vitest as well— migrated the Electronnode --testsuite to vitest'selectronproject so both UI and Electron tests run under a singlevitest runinvocation.fix(desktop): stage-native-deps falls back to electron-rebuild—stageNodePty({ platform, arch })now takes explicit target platform/arch, checksprebuilds/<platform>-<arch>for the correct target, and falls back toelectron-rebuild -f -w node-ptywhen no prebuild or compiled binary exists. Fixes the "Missing node-pty native binary dir for linux-x64" CI failure.Desktop: build cleanup
tsc -bfrom the build script — it was generating 691 orphaned.jsfiles + 1528.d.tsfiles into a gitignoredbuild/electron-types/directory that nothing ever consumed.HERMES_DESKTOP_ALLOW_UNSUPPORTED_PLATFORM_BUILDenv var hack — linux is now natively allowed inensurePlatformBuilds().'hermes'→'Hermes'(capital H, matching electron-builder's rename).TUI: test fixes
fix(js): fix long-time broken tests—statusRule.test.ts(stalecostfield),virtualHeights.test.ts(horizontalReservebump made both prompts collapse to the 20-col floor).tests(tui): longer timeout for wrap ansi test— thecursorDriftRegressiontest is a thorough brute-force O(n²) loop; gave it a longer timeout.Lint & formatting
change(lint): don't ignore config files in eslint conf— removes*.config.*from eslint ignores so config files get linted/formatted; fixes theno-restricted-globalsrule style (single quotes, proper indentation).cleanup(desktop): lint&fmt all— prettier + eslint across all changed files.cleanup(desktop): remove 'use strict' in ts & mjs— ESM modules are strict by default.cleanup(desktop): note that ts imports don't need extension in one comment.fix(desktop): prettier-clean vitest.config.ts— single quotes + trailing newline to satisfy prettier.How to Test
npm run --prefix ui-tui check— builds@hermes/ink, typechecks, runs vitest (107 test files, 1117 tests pass, 1 skipped)npm run --prefix apps/desktop check— typecheck + vitest run (bothuiandelectronprojects, 146 test files / 1180 tests) + desktop bundle validation + build, all on Linuxnpm run --prefix ui-tui/packages/hermes-ink check— nested@hermes/inktypecheck (also covered by the CI matrix)Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests pass — N/A (JS/TS only)Documentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — AGENTS.md updated with new antipattern section