Skip to content

fix(ci): exclude check-test-masking.test.ts fixtures from self-referential tautology gate (#6634) - #6884

Merged
diegosouzapw merged 4 commits into
release/v3.8.47from
fix/6634-testmasking-selfref
Jul 12, 2026
Merged

diegosouzapw merged 4 commits into
release/v3.8.47from
fix/6634-testmasking-selfref

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Closes #6634

Root cause

scripts/check/check-test-masking.mjs's diff-based tautology subcheck (evaluateMasking(), fed by countTautologies()/countExtendedTautologies()) had no exclusion for tests/unit/check-test-masking.test.ts, even though that file legitimately embeds tautology-pattern literals (expect(true).toBe(true);, assert.equal(1,1);) as fixtures exercising countBareTautologies()/scanBareTautologies(). scanBareTautologies() (a sibling absolute-floor subcheck in the same file) already special-cases this exact file for exactly this reason — but the diff-based counters had no equivalent exclusion. Any PR that touches this file's fixtures (e.g. #6603/#6404) permanently self-trips a HARD, blocking "new tautology" flag on the gate's own regression-test file, which is why release/v3.8.47's nightly release-green run reported a false-positive "Test-masking (weakened-assert guard)" HARD failure.

Secondary bug fixed in the same PR: validate-release-green.mjs's firstFailureLine() regex looked for ✖ (U+2716) but check-test-masking.mjs emits ✗ (U+2717) — the regex never matched, so it silently fell back to the last output line (often an unrelated ⚠ warning or even a passing test line) instead of the actual failure line when reporting gate details.

Fix

  • scripts/check/check-test-masking.mjs: evaluateMasking() now skips the assert.ok(true) and extended-tautology diff signals for check-test-masking.test.ts, mirroring the exclusion scanBareTautologies() already applies. The net-assert-reduction and skip-marker signals are unaffected (still checked for this file, since those aren't the fixture false positive).
  • scripts/quality/validate-release-green.mjs: firstFailureLine()'s regex now also matches ✗ (U+2717).

Regression test (TDD, Hard Rule #18)

New file tests/unit/check-test-masking-selfref-6634.test.ts:

  • Reproduces the exact base(origin/main)/head(HEAD) diff of tests/unit/check-test-masking.test.ts through the real exported countTautologies()/countExtendedTautologies()/evaluateMasking() and asserts the gate returns [] (no flags) — this is RED on the unfixed code (returns the 2 real false-positive flags from the nightly run) and GREEN after the fix.
  • A second guard test confirms the exclusion is file-scoped, not global: an unrelated file with a genuinely new tautology still trips both signals.

RED reproduced against the pre-fix check-test-masking.mjs (verbatim, via git show HEAD:... before the fix was applied):

[
  "tests/unit/check-test-masking.test.ts: nova(s) 3 tautologia(s) assert.ok(true)",
  "tests/unit/check-test-masking.test.ts: nova(s) 11 tautologia(s) estendida(s) (expect(true).toBe(true) / assert.equal(1,1))"
]

GREEN after the fix — node --import tsx/esm --test tests/unit/check-test-masking-selfref-6634.test.ts:

✔ #6634: check-test-masking.test.ts's own tautology fixtures must not self-flag as weakening
✔ #6634: unrelated files still get flagged for genuinely new tautologies (guard is file-scoped, not global)
ℹ tests 2, pass 2, fail 0

Existing tests/unit/check-test-masking.test.ts (36 tests) and tests/unit/validate-release-green.test.ts (35 tests) still pass — 71/71 green, no existing assertions weakened.

Scope note

This PR is scoped only to the false-positive gate bug (#6634's headline "Test-masking" HARD failure) and the related firstFailureLine() detail-rendering bug. The issue's other reported item — tests/unit/proxy-dispatcher-family.test.ts: asserts 24 → 21 — is a Phase-5 sync-back gap (main is ahead of this release branch via commit 9cd18bf9a/#6620, a post-merge hotfix not yet synced forward), not a code defect; it is out of scope here and flagged for the release captain's sync-back pass. The 2 ratchet-drift lines (cognitive/cyclomatic complexity) are explicitly non-blocking per the script's own design.

Gates run (all green)

  • node scripts/check/check-file-size.mjs — OK
  • node scripts/check/check-complexity.mjs — OK (2050 violações, baseline 2054)
  • node scripts/check/check-cognitive-complexity.mjs — OK (885 violações, baseline 885)
  • node scripts/check/check-changelog-integrity.mjs — OK
  • npm run typecheck:core — clean
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json <changed files> — 0 errors
  • node --import tsx/esm --test tests/unit/check-test-masking.test.ts tests/unit/validate-release-green.test.ts tests/unit/check-test-masking-selfref-6634.test.ts — 73/73 pass

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request resolves a self-referential false positive in the test-masking gate (issue #6634) by excluding the check-test-masking.test.ts file from diff-based tautology checks, as it legitimately contains tautology fixtures. It also updates validate-release-green.mjs to recognize the ✗ character in failure lines and adds regression tests. The reviewer correctly identified that the top-level comment block in the new test file contains literal tautology patterns that will trigger the absolute floor scan (scanBareTautologies), and provided a helpful suggestion to rephrase the comment to avoid executable-like syntax.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +1 to +19
/**
* Regression test for issue #6634 ("release/v3.8.47 branch not green — nightly
* release-green found HARD failures"). The nightly's "Test-masking
* (weakened-assert guard)" HARD failure was a SELF-REFERENTIAL false positive:
* tests/unit/check-test-masking.test.ts legitimately embeds tautology-pattern
* string literals (e.g. `expect(true).toBe(true);`, `assert.equal(1, 1);`) as
* FIXTURES to exercise countBareTautologies()/scanBareTautologies() — the same
* literal text that the diff-based subcheck (evaluateMasking(), fed by
* countTautologies()/countExtendedTautologies()) treated as "new tautologies in
* the file itself" because those counters are dumb regex scans of raw source
* text, blind to "this is inside a fixture string, not real assertion code".
*
* scanBareTautologies() already special-cases this exact file
* (`if (file.endsWith("check-test-masking.test.ts")) continue;` in
* scripts/check/check-test-masking.mjs) for precisely this reason — this test
* asserts evaluateMasking() now applies the same exclusion for its diff-based
* tautology counters, using the real base(origin/main)/head(HEAD) diff of
* tests/unit/check-test-masking.test.ts.
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The top-level comment block in this new test file contains literal tautology patterns (such as expect(true).toBe(true); and assert.equal(1, 1);). Because scanBareTautologies is a simple regex scan over all tracked test files, it will match these patterns in the comment and cause the absolute floor scan to fail on this file. Please modify the comment to avoid using the exact executable-like syntax of these tautologies (e.g., describe them in plain English or break up the strings).

Suggested change
/**
* Regression test for issue #6634 ("release/v3.8.47 branch not green — nightly
* release-green found HARD failures"). The nightly's "Test-masking
* (weakened-assert guard)" HARD failure was a SELF-REFERENTIAL false positive:
* tests/unit/check-test-masking.test.ts legitimately embeds tautology-pattern
* string literals (e.g. `expect(true).toBe(true);`, `assert.equal(1, 1);`) as
* FIXTURES to exercise countBareTautologies()/scanBareTautologies() — the same
* literal text that the diff-based subcheck (evaluateMasking(), fed by
* countTautologies()/countExtendedTautologies()) treated as "new tautologies in
* the file itself" because those counters are dumb regex scans of raw source
* text, blind to "this is inside a fixture string, not real assertion code".
*
* scanBareTautologies() already special-cases this exact file
* (`if (file.endsWith("check-test-masking.test.ts")) continue;` in
* scripts/check/check-test-masking.mjs) for precisely this reason — this test
* asserts evaluateMasking() now applies the same exclusion for its diff-based
* tautology counters, using the real base(origin/main)/head(HEAD) diff of
* tests/unit/check-test-masking.test.ts.
*/
/**
* Regression test for issue #6634 ("release/v3.8.47 branch not green — nightly
* release-green found HARD failures"). The nightly's "Test-masking
* (weakened-assert guard)" HARD failure was a SELF-REFERENTIAL false positive:
* tests/unit/check-test-masking.test.ts legitimately embeds tautology-pattern
* string literals (e.g. expect-true-toBe-true, assert-equal-1-1) as
* FIXTURES to exercise countBareTautologies()/scanBareTautologies() — the same
* literal text that the diff-based subcheck (evaluateMasking(), fed by
* countTautologies()/countExtendedTautologies()) treated as "new tautologies in
* the file itself" because those counters are dumb regex scans of raw source
* text, blind to "this is inside a fixture string, not real assertion code".
*
* scanBareTautologies() already special-cases this exact file
* (`if (file.endsWith("check-test-masking.test.ts")) continue;` in
* scripts/check/check-test-masking.mjs) for precisely this reason — this test
* asserts evaluateMasking() now applies the same exclusion for its diff-based
* tautology counters, using the real base(origin/main)/head(HEAD) diff of
* tests/unit/check-test-masking.test.ts.
*/

diegosouzapw and others added 2 commits July 11, 2026 19:31
…egression files (#6634)

The #6634 fix added isSelfTestFixtureFile()/scanBareTautologies() exclusions
that only matched check-test-masking.test.ts exactly. Its own new regression
file check-test-masking-selfref-6634.test.ts also embeds tautology-pattern
literals as fixtures/documentation, so the absolute-floor scanBareTautologies
gate self-tripped a HARD failure on the PR's own file. Generalize the
exclusion to the whole check-test-masking* self-test family and lock it with
two regression tests.
@diegosouzapw
diegosouzapw merged commit ec1e79b into release/v3.8.47 Jul 12, 2026
3 checks passed
@diegosouzapw
diegosouzapw deleted the fix/6634-testmasking-selfref branch July 19, 2026 21:00
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…ntial tautology gate (cheaper-inference#6634) (cheaper-inference#6884)

* fix(ci): exclude check-test-masking.test.ts fixtures from self-referential tautology gate (cheaper-inference#6634)

* fix(ci): extend test-masking self-fixture exclusion to sibling gate regression files (cheaper-inference#6634)

The cheaper-inference#6634 fix added isSelfTestFixtureFile()/scanBareTautologies() exclusions
that only matched check-test-masking.test.ts exactly. Its own new regression
file check-test-masking-selfref-6634.test.ts also embeds tautology-pattern
literals as fixtures/documentation, so the absolute-floor scanBareTautologies
gate self-tripped a HARD failure on the PR's own file. Generalize the
exclusion to the whole check-test-masking* self-test family and lock it with
two regression tests.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ntial tautology gate (cheaper-inference#6634) (cheaper-inference#6884)

* fix(ci): exclude check-test-masking.test.ts fixtures from self-referential tautology gate (cheaper-inference#6634)

* fix(ci): extend test-masking self-fixture exclusion to sibling gate regression files (cheaper-inference#6634)

The cheaper-inference#6634 fix added isSelfTestFixtureFile()/scanBareTautologies() exclusions
that only matched check-test-masking.test.ts exactly. Its own new regression
file check-test-masking-selfref-6634.test.ts also embeds tautology-pattern
literals as fixtures/documentation, so the absolute-floor scanBareTautologies
gate self-tripped a HARD failure on the PR's own file. Generalize the
exclusion to the whole check-test-masking* self-test family and lock it with
two regression tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant