Skip to content

fix(ci): bound the forgotten-sibling report so an advisory step stops failing the job - #13889

Merged
diegosouzapw merged 1 commit into
release/v3.8.51from
fix/release-v3.8.51-basereds-sibling-report
Sep 16, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.51from
fix/release-v3.8.51-basereds-sibling-report

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

"Fast Quality Gates" is currently red on every open PR against release/v3.8.51. The failing
step is forgotten-sibling-tests — which is explicitly advisory; its own output says
"Report-only calibration: these findings do not fail the job." — yet it exits 1.

Why

When a diff touches a hub module (open-sse/config/providerRegistry.ts in the current reds), the
analysis walks every import edge in the repo and multiplies each consumer by its candidate tests.
The cross-product reaches millions of rows, and lines.join("\n") then exceeds V8's maximum string
length. The throw lands in main()'s catch, which exits 1 — so an advisory report takes the whole
job down, on PRs that have nothing to do with it.

Measured with a synthetic hub cross-product against the current script:

findings result
3 000 000 a 435 MB report string (no throw, but absurd)
4 500 000 Invalid string length — the CI failure, verbatim

With this change the same 4 500 000 findings render as 27 KB.

What changed

Only the enumeration is bounded — never the signal:

  • the header keeps the exact totals (Unallowlisted findings: 4500000 in the probe above);
  • at most 200 rows per section are listed, followed by a line naming how many were withheld;
  • the JSON artifact gets the same treatment (5 000 items per array) plus an explicit totals
    object — JSON.stringify would throw on the same input for the same reason;
  • markdown() is exported so the bound is testable without a CI-sized diff.

No change to the analysis, the allowlist, or what counts as a finding.

Note

This is the second base-red in a row from the same family: #13875 fixed the env/docs contract, and
this one clears the remaining always-red gate. Both are infrastructure, not product defects.

… failing the job

"Fast Quality Gates" is red on every open PR against release/v3.8.51. The failing
step is `forgotten-sibling-tests`, which is explicitly advisory — its own output
says "Report-only calibration: these findings do not fail the job" — yet it exits 1.

When a PR diff touches a hub module (`open-sse/config/providerRegistry.ts` in the
current reds), the analysis walks every import edge in the repo and multiplies each
consumer by its candidate tests. The result reaches millions of rows, and
`lines.join("\n")` then exceeds V8's maximum string length. The throw lands in
main()'s catch, which exits 1 — so an advisory report takes the whole job down.

Measured with a synthetic hub cross-product, before the change:

  3,000,000 findings -> a 435 MB report string (no throw, but absurd)
  4,500,000 findings -> Invalid string length   (the CI failure, verbatim)

After: the same 4,500,000 findings render as 27 KB.

The fix bounds only the ENUMERATION. The header keeps the exact totals, so the
signal ("this diff has N unreviewed sibling tests") is unchanged; at most 200 rows
per section are listed, followed by a line naming how many were withheld. The JSON
artifact gets the same treatment (5,000 items per array) plus an explicit `totals`
object, since `JSON.stringify` would throw on the same input for the same reason.

`markdown()` is exported so the bound is testable without a CI-sized diff.
@diegosouzapw
diegosouzapw merged commit 1cb4c82 into release/v3.8.51 Sep 16, 2026
6 of 10 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
… failing the job (diegosouzapw#13889)

"Fast Quality Gates" is red on every open PR against release/v3.8.51. The failing
step is `forgotten-sibling-tests`, which is explicitly advisory — its own output
says "Report-only calibration: these findings do not fail the job" — yet it exits 1.

When a PR diff touches a hub module (`open-sse/config/providerRegistry.ts` in the
current reds), the analysis walks every import edge in the repo and multiplies each
consumer by its candidate tests. The result reaches millions of rows, and
`lines.join("\n")` then exceeds V8's maximum string length. The throw lands in
main()'s catch, which exits 1 — so an advisory report takes the whole job down.

Measured with a synthetic hub cross-product, before the change:

  3,000,000 findings -> a 435 MB report string (no throw, but absurd)
  4,500,000 findings -> Invalid string length   (the CI failure, verbatim)

After: the same 4,500,000 findings render as 27 KB.

The fix bounds only the ENUMERATION. The header keeps the exact totals, so the
signal ("this diff has N unreviewed sibling tests") is unchanged; at most 200 rows
per section are listed, followed by a line naming how many were withheld. The JSON
artifact gets the same treatment (5,000 items per array) plus an explicit `totals`
object, since `JSON.stringify` would throw on the same input for the same reason.

`markdown()` is exported so the bound is testable without a CI-sized diff.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant