Skip to content

fix(dev-setup): name the Node.js heap limit when a type check exhausts it - #8865

Merged
cv merged 8 commits into
NVIDIA:mainfrom
VishnuR23:fix/report-heap-exhaustion-in-doctor
Aug 13, 2026
Merged

fix(dev-setup): name the Node.js heap limit when a type check exhausts it#8865
cv merged 8 commits into
NVIDIA:mainfrom
VishnuR23:fix/report-heap-exhaustion-in-doctor

Conversation

@VishnuR23

@VishnuR23 VishnuR23 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

The contributor doctor reported every failed type check as CLI type check: failed with Next: Run: npm run typecheck:cli. When Node.js ran out of heap, that message pointed at the checked sources rather than the host. check_quiet_command now reports the memory cause with the existing NODE_OPTIONS remedy.

The classifier drains the complete compiler stream and retains only a bounded heap or other result plus process statuses. Compiler output never enters the human-readable or JSON report, and large output cannot trigger an early-close pipe failure or grow a shell variable without a bound.

Related Issue

Refs #8688. It does not close the issue: maintainers still need to decide whether setup should set a default heap size or whether the TypeScript configuration should change. This PR does not make that product or configuration choice and does not change any heap limit.

Changes

  • scripts/dev-setup.sh: stream and classify discarded command output while preserving the command and matcher statuses; retain the existing heap and generic remediations.
  • test/dev-setup-doctor.test.ts: cover heap, generic, and more-than-128-KiB discarded-output behavior, including report redaction.

Verification

  • npx vitest run --project integration test/dev-setup-doctor.test.ts — 41 passed
  • applicable prek hooks — passed, including ShellCheck and secret scanning
  • npm run test:titles:check — passed
  • npm run test-size:check — passed
  • npm run source-shape:check — passed
  • bash -n scripts/dev-setup.sh — passed
  • git diff --check — passed
  • CLI and plugin builds plus the required pre-push type-check — passed

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: The public command, remediation, and defaults are unchanged. The troubleshooting guidance already documents the cause and remedy.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification:
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: Independent review: PASS — the current-main merge preserves the reviewed bounded and redacted Node.js heap diagnostic and its existing remediation; the staging text names the exact libssh2 mismatch; contributor documentation remains accurate.
  • Agent: Codex Desktop documentation writer

Security Review

The change adds no input interpolation, output exposure, credential handling, dependency, or privilege boundary. The matcher program is static, drains all input, and handles a matcher failure conservatively as a generic check failure. The regression confirms that discarded compiler output remains redacted.

Verification Checklist

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, including ShellCheck
  • Targeted behavior tests pass — command/result: 41 passed
  • Applicable broad gate passed — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

The owning integration suite is the proportionate local gate for this bounded contributor-doctor change. Fresh repository checks and human re-review are still required.


Signed-off-by: Vishnu Rajeev 19866703+VishnuR23@users.noreply.github.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved developer setup diagnostics for TypeScript check failures.
    • Added specific NODE_OPTIONS guidance when Node.js runs out of memory.
    • Suppressed excessive compiler output while preserving actionable failure information.
    • Continued displaying generic remediation guidance for unrelated type-check failures.
  • Maintenance

    • Updated the staging QA image source to use the correct candidate build.

…s it

The doctor discarded command output and reported every type-check failure
as 'failed' with a build remediation. A contributor on a small host saw
that message when Node.js had run out of heap, which points at the
sources rather than the host.

The check now classifies that failure and gives the NODE_OPTIONS remedy.
It matches the captured output without printing it, because a failing
command can name paths the report should not carry.

Refs NVIDIA#8688
@copy-pr-bot

copy-pr-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 472df08b-ee0c-4d8b-9396-8b5b032582e8

📥 Commits

Reviewing files that changed from the base of the PR and between 6c92d16 and 3d08ea1.

📒 Files selected for processing (2)
  • .github/workflows/managed-images.yaml
  • test/managed-image-publication-workflow.test.ts

📝 Walkthrough

Walkthrough

The setup doctor now suppresses command output, detects Node.js heap exhaustion, and reports targeted NODE_OPTIONS remediation. The staging QA workflow and its test fixture now use an updated source commit.

Changes

Setup doctor diagnostics

Layer / File(s) Summary
Heap exhaustion detection and reporting
scripts/dev-setup.sh
check_quiet_command classifies captured output and reports heap-specific remediation with --max-old-space-size=5120. Other failures retain the generic report.
Diagnostic behavior tests
test/dev-setup-doctor.test.ts
Tests verify heap-specific reporting, suppression of large compiler output, and generic remediation for non-heap TypeScript failures.

Staging QA source update

Layer / File(s) Summary
Staging source reference and validation
.github/workflows/managed-images.yaml, test/managed-image-publication-workflow.test.ts
The workflow records the updated staging QA source commit and annotation. The publication test expects the same commit.

Estimated code review effort: 2 (Simple) | ~10 minutes

Mergeability Score: ⚪ Minimal · up to 3d08e

This localized change improves heap-exhaustion reporting without changing heap limits, defaults, or public remediation behavior. No actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: cv, prekshivyas

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: reporting the Node.js heap limit when a type check exhausts it.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
scripts/dev-setup.sh (1)

240-240: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Replace the early-closing grep -q pipeline with a shell-native substring check.

scripts/dev-setup.sh enables pipefail. Large output causes printf to receive SIGPIPE, so the heap-remediation branch is skipped. The existing test covers the message but not large output. Add a large-output regression case.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/dev-setup.sh` at line 240, In the error-handling branch around the
JavaScript heap out-of-memory check, replace the pipefail-unsafe printf/grep -q
pipeline with a shell-native substring check against output, preserving the
existing message match and remediation behavior. Extend the relevant regression
coverage with a large-output case that confirms this branch still executes
without being skipped by SIGPIPE.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@scripts/dev-setup.sh`:
- Line 240: In the error-handling branch around the JavaScript heap
out-of-memory check, replace the pipefail-unsafe printf/grep -q pipeline with a
shell-native substring check against output, preserving the existing message
match and remediation behavior. Extend the relevant regression coverage with a
large-output case that confirms this branch still executes without being skipped
by SIGPIPE.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 40ab120b-d07b-4d34-97fd-998d4447fa55

📥 Commits

Reviewing files that changed from the base of the PR and between 5aaf22b and 26b99f0.

📒 Files selected for processing (2)
  • scripts/dev-setup.sh
  • test/dev-setup-doctor.test.ts

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · medium confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized terminology decisions differ; normalized E2E selections differ; severity counts match.
1 terminology difference from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

  • heap at scripts/dev-setup.sh:241: selected only by the second-opinion lane as justified.
5 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • ubuntu-repo-cloud-langchain-deepagents-code: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • ubuntu-repo-cloud-openclaw: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • onboard-managed-image-buildless-e2e: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • managed-image-protected-runtime: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • staging-brev-launchable: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

1 semantic terminology decision

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • justified — Node.js heap at scripts/dev-setup.sh:254: Retain “Node.js heap” for the heap-exhaustion diagnostic.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite for the commit under review.

Recommended E2E: None

Manual-only E2E: cloud-onboard, managed-image-multiarch-startup
The manual PR workflow does not run these selectors for the commit under review. Run them from reviewed code on main.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

cv
cv previously requested changes Aug 12, 2026

@cv cv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scripts/dev-setup.sh:238-240: check_quiet_command stores all command output in a shell variable, then uses printf | grep -q under pipefail. When a heap marker is followed by enough output, grep exits early and printf exits 141, so the function selects the generic type-check remediation. The unbounded variable can also consume the doctor's memory. This can hide the heap remedy or terminate the doctor, which defeats the accepted diagnostic behavior. Use a bounded or streaming matcher that drains discarded output, preserves the checked command's exit status, and avoids an early-close pipeline. Add a regression at test/dev-setup-doctor.test.ts:343 with a heap marker followed by large output; assert the heap remediation and output redaction. Regression evidence: the exact conditional selected the heap branch with 64 KiB after the marker but selected the generic branch with 128 KiB and 1 MiB.

@wscurran wscurran added area: cli Command line interface, flags, terminal UX, or output area: install Install, setup, prerequisites, or uninstall flow bug-fix PR fixes a bug or regression labels Aug 12, 2026
@apurvvkumaria apurvvkumaria self-assigned this Aug 12, 2026
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria

Copy link
Copy Markdown
Collaborator

Addressed the large-output review finding in the latest PR commit.

check_quiet_command now drains the complete compiler stream through a fixed streaming matcher and retains only a bounded heap/other classification plus the command and matcher statuses. This removes the unbounded shell variable and avoids an early-close pipe failure while preserving the original command status and keeping compiler output out of both human-readable and JSON reports.

The regression emits more than 128 KiB after the heap marker and verifies that the heap remediation still appears while the discarded marker remains redacted. Validation completed:

  • npx vitest run --project integration test/dev-setup-doctor.test.ts — 41 passed
  • applicable prek hooks, including ShellCheck and secret scanning — passed
  • test-title, test-size, source-shape, syntax, and whitespace checks — passed
  • CLI and plugin builds plus the required pre-push type-check — passed

The documentation writer found no documentation change necessary because the public command, remediation, and defaults are unchanged. The security assessment found no new input interpolation, output exposure, credential handling, or privilege boundary; the matcher program is static and a matcher failure is handled conservatively as a generic check failure.

Fresh repository checks and human re-review are still required.

@cv
cv dismissed their stale review August 12, 2026 17:39

The latest PR commit drains command output through a bounded matcher and adds a large-output regression. Dismissing this resolved review before a full current-commit review.

@VishnuR23

Copy link
Copy Markdown
Contributor Author

Thanks both. @cv's finding was correct and the bug was mine.

My version piped a captured variable into grep -q. Under pipefail, grep exits as soon as it matches, printf then takes SIGPIPE and exits 141, and the pipeline reports non-zero, so the elif fell through to the generic remediation. The heap remedy disappeared exactly when the compiler produced the most output, which is when a contributor most needs it. The 64 KiB versus 128 KiB evidence in the review pins that precisely.

I verified @apurvvkumaria's replacement rather than taking it on trust:

  • npx vitest run --project integration test/dev-setup-doctor.test.ts reports 41 passed on my checkout.
  • Running the new awk matcher directly with 1 MB of output after the marker returns heap. My previous conditional returned other for that input.

The streaming matcher is the right shape. It drains the whole stream so nothing closes the pipe early, keeps only a bounded classification word instead of the full output, preserves both PIPESTATUS values so a matcher failure degrades to the generic check rather than a false pass, and still keeps compiler output out of the report.

I have no further changes to make here. Ready for re-review when convenient.

@apurvvkumaria

apurvvkumaria commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Readiness update:

  • The current revision includes the bounded streaming fix for the prior large-output review finding. The contributor commit, remediation commit, and current revision are all GitHub Verified; DCO, commit lint, documentation review, static analysis, build/type checks, security scanning, installer integrity, and both automated review lanes pass.
  • The focused doctor suite previously passed all 41 tests, including more than 128 KiB of discarded output after the heap marker. No unresolved review threads or automated-review findings remain.
  • The installer lane failed in an unchanged dual-station qualification test, which returned single-station instead of ready. The changed files do not touch station pairing.
  • Test shard 8 failed in the unchanged sandbox auto-pair approval suite, which returned approve-failed instead of approved-one. The changed files do not touch sandbox approval.
  • The staging image lane rebuilt its base successfully, then failed a security-package inventory assertion while layering the PR. All three agent images built and started directly. This is shared image-fixture drift, not behavior introduced by this two-file diagnostic change.
  • The workflows are already on attempt two, so I did not rerun them again.
  • The change is 78 additions and 1 deletion; it is not a large-LOC increase.

Remaining blockers are successful infrastructure checks and maintainer re-review. No further branch change is indicated by the current failures.

@prekshivyas prekshivyas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved based on code review. Required CI checks remain merge gates.

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@cv
cv merged commit 1c78464 into NVIDIA:main Aug 13, 2026
66 of 71 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Command line interface, flags, terminal UX, or output area: install Install, setup, prerequisites, or uninstall flow bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants