Skip to content

test: clean up temporary files after runs - #6690

Merged
cv merged 3 commits into
mainfrom
codex/test-temp-cleanup
Jul 11, 2026
Merged

test: clean up temporary files after runs#6690
cv merged 3 commits into
mainfrom
codex/test-temp-cleanup

Conversation

@cv

@cv cv commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Vitest runs now use one private, run-scoped temp root that is removed automatically after the run, including the process-exit fallback path. Test fixtures that previously bypassed the configured temp directory now use owned, unique paths and explicit cleanup. Set NEMOCLAW_TEST_KEEP_TEMP=1 to retain the run root for debugging.

Changes

  • Add shared Vitest global setup for the root projects and standalone plugin suite.
  • Redirect TMPDIR, TMP, and TEMP into the run root, then restore the caller environment during teardown.
  • Clean installer, service, alias, advisor, and runtime-shell fixtures that previously escaped or outlived their tests.
  • Add regression coverage for recursive cleanup, caller-environment restoration, keep-temp behavior, and config wiring.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: test infrastructure and fixtures only; no user-facing CLI, configuration, API, policy, or sandbox behavior changed
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification:
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 124 integration tests, 35 CLI tests, 1 focused installer test, and 9 standalone plugin tests passed; keep-temp mode passed 4/4; npm run test:projects:check passed for 1,483 files
  • Applicable broad gate passed — npm test exercised 1,406 files; two aggregate-only failures passed in exact CI-mode isolation (10/10 and 64/64), so PR CI is the authoritative broad gate
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • Tests
    • Improved isolation by using dynamically created, OS-appropriate temp directories/files instead of fixed /tmp paths.
    • Added a shared temp-root initializer for tests, with robust teardown, environment restoration, and optional temp retention.
    • Updated e2e/installer/checkout/service/runtime tests to use dynamic target and sandbox directories and to ensure staged files and temp artifacts are cleaned up.
  • Tooling
    • Configured the test runner to perform global temp initialization before executing tests.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv self-assigned this Jul 11, 2026
@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Vitest now initializes an isolated temporary root for test runs. Test harnesses replace fixed temporary paths with dynamically generated directories, add cleanup assertions, and isolate installer, workflow checkout, sandbox, service, and temporary-home state.

Changes

Test isolation and portability

Layer / File(s) Summary
Vitest temporary-root setup
test/helpers/vitest-temp-root.ts, test/vitest-temp-root.test.ts, vitest.config.ts, nemoclaw/vitest.config.ts
Adds global temporary-directory setup with environment restoration, cleanup, optional artifact retention, and configuration assertions.
Installer staging and cleanup
test/install-preflight.test.ts, test/install-stage-from-stdin.test.ts
Stages installer payloads through temporary files and verifies staged files and fixture directories are removed.
Dynamic workflow checkout targets
test/e2e-advisor.test.ts, test/pr-review-advisor-workflow-boundary.test.ts
Substitutes per-test checkout directories into workflow scripts and updates git and environment assertions accordingly.
Portable temporary fixtures and sandbox state
src/lib/tunnel/services.test.ts, test/nemo-deepagents-alias.test.ts, test/nemohermes-alias.test.ts, test/runtime-shell.test.ts, test/service-env.test.ts
Replaces fixed temporary paths with OS-derived or per-run paths and isolates service, sandbox-init, nameserver, home, and tunnel test state.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested labels: chore, area: ci

Suggested reviewers: ericksoa, jyaunches

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 9.09% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: test infrastructure now cleans up temporary files after runs.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/test-temp-cleanup

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the codex/test-temp-clea... branch remains at 79%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.
File main 1e82a06 codex/test-temp-clea... 7dbcbff +/-
src/lib/name-validation.ts 100% 94% -6%
src/lib/runner.ts 75% 74% -1%
src/lib/adapter...tp/curl-args.ts 97% 96% -1%
src/lib/adapters/http/probe.ts 88% 91% +3%
src/lib/messagi...l-validation.ts 70% 74% +4%
src/lib/messagi...ridge-health.ts 91% 96% +5%
src/lib/messagi...way-conflict.ts 81% 90% +9%
src/lib/messagi...ink-base-url.ts 90% 100% +10%
src/lib/messagi...ridge-health.ts 68% 79% +11%
src/lib/adapter...ocker/volume.ts 70% 85% +15%

Updated July 11, 2026 19:07 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • None. No E2E is required. The deterministic risk plan lists no required jobs, and the PR is limited to test files plus Vitest test-runner configuration/helper changes for temporary directory isolation and fixture cleanup. These changes should be covered by the normal unit/integration test CI lanes rather than live E2E jobs.

Optional E2E

  • None.

New E2E recommendations

  • None.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
test/e2e-advisor.test.ts (1)

78-83: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Avoid locking this test to the placeholder implementation.

gitCalls and githubEnv already prove that the per-run directory reaches the workflow boundary, so expect(workflowScript).toContain(E2E_ADVISOR_TARGET_DIR) unnecessarily couples the test to the workflow’s literal placeholder. Remove this assertion and retain the behavioral checks. As per path instructions, tests should prioritize behavioral confidence over implementation lock-in.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e-advisor.test.ts` around lines 78 - 83, Remove the
expect(workflowScript).toContain(E2E_ADVISOR_TARGET_DIR) assertion from the
test, while retaining the gitCalls and githubEnv behavioral checks that verify
the per-run directory reaches the workflow boundary.

Source: Path instructions

test/pr-review-advisor-workflow-boundary.test.ts (1)

51-56: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Avoid locking this test to the placeholder implementation.

gitCalls and githubEnv already prove that the per-run directory reaches the workflow boundary, so expect(workflowScript).toContain(PR_REVIEW_ADVISOR_TARGET_DIR) unnecessarily couples the test to the workflow’s literal placeholder. Remove this assertion and retain the behavioral checks. As per path instructions, tests should prioritize behavioral confidence over implementation lock-in.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/pr-review-advisor-workflow-boundary.test.ts` around lines 51 - 56,
Remove the implementation-specific
expect(workflowScript).toContain(PR_REVIEW_ADVISOR_TARGET_DIR) assertion from
the boundary test, while retaining the gitCalls and githubEnv behavioral checks
that verify the per-run directory reaches the workflow boundary.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/install-stage-from-stdin.test.ts`:
- Around line 26-31: The test helper runEntryGuardInFixture duplicates the
installer’s entry-guard logic and bypasses its EXIT-trap cleanup. Replace this
copied harness with a contract test that executes scripts/install.sh using only
a controlled curl/input setup, then assert the real installer removes the staged
path during EXIT cleanup; alternatively extract and invoke one shared
entry-guard implementation so the test cannot drift from production.

In `@test/vitest-temp-root.test.ts`:
- Around line 27-35: Remove the added if statements from restoreTempEnv and the
related finally cleanup paths in this test file while preserving identical
environment restoration and cleanup behavior. Refactor the branching through an
approved shared helper or equivalent expression-based logic, including all
locations noted by the review, so the codebase-growth guard no longer reports
these five if statements.

---

Nitpick comments:
In `@test/e2e-advisor.test.ts`:
- Around line 78-83: Remove the
expect(workflowScript).toContain(E2E_ADVISOR_TARGET_DIR) assertion from the
test, while retaining the gitCalls and githubEnv behavioral checks that verify
the per-run directory reaches the workflow boundary.

In `@test/pr-review-advisor-workflow-boundary.test.ts`:
- Around line 51-56: Remove the implementation-specific
expect(workflowScript).toContain(PR_REVIEW_ADVISOR_TARGET_DIR) assertion from
the boundary test, while retaining the gitCalls and githubEnv behavioral checks
that verify the per-run directory reaches the workflow boundary.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: fd829377-bf80-41bb-a3f8-6ac616b59c9c

📥 Commits

Reviewing files that changed from the base of the PR and between 235afc6 and 48889db.

📒 Files selected for processing (13)
  • nemoclaw/vitest.config.ts
  • src/lib/tunnel/services.test.ts
  • test/e2e-advisor.test.ts
  • test/helpers/vitest-temp-root.ts
  • test/install-preflight.test.ts
  • test/install-stage-from-stdin.test.ts
  • test/nemo-deepagents-alias.test.ts
  • test/nemohermes-alias.test.ts
  • test/pr-review-advisor-workflow-boundary.test.ts
  • test/runtime-shell.test.ts
  • test/service-env.test.ts
  • test/vitest-temp-root.test.ts
  • vitest.config.ts

Comment thread test/install-stage-from-stdin.test.ts
Comment thread test/vitest-temp-root.test.ts
@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: No advisor follow-up required beyond maintainer review.
Findings: 0 required · 0 warnings · 0 optional suggestions
Since last review: 1 prior item resolved · 0 still apply · 0 new items found

Workflow run details

This is an automated review. Required findings need action before merge. Warnings and optional suggestions do not require a response or follow-up. A human maintainer makes the final merge decision.

cv added 2 commits July 11, 2026 11:54
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/install-preflight.test.ts`:
- Around line 401-403: Update the stagedFixturePath construction in the test
setup to create the fixture under the existing per-test temporary directory
represented by tmp, rather than hardcoding /tmp. Preserve the unique filename
and exclusive write behavior in the writeFileSync call so run-scoped cleanup
includes the staged fixture.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5a3a6633-de48-42ee-89c1-d220650a0963

📥 Commits

Reviewing files that changed from the base of the PR and between 5dfa108 and 7dbcbff.

📒 Files selected for processing (2)
  • test/install-preflight.test.ts
  • test/vitest-temp-root.test.ts

Comment thread test/install-preflight.test.ts
@cv
cv merged commit 24f5489 into main Jul 11, 2026
55 checks passed
@cv
cv deleted the codex/test-temp-cleanup branch July 11, 2026 19:11
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Vitest runs now use one private, run-scoped temp root that is removed
automatically after the run, including the process-exit fallback path.
Test fixtures that previously bypassed the configured temp directory now
use owned, unique paths and explicit cleanup. Set
`NEMOCLAW_TEST_KEEP_TEMP=1` to retain the run root for debugging.

## Changes

- Add shared Vitest global setup for the root projects and standalone
plugin suite.
- Redirect `TMPDIR`, `TMP`, and `TEMP` into the run root, then restore
the caller environment during teardown.
- Clean installer, service, alias, advisor, and runtime-shell fixtures
that previously escaped or outlived their tests.
- Add regression coverage for recursive cleanup, caller-environment
restoration, keep-temp behavior, and config wiring.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: test infrastructure and
fixtures only; no user-facing CLI, configuration, API, policy, or
sandbox behavior changed
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — 124 integration tests, 35 CLI tests, 1
focused installer test, and 9 standalone plugin tests passed; keep-temp
mode passed 4/4; `npm run test:projects:check` passed for 1,483 files
- [ ] Applicable broad gate passed — `npm test` exercised 1,406 files;
two aggregate-only failures passed in exact CI-mode isolation (10/10 and
64/64), so PR CI is the authoritative broad gate
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: Carlos Villela <cvillela@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Improved isolation by using dynamically created, OS-appropriate temp
directories/files instead of fixed `/tmp` paths.
* Added a shared temp-root initializer for tests, with robust teardown,
environment restoration, and optional temp retention.
* Updated e2e/installer/checkout/service/runtime tests to use dynamic
target and sandbox directories and to ensure staged files and temp
artifacts are cleaned up.
* **Tooling**
* Configured the test runner to perform global temp initialization
before executing tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
cv pushed a commit that referenced this pull request Jul 22, 2026
## Summary

- raise the Hermes Python guard test-harness allowance from 5 seconds to
90 seconds
- apply the shared allowance to the three runtime-config-guard helper
paths only
- correct the WSL root-only peer fixture to start from the production
mutable topology, `sandbox:sandbox 03770`
- assert sealing produces `root:sandbox 03770`, permits sandbox-group
runtime-state creation, rejects unlink of root-owned sealed config, and
restores the original ownership
- let the existing config-reclaim peer write probe traverse Vitest's
private temp root only for the duration of that probe, then restore its
exact mode
- leave production runtime behavior and non-guard command limits
unchanged

## Evidence

- main CI job 88920894839 returned `status: null` after the first guard
child exceeded its 5-second `spawnSync` limit; the runner and later
guard cases remained healthy
- four prior main runs passed the same case in about 0.3 seconds,
confirming a load-sensitive harness limit rather than a Hermes behavior
regression
- current-main Platform Vitest run 29919416442 passed the complete WSL
suite (1,642 files / 19,227 tests) and then reproduced the isolated
root-fixture failure
- historical run 29307612216 reproduced that fixture failure before the
OpenClaw upgrade and before #7379
- first exact-head WSL proof 29923652396 passed the complete WSL suite
and both corrected Hermes root contracts; it then exposed the next
root-only fixture defect
- that config-reclaim test completed production normalization and all
ownership/mode assertions; only its stepped-down write probe failed
because PR #6690 made Vitest's shared temp ancestor private
- current exact head `b7414d2f7f1bfd68028e9f43c11c1cc61033e266` includes
current `main` at `f9924949922f8e554f94aeefdd23a993a801e7b4`
- current exact-head platform proof:
https://github.com/NVIDIA/NemoClaw/actions/runs/29929199550
- focused `nemoclaw-start-perms` suite: 16 passed, 3 Linux-root
capability skips on macOS
- `npm run build:cli` and `npm run check:diff`: passed
- independent review of the narrow changes: no production findings

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

---------

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@wscurran wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance labels Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants