Skip to content

test: remove duplicate behavior coverage - #9501

Merged
prekshivyas merged 2 commits into
NVIDIA:mainfrom
rsliter:codex/remove-duplicate-test-coverage
Aug 18, 2026
Merged

test: remove duplicate behavior coverage#9501
prekshivyas merged 2 commits into
NVIDIA:mainfrom
rsliter:codex/remove-duplicate-test-coverage

Conversation

@rsliter

@rsliter rsliter commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Remove 60 duplicated test declarations across 18 audited clusters, plus one orphaned assertion that exercised only a test-local BuildKit helper. Focused owner files now hold each behavior contract, and the unique vLLM size and model assertions move to the focused profile test.

Related Issue

Closes #9497

Changes

  • Remove redundant coverage from broad snapshot, rebuild, onboarding, inference, messaging, entrypoint, and E2E support suites.
  • Delete three test files whose coverage is fully owned elsewhere or did not exercise product code.
  • Preserve seven unique vLLM assertions in test/detect-vllm-profile.test.ts.
  • Lower the legacy test/nemoclaw-start.test.ts size ceiling to its new line count.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior, justification:
  • Tests not applicable, justification: No product behavior changes. This PR changes test ownership and retains the focused behavior assertions.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded, reviewer/approval link/justification: The test-only diff was audited against each focused canonical owner. No production path or security assertion was removed, and the focused credential, sandbox, inference, and messaging suites pass.
  • Non-success, skipped, or missing CI check accepted by maintainer, check name, approval link, and follow-up issue:

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit: Not applicable
  • Station profile/scenario: Not applicable
  • Result: Not applicable
  • Supporting evidence: Not applicable

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above, command/result or justification: 457 focused CLI tests passed; 120 focused integration tests passed with 1 existing skip; 159 focused E2E support tests passed across host-compatible runs; 32 growth-guardrail tests passed; the reviewed profile fix passed 21 of 21 tests; CLI and plugin type-checking passed.
  • Applicable broad gate passed, command/result: npm test was attempted and reached its 600-second local ceiling with unrelated macOS, service, network, and cross-suite failures. GitHub CI will provide the clean aggregate result.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Documentation Review

  • Documentation writer subagent reviewed the completed implementation
  • Result: no-docs-needed
  • Evidence: Branch commit ad44f19a0 changes only tests and ci/test-file-size-budget.json. The follow-up commit restores the Spark profile environment-value assertion and changes no user-facing product behavior or documentation contract.
  • Agent: Codex Desktop
  • PR: test: remove duplicate behavior coverage #9501

Signed-off-by: Rebecca Sliter 571084+rsliter@users.noreply.github.com

Summary by CodeRabbit

  • Tests

    • Streamlined automated coverage by removing obsolete or narrowly scoped scenarios across sandbox recovery, snapshots, inference, onboarding, security, image layouts, messaging, and service configuration.
    • Strengthened vLLM profile checks with exact model identifiers and image-size expectations.
    • Removed unused test fixtures and imports.
    • Updated the legacy test-file size budget to reflect the reduced test suite.
  • Chores

    • No production behavior or public interfaces changed.

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d84a6c94-f867-445d-b861-86f7fde92074

📥 Commits

Reviewing files that changed from the base of the PR and between 751f66f and ad44f19.

📒 Files selected for processing (1)
  • test/detect-vllm-profile.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/detect-vllm-profile.test.ts

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.


📝 Walkthrough

Walkthrough

The PR removes duplicate tests and unused test-local helpers from broad suites. Focused suite coverage remains. Selected vLLM profile assertions are more exact. The line budget for test/nemoclaw-start.test.ts is reduced.

Changes

Test coverage consolidation

Layer / File(s) Summary
Sandbox recovery and snapshot coverage
src/lib/actions/sandbox/*
Removed duplicate recovery and snapshot cases for restoration, gateway conflicts, observability, cloning, and OTLP reconciliation.
Inference and installation coverage
src/lib/inference/vllm.test.ts, src/lib/onboard/docker-driver-gateway-env.test.ts, src/lib/skill-install.test.ts, test/detect-vllm-profile.test.ts
Removed duplicate vLLM, gateway environment, and shell quoting tests. Added exact model, environment, and image-size assertions to the focused vLLM profile tests.
E2E and probe coverage
test/e2e/support/*, test/credential-exposure.test.ts, test/hosted-inference.test.ts, test/wsl2-probe-timeout.test.ts, test/hermes-inference-switch-command-shape.test.ts, test/nemoclaw-start-gateway-health.test.ts
Removed duplicate tests for phase labels, trace sanitization, endpoint validation, probe timing, Hermes routing, and gateway marker idempotence.
Platform and integration coverage
test/build-base-image-platform-action.test.ts, test/hermes-final-image-layout.test.ts, test/service-env.test.ts, test/nemoclaw-start.test.ts
Removed redundant platform, BuildKit parser, OpenShell resolution, and environment-wrapper tests.
Test size budget
ci/test-file-size-budget.json
Reduced the maximum line count for test/nemoclaw-start.test.ts from 4761 to 4671.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to ad44f

This change removes duplicate test coverage while retaining focused behavior assertions and does not alter product behavior; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the primary change: removing duplicate test coverage.
Linked Issues check ✅ Passed The changes remove the listed redundant tests, preserve unique vLLM assertions, and avoid production or documentation changes required by issue #9497.
Out of Scope Changes check ✅ Passed All changes are limited to test cleanup and the related test-file size budget, which are within issue #9497 scope.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@jyaunches jyaunches left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOC Reduction / Codebase Simplicity Review

Why this blocks

The consolidation is substantially net-negative and the focused owners retain the other audited contracts, but one unique Spark assertion was deleted rather than moved.

The removed case at src/lib/inference/vllm.test.ts:296-308 was the only test that tied detectVllmProfile({ platform: "spark" }) to defaultModel.envValue === "qwen3.6-35b-a3b-nvfp4". The retained focused case at test/detect-vllm-profile.test.ts:13-23 now covers the Spark name, model ID, image, and image sizes, but not that environment value. Other occurrences of the slug exercise model catalog or resume inputs, not the Spark profile association.

That contradicts #9497's requirement to preserve every unique assertion. Negative LOC is not a simplification when it silently drops the sole executable contract for a selected runtime value.

Refactor direction

Move the one defaultModel.envValue assertion into the focused Spark profile case before deleting the broad duplicate. No helper or additional scenario is needed.

Expected result

Each behavior still has one focused owner, the PR remains roughly 1,619 net lines smaller, and the Spark model ID plus its runtime environment value cannot drift independently.

@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite for the commit under review.

Recommended E2E: None

Manual-only E2E: cloud-onboard
The manual PR workflow does not run these selectors for the commit under review. Run them from reviewed code on main.

3 optional E2E recommendations
  • snapshot-commands
  • rebuild-hermes
  • inference-routing

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
@jyaunches
jyaunches dismissed their stale review August 18, 2026 18:16

Resolved by ad44f19: the focused Spark profile owner now preserves the unique envValue assertion without restoring duplicate coverage.

@jyaunches jyaunches left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOC Reduction / Codebase Simplicity Review

The prior blocking finding is resolved at ad44f19a099da2509b813df66880e89d22d63adb.

The retained focused Spark profile case now preserves defaultModel.envValue === "qwen3.6-35b-a3b-nvfp4" beside the model ID, image, and sizing assertions. The fix adds one line without restoring the broad duplicate suite, so each audited behavior keeps one focused owner and the PR remains 1,619 net lines smaller.

This follow-up review is limited to the prior finding and the new-head delta. I found no replacement LOC-reduction or codebase-simplicity blocker.

@prekshivyas prekshivyas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at exact head ad44f19. Reviewed the duplicate-test removal and retained unique assertions; no unresolved review threads remain and all required checks pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: remove duplicate behavior coverage after suite splits

5 participants