Skip to content

test(vitest): strengthen coverage safeguards - #6719

Closed
souvikDevloper wants to merge 1 commit into
NVIDIA:mainfrom
souvikDevloper:agent/vitest-coverage-guards
Closed

test(vitest): strengthen coverage safeguards#6719
souvikDevloper wants to merge 1 commit into
NVIDIA:mainfrom
souvikDevloper:agent/vitest-coverage-guards

Conversation

@souvikDevloper

Copy link
Copy Markdown
Contributor

Summary

Strengthens Vitest coverage safeguards so aggregate coverage cannot drift below its floor, security-sensitive files retain native per-file coverage minimums, and changed-file coverage is visible as a non-blocking CI advisory. It also pilots assertion-presence enforcement for the plugin project.

Related Issue

Part of #6692. Follows #6702, the preceding entry in the Vitest hardening stack.

Changes

  • Remove the one-point aggregate coverage tolerance and add unit coverage for exact-floor and below-floor behavior.
  • Add native per-file coverage thresholds for SSRF, credential filtering, redaction, policy, and transition-lock code; defer these floors for partial CLI shard runs so the merged report remains their consumer.
  • Add a changed-file coverage reporter to the CLI and plugin coverage actions as an advisory, continue-on-error step.
  • Require at least one assertion in plugin tests and protect the project configuration and workflow wiring with contract tests.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: this changes test and CI enforcement only; the documentation impact review found no user-facing behavior or workflow documentation change.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: self-review confirmed that only coverage floors for sensitive files change, not their production behavior; the focused SSRF coverage run passed all 101 tests at the new per-file floors.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: vitest run test/coverage-ratchet.test.ts (5 passed); focused workflow and plugin project contract tests (1 passed each); focused SSRF coverage run (101 passed, native per-file thresholds satisfied).
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: direct root CLI and plugin production/test typechecks pass. The POSIX shell portions of the broad gate are not runnable in this Windows checkout; CI is authoritative.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Windows validation note: the normal hooks passed formatting, lint, commit message, plugin TypeScript, CLI TypeScript, and version-sync checks. The repository-check aggregator and JS-config pre-push command hit Windows process/shell incompatibilities (spawnSync tsx.cmd EINVAL and POSIX if parsed by cmd.exe); their direct TypeScript equivalents passed.


Signed-off-by: souvikDevloper 138186578+souvikDevloper@users.noreply.github.com

@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 322a678c-d0b6-40bd-8123-e1f28a880927

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@copy-pr-bot

copy-pr-bot Bot commented Jul 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: target/main
Head: HEAD
Confidence: high

Required E2E

  • None. This is CI and test-quality tooling only. It changes coverage enforcement, reporting, and Vitest assertion requirements without changing production installer/onboarding, sandbox lifecycle, credentials, security-policy enforcement, network behavior, inference routing, deployment, or real assistant flows. Unit and workflow-contract tests are the appropriate validation layer.

Optional E2E

  • None.

New E2E recommendations

  • None.

@souvikDevloper

Copy link
Copy Markdown
Contributor Author

Closing this draft as a duplicate. Upstream main advanced during publication and now contains PR 8 via #6705 and the final stack entry via #6707; issue #6692 is fully completed. Thanks.

@souvikDevloper
souvikDevloper deleted the agent/vitest-coverage-guards branch July 12, 2026 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant