Skip to content

test(e2e): derive inventory and selector contracts - #6710

Merged
cv merged 2 commits into
codex/6708-schema-blueprint-contractsfrom
codex/6708-e2e-derived-inventory
Jul 12, 2026
Merged

test(e2e): derive inventory and selector contracts#6710
cv merged 2 commits into
codex/6708-schema-blueprint-contractsfrom
codex/6708-e2e-derived-inventory

Conversation

@cv

@cv cv commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

E2E support tests now derive manifest, matrix, and dispatch expectations from the production registries and workflow planner instead of copying the current inventory. Focused malformed-selector, secret-bearing-job, artifact, runner, and unsupported-target behavior remains explicit.

Related Issue

Part of #6708.

Changes

  • Remove literal manifest count, target-field, and observability snapshots.
  • Verify every typed target resolves to a validated manifest path.
  • Replace the hard-coded live matrix with supported-target, uniqueness, runner-resolution, unsupported-selection, and CLI parity behavior.
  • Collapse hundreds of repeated selector assertions into registry-derived coverage while retaining invalid and mixed selector cases.
  • Replace exact workflow target-to-job literals with referential-integrity checks against actual workflow jobs.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: Test methodology changes only; E2E targets, workflows, commands, and runtime behavior are unchanged.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Test-only cleanup preserves malformed selector rejection, secret-bearing job gating, artifact-path rejection, runner resolution, and unsupported-target evidence.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — vitest --project e2e-support: 3 files and 30 tests passed; npm run test:titles:check passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

@cv cv self-assigned this Jul 12, 2026
@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 86cc6a09-4650-4ec6-8b0f-bfffb0b09ea3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/6708-e2e-derived-inventory

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the codex/6708-schema-bl... branch.

TypeScript / code-coverage/cli

The overall coverage in the codex/6708-e2e-deriv... branch remains at 79%, unchanged from the codex/6708-schema-bl... branch.

Show a code coverage summary of the most impacted files.
File codex/6708-schema-bl... e04a344 codex/6708-e2e-deriv... 0161c69 +/-
src/lib/messagi...ate-resolver.ts 88% 76% -12%
src/lib/messagi...parser-utils.ts 90% 86% -4%
src/lib/security/redact-url.ts 100% 97% -3%
src/lib/actions...ateway-state.ts 80% 79% -1%
src/lib/adapters/http/probe.ts 90% 89% -1%
src/lib/security/redact.ts 100% 99% -1%
src/lib/agent/s...store-reader.ts 88% 88% 0%
src/lib/actions...flow-helpers.ts 80% 84% +4%
src/lib/onboard...eway-cleanup.ts 58% 70% +12%
src/lib/name-validation.ts 81% 100% +19%

Updated July 12, 2026 09:09 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@github-actions

github-actions Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/codex/6708-schema-blueprint-contracts
Head: HEAD
Confidence: high

Required E2E

  • None. Tests-only changes to E2E support assertions. They cannot affect installer/onboarding, sandbox lifecycle, credentials, security boundaries, network policy, inference routing, deployment, or real assistant behavior at runtime; run the affected Vitest support tests rather than live E2E.

Optional E2E

  • None.

New E2E recommendations

  • None.

@github-actions

github-actions Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: No advisor follow-up required beyond maintainer review.
Findings: 0 required · 0 warnings · 0 optional suggestions
Since last review: 0 prior items resolved · 0 still apply · 0 new items found

Workflow run details

This is an automated review. Required findings need action before merge. Warnings and optional suggestions do not require a response or follow-up. A human maintainer makes the final merge decision.

@cv
cv merged commit 60f895f into codex/6708-schema-blueprint-contracts Jul 12, 2026
48 checks passed
@cv
cv deleted the codex/6708-e2e-derived-inventory branch July 12, 2026 09:10
@wscurran wscurran added area: e2e End-to-end tests, nightly failures, or validation infrastructure refactor PR restructures code without intended behavior change labels Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure refactor PR restructures code without intended behavior change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants