Skip to content

Validate playwrightCliVersion shape with strict SemVer#18225

Merged
mitchdenny merged 1 commit into
mainfrom
mitchdenny/validate-playwright-cli-version-main
Jun 16, 2026
Merged

Validate playwrightCliVersion shape with strict SemVer#18225
mitchdenny merged 1 commit into
mainfrom
mitchdenny/validate-playwright-cli-version-main

Conversation

@mitchdenny

Copy link
Copy Markdown
Member

Port of #18205 to main.

The playwrightCliVersion configuration value is forwarded to npm as the
package version specifier. Previously any non-empty string would be
passed through, so a typo or unsupported shape (a range, an npm
dist-tag like 'latest', a v-prefixed version, etc.) would surface as a
generic 'failed to resolve' error from npm.

Validate the override with SemVersion.TryParse using SemVersionStyles.Strict
and fail fast with a clear message that names the configuration key and
the offending value when it is not a valid SemVer 2.0 version.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 16, 2026 01:05
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 18225

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 18225"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ports the strict SemVer validation fix from the release/13.4 branch (#18205) to main. It prevents the playwrightCliVersion configuration override from accepting non-version strings (npm ranges, dist-tags, incomplete versions, etc.) that could be interpreted by npm in unexpected ways, instead returning a clear validation error early.

Changes:

  • Adds SemVersion.TryParse validation in PlaywrightCliInstaller.InstallCoreAsync to reject non-strict SemVer values before they reach npm resolution
  • Adds a new localized error message (PlaywrightCliInstaller_InvalidVersionOverride) to the .resx and all .xlf translation files
  • Adds a parameterized test covering five invalid version format variants
Show a summary per file
File Description
src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs Refactors ternary into if/else and adds strict SemVer validation before using the version override
src/Aspire.Cli/Resources/AgentCommandStrings.resx Adds PlaywrightCliInstaller_InvalidVersionOverride resource string
src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs Auto-generated accessor for the new resource string
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.*.xlf (12 files) Adds the new string to all localization files
tests/Aspire.Cli.Tests/Agents/PlaywrightCliInstallerTests.cs Adds [Theory] test for five invalid version formats verifying early failure

Copilot's findings

Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file
  • Files reviewed: 16/17 changed files
  • Comments generated: 0

@mitchdenny

Copy link
Copy Markdown
Member Author

PR Testing Report

PR Information

Artifact Version Verification

  • Expected Commit: 7c9a0e3
  • Installed Version: 13.5.0-pr.18225.g7c9a0e38
  • Status: ✅ Verified

Changes Analyzed

Files Changed

  • src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs - Modified
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs - Modified
  • src/Aspire.Cli/Resources/AgentCommandStrings.resx - Modified
  • src/Aspire.Cli/Resources/xlf/AgentCommandStrings.*.xlf - Modified (localizations)
  • tests/Aspire.Cli.Tests/Agents/PlaywrightCliInstallerTests.cs - Modified

Change Categories

  • CLI changes detected
  • Hosting integration changes
  • Dashboard changes
  • Template changes
  • Client/Component changes
  • VS Code extension changes
  • CI infrastructure changes (GitHub Actions / Azure DevOps)
  • Test changes

Test Scenarios Executed

Scenario 1: Dogfood install + commit verification

Objective: Verify dogfood artifact corresponds to the PR head.
Coverage Type: Artifact verification
Status: ✅ Passed

Steps:

  1. Installed PR CLI using dogfood script with isolated install path.
  2. Resolved PR head SHA from GitHub.
  3. Ran installed binary from the isolated install path and captured --version.

Evidence:

  • Log: /var/folders/gn/7n52dh0j0gqfk0qshzr592500000gn/T/aspire-pr-test-XXXXXX.0Wm3Buzc3i/install.log
  • Version: /var/folders/gn/7n52dh0j0gqfk0qshzr592500000gn/T/aspire-pr-test-XXXXXX.0Wm3Buzc3i/version.txt

Observations:

  • Installed CLI version includes short SHA 7c9a0e38 from PR head.

Scenario 2: Strict SemVer accepted (happy path)

Objective: Verify strict SemVer override values are accepted.
Coverage Type: Happy path
Status: ✅ Passed

Steps:

  1. Ran targeted test InstallAsync_WhenVersionOverrideConfigured_UsesOverrideVersion.
  2. Verified installer proceeds and uses the override value.

Evidence:

  • Log: /var/folders/gn/7n52dh0j0gqfk0qshzr592500000gn/T/aspire-pr-test-XXXXXX.0Wm3Buzc3i/scenario2-happy.log

Observations:

  • Test passed with no failures.

Scenario 3: Invalid override rejected (unhappy path)

Objective: Verify non-strict SemVer values are rejected with clear failure behavior.
Coverage Type: Unhappy path
Status: ✅ Passed

Steps:

  1. Ran targeted test InstallAsync_WhenVersionOverrideIsNotStrictSemVer_ReturnsFailed.
  2. Validated failure behavior for invalid values (range/dist-tag/partial/v-prefixed/invalid).

Evidence:

  • Log: /var/folders/gn/7n52dh0j0gqfk0qshzr592500000gn/T/aspire-pr-test-XXXXXX.0Wm3Buzc3i/scenario3-unhappy.log

Observations:

  • Test suite passed with expected rejection behavior.

Expected Unhappy-Path Outcome: Validation failure status and non-empty message including the invalid value.

Summary

Scenario Status Notes
Dogfood install + commit verification ✅ Passed Version matches PR head short SHA
Strict SemVer accepted (happy path) ✅ Passed Targeted method passed
Invalid override rejected (unhappy path) ✅ Passed 5 invalid inputs covered

Overall Result

✅ PR VERIFIED

Recommendations

  • No blocking issues found for this change.

@mitchdenny

Copy link
Copy Markdown
Member Author

PR Testing Complete\n\n✅ Verified\n\n- Installed dogfood CLI version matched PR head commit ().\n- Happy path passed: strict SemVer override accepted.\n- Unhappy path passed: invalid override values were rejected with validation failure behavior.\n\nOverall result: PR VERIFIED.

@mitchdenny

Copy link
Copy Markdown
Member Author

PR Testing Complete

✅ Verified

  • Installed dogfood CLI version matched PR head commit (7c9a0e38).
  • Happy path passed: strict SemVer override accepted.
  • Unhappy path passed: invalid override values were rejected with validation failure behavior.

Overall result: PR VERIFIED.

@mitchdenny mitchdenny merged commit ef982a6 into main Jun 16, 2026
338 checks passed
@mitchdenny mitchdenny deleted the mitchdenny/validate-playwright-cli-version-main branch June 16, 2026 01:48
@github-actions github-actions Bot added this to the 13.5 milestone Jun 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants