Replace Playwright MCP server with Playwright CLI installation

docs/specs/safe-npm-tool-install.md

@@ -0,0 +1,189 @@
+# Safe npm Global Tool Installation
+
+## Overview
+
+The Aspire CLI installs the `@playwright/cli` npm package as a global tool during `aspire agent init`. Because this tool runs with the user's full privileges, we must verify its authenticity and provenance before installation. This document describes the verification process, the threat model, and the reasoning behind each step.
+
+## Threat Model
+
+### What we're protecting against
+
+1. **Registry compromise** — An attacker gains write access to the npm registry and publishes a malicious version of `@playwright/cli`
+2. **Publish token theft** — An attacker steals a maintainer's npm publish token and publishes a tampered package
+3. **Man-in-the-middle** — An attacker intercepts the network request and substitutes a different tarball
+4. **Dependency confusion** — A malicious package with a similar name is installed instead of the intended one
+
+### What we're NOT protecting against
+
+- Compromise of the legitimate source repository (`microsoft/playwright-cli`) itself
+- Compromise of the GitHub Actions build infrastructure (Sigstore OIDC provider)
+- Compromise of the Sigstore transparency log infrastructure
+- Malicious code introduced through legitimate dependencies of `@playwright/cli`
+
+### Trust anchors
+
+Our verification chain relies on these trust anchors:
+
+| Trust anchor | What it provides | How it's protected |
+|---|---|---|
+| **npm registry** | Package metadata, tarball hosting | HTTPS/TLS, npm's infrastructure security |
+| **Sigstore (Fulcio + Rekor)** | Cryptographic attestation signatures | Public CA with OIDC federation, append-only transparency log |
+| **GitHub Actions OIDC** | Builder identity claims in Sigstore certificates | GitHub's infrastructure security |
+| **Hardcoded expected values** | Package name, version range, expected source repository | Code review, our own release process |
+
+## Verification Process
+
+### Step 1: Resolve package version and metadata
+
+**Action:** Run `npm view @playwright/cli@{versionRange} version` and `npm view @playwright/cli@{version} dist.integrity` to get the resolved version and the registry's SRI integrity hash.
+
+**What this establishes:** We know the exact version we intend to install and the hash the registry claims for its tarball.
+
+**Trust basis:** npm registry over HTTPS/TLS.
+
+**Limitations:** If the registry is compromised, both the version and hash could be attacker-controlled. This step alone is insufficient — it only establishes what the registry *claims*.
+
+### Step 2: Check if already installed at a suitable version
+
+**Action:** Run `playwright-cli --version` and compare against the resolved version.
+
+**What this establishes:** Whether installation can be skipped entirely (already up-to-date or newer).
+
+**Trust basis:** The previously-installed binary. If the user's system is compromised, this could be spoofed, but that's outside our threat model.
+
+### Step 3: Verify Sigstore attestations via npm
+
+**Action:**
+1. Create a temporary directory with a minimal `package.json`
+2. Run `npm install @playwright/cli@{version} --ignore-scripts` to install the package from the registry as a project dependency
+3. Run `npm audit signatures` to verify Sigstore attestation signatures
+
+**What this establishes:** That valid Sigstore-signed attestations exist for `@playwright/cli@{version}`. Specifically:
+
+- The npm registry has attestation bundles for this package version
+- The attestation signatures are cryptographically valid (signed by Sigstore's Fulcio CA)
+- The attestation entries are present in the Rekor transparency log (inclusion proof verified)
+- The OIDC identity in the signing certificate corresponds to a GitHub Actions workflow
+
+**Trust basis:** Sigstore's public key infrastructure. Even if the npm registry is compromised, an attacker cannot forge valid Sigstore signatures — they would need to compromise Fulcio (the Sigstore CA) or obtain a valid OIDC token from GitHub Actions for the legitimate repository's workflow.
+
+**Why a temporary project is needed:** `npm audit signatures` operates on installed project dependencies. It requires `node_modules` and a `package-lock.json` to know which packages to verify. For a global tool install there is no project context, so we create one temporarily. The package must be installed from the registry (not from a local tarball) because `npm audit signatures` skips packages with `resolved: file:...` in the lockfile.
+
+**Limitations:** `npm audit signatures` verifies that *valid attestations exist* but does not expose the attestation *content*. It confirms "this package has authentic Sigstore-signed attestations" but does not tell us *what* those attestations say (e.g., which repository built the package). That's addressed in Step 4.
+
+### Step 4: Verify provenance source repository
+
+**Action:**
+1. Fetch the attestation bundle from `https://registry.npmjs.org/-/npm/v1/attestations/@playwright/cli@{version}`
+2. Find the attestation with `predicateType: "https://slsa.dev/provenance/v1"` (SLSA Build L3 provenance)
+3. Base64-decode the DSSE envelope payload to extract the in-toto statement
+4. Parse `predicate.buildDefinition.externalParameters.workflow.repository`
+5. Verify it equals `https://github.com/microsoft/playwright-cli`
+
+**What this establishes:** That the Sigstore-attested provenance for this package version claims it was built from the `microsoft/playwright-cli` GitHub repository via a GitHub Actions workflow.
+
+**Trust basis:** The attestation content is protected by the Sigstore signature verified in Step 3. Since Step 3 confirmed the attestation is cryptographically authentic (signed by a valid Sigstore certificate corresponding to a GitHub Actions OIDC identity), the content we read in Step 4 cannot have been tampered with by the npm registry. An attacker would need to compromise Sigstore itself to forge attestation content pointing to `microsoft/playwright-cli`.
+
+**Why we fetch from the registry API:** The npm CLI (`npm audit signatures`) verifies Sigstore signatures but does not expose provenance content in its output. The `--json` flag only produces `{"invalid":[],"missing":[]}`. There is no npm CLI command to read the source repository from a SLSA provenance attestation. We must fetch the attestation bundle directly from the registry API.
+
+**Note on reading attested content from an untrusted source:** We are reading the attestation JSON from the same npm registry that could theoretically be compromised. However, the attestation *signature* was already verified by `npm audit signatures` in Step 3 using Sigstore's independent trust chain. We are relying on the fact that the registry serves the same attestation bundle that npm verified. If the registry served different attestation data to our HTTP request than what `npm audit signatures` verified, the provenance content could be spoofed. This is a residual risk — see "Residual Risks" below.
+
+### Step 5: Download and verify tarball integrity
+
+**Action:**
+1. Run `npm pack @playwright/cli@{version}` to download the tarball
+2. Compute SHA-512 hash of the downloaded tarball
+3. Compare against the SRI integrity hash obtained in Step 1
+
+**What this establishes:** That the tarball we have on disk is bit-for-bit identical to what the npm registry published for this version.
+
+**Trust basis:** Cryptographic hash comparison (SHA-512). If the hash matches, the content is the same regardless of how it was delivered.
+
+**Relationship to Step 3:** The Sigstore attestations verified in Step 3 are bound to the package version and its published content. The integrity hash in the registry packument is the canonical identifier for the tarball content. By verifying our tarball matches this hash, we establish that our tarball is the same artifact that the Sigstore attestations cover.
+
+### Step 6: Install globally from verified tarball
+
+**Action:** Run `npm install -g {tarballPath}` to install the verified tarball as a global tool.
+
+**What this establishes:** The tool is installed and available on the user's PATH.
+
+**Trust basis:** All preceding verification steps have passed. The tarball content has been verified against the registry's published hash (Step 5), the Sigstore attestations for that content are cryptographically valid (Step 3), and the attestations claim the correct source repository (Step 4).
+
+### Step 7: Generate skill files
+
+**Action:** Run `playwright-cli install --skills` to generate agent skill files.
+
+**What this establishes:** The Playwright CLI skill files are available for agent environments.
+
+## Verification Chain Summary
+
+```text
+                    │   Hardcoded expectations     │
+                    │   • Package: @playwright/cli │
+                    │   • Version range: 0.1.1     │
+                    │   • Source: microsoft/        │
+                    │     playwright-cli            │
+                    └──────────────┬────────────────┘
+                                   │
+                    ┌──────────────▼────────────────┐
+                    │  Step 1: Resolve version +     │
+                    │  integrity hash from registry  │
+                    └──────────────┬────────────────┘
+                                   │
+              ┌────────────────────┼────────────────────┐
+              │                    │                     │
+   ┌──────────▼──────────┐  ┌─────▼──────────┐  ┌──────▼──────────┐
+   │ Step 3: npm audit    │  │ Step 4: Verify  │  │ Step 5: npm pack│
+   │ signatures           │  │ provenance repo │  │ + SHA-512 check │
+   │ (Sigstore crypto)    │  │ (attestation    │  │ (tarball        │
+   │                      │  │  content)       │  │  integrity)     │
+   └──────────┬───────────┘  └─────┬──────────┘  └──────┬──────────┘
+              │                    │                     │
+              │  Attestation is    │  Built from         │  Tarball matches
+              │  authentic         │  expected repo      │  published hash
+              └────────────────────┼─────────────────────┘
+                                   │
+                    ┌──────────────▼────────────────┐
+                    │  Step 6: npm install -g        │
+                    │  (from verified tarball)        │
+                    └───────────────────────────────┘
+```
+
+## Residual Risks
+
+### 1. Registry serving different attestation data to different clients
+
+**Risk:** The npm registry could theoretically serve one attestation bundle to `npm audit signatures` (which passes verification) and a different bundle to our HTTP API request (with spoofed provenance content).
+
+**Mitigation:** This would require active, targeted compromise of the npm registry's serving infrastructure — not just a publish token theft or package tampering. The Rekor transparency log provides a public record of all attestations, making such targeted serving detectable.
+
+**Alternative mitigation:** We could eliminate this risk entirely by parsing the attestation bundle ourselves and verifying the Sigstore signature directly in C#. This would require implementing ECDSA signature verification, X.509 certificate chain validation, and Merkle inclusion proof verification. This significantly increases implementation complexity and is not recommended for the initial implementation.
+
+### 2. Time-of-check-to-time-of-use (TOCTOU)
+
+**Risk:** The package could be replaced on the registry between our verification steps and the global install.
+
+**Mitigation:** We verify the SHA-512 hash of the tarball we actually install (Step 5), and we install from the local tarball file (not from the registry again). The verified tarball is the same file that gets installed.
+
+### 3. Transitive dependency attacks
+
+**Risk:** `@playwright/cli` has dependencies that could be compromised.
+
+**Mitigation:** The `--ignore-scripts` flag prevents execution of install scripts. However, the dependencies' code runs when the tool is invoked. This is partially mitigated by Sigstore attestations covering the dependency tree, but comprehensive supply chain verification of all transitive dependencies is out of scope.
+
+## Implementation Constants
+
+```csharp
+internal const string PackageName = "@playwright/cli";
+internal const string VersionRange = "0.1.1";
+internal const string ExpectedSourceRepository = "https://github.com/microsoft/playwright-cli";
+internal const string NpmRegistryAttestationsUrl = "https://registry.npmjs.org/-/npm/v1/attestations";
+internal const string SlsaProvenancePredicateType = "https://slsa.dev/provenance/v1";
+```
+
+## Future Improvements
+
+1. **Direct Sigstore verification in C#** — Eliminate the dependency on `npm audit signatures` by implementing Sigstore bundle verification natively. This would remove residual risk #1 and reduce the dependency on npm CLI.
+2. **Rekor log verification** — Independently verify the Rekor inclusion proof to confirm the attestation was logged in the public transparency log.
+3. **Certificate identity verification** — Check the OIDC identity claims in the Sigstore signing certificate (issuer, subject, workflow reference) rather than just the provenance payload.
+4. **Pinned tarball hash** — Ship a known-good SRI hash with each Aspire release, eliminating the need to trust the registry for the hash at all.

src/Aspire.Cli/Agents/AgentEnvironmentScanContext.cs

@@ -10,6 +10,7 @@ internal sealed class AgentEnvironmentScanContext
 {
     private readonly List<AgentEnvironmentApplicator> _applicators = [];
     private readonly HashSet<string> _skillFileApplicatorPaths = new(StringComparer.OrdinalIgnoreCase);
+    private readonly HashSet<string> _skillBaseDirectories = new(StringComparer.OrdinalIgnoreCase);
 
     /// <summary>
     /// Gets the working directory being scanned.
@@ -24,31 +25,11 @@ internal sealed class AgentEnvironmentScanContext
     public required DirectoryInfo RepositoryRoot { get; init; }
 
     /// <summary>
-    /// Gets or sets a value indicating whether a Playwright applicator has been added.
+    /// Gets or sets a value indicating whether a Playwright CLI applicator has been added.
     /// This is used to ensure only one applicator for Playwright is added across all scanners.
     /// </summary>
     public bool PlaywrightApplicatorAdded { get; set; }
 
-    /// <summary>
-    /// Stores the Playwright configuration callbacks from each scanner.
-    /// These will be executed if the user selects to configure Playwright.
-    /// </summary>
-    private readonly List<Func<CancellationToken, Task>> _playwrightConfigurationCallbacks = [];
-
-    /// <summary>
-    /// Adds a Playwright configuration callback for a specific environment.
-    /// </summary>
-    /// <param name="callback">The callback to execute if Playwright is configured.</param>
-    public void AddPlaywrightConfigurationCallback(Func<CancellationToken, Task> callback)
-    {
-        _playwrightConfigurationCallbacks.Add(callback);
-    }
-
-    /// <summary>
-    /// Gets all registered Playwright configuration callbacks.
-    /// </summary>
-    public IReadOnlyList<Func<CancellationToken, Task>> PlaywrightConfigurationCallbacks => _playwrightConfigurationCallbacks;
-
     /// <summary>
     /// Checks if a skill file applicator has already been added for the specified path.
     /// </summary>
@@ -82,4 +63,19 @@ public void AddApplicator(AgentEnvironmentApplicator applicator)
     /// Gets the collection of detected applicators.
     /// </summary>
     public IReadOnlyList<AgentEnvironmentApplicator> Applicators => _applicators;
+
+    /// <summary>
+    /// Registers a skill base directory for an agent environment (e.g., ".claude/skills", ".github/skills").
+    /// These directories are used to mirror skill files across all detected agent environments.
+    /// </summary>
+    /// <param name="relativeSkillBaseDir">The relative path to the skill base directory from the repository root.</param>
+    public void AddSkillBaseDirectory(string relativeSkillBaseDir)
+    {
+        _skillBaseDirectories.Add(relativeSkillBaseDir);
+    }
+
+    /// <summary>
+    /// Gets the registered skill base directories for all detected agent environments.
+    /// </summary>
+    public IReadOnlyCollection<string> SkillBaseDirectories => _skillBaseDirectories;
 }