Skip to content

perf(ci): remove unnecessary pnpm install from merge-reports job #8030

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Myestery merged 2 commits into from
Jan 13, 2026
Merged

perf(ci): remove unnecessary pnpm install from merge-reports job #8030

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0041b67
perf(ci): remove unnecessary pnpm install from merge-reports job
Myestery Jan 13, 2026
14c3808
perf(ci): use pnpm dlx instead of npx for better caching
Myestery Jan 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/ci-tests-e2e.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,9 +144,10 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v5

# Setup pnpm/node to run playwright merge-reports (no browsers needed)
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 10

- name: Download blob reports
uses: actions/download-artifact@v4
Expand All @@ -158,10 +159,10 @@ jobs:
- name: Merge into HTML Report
run: |
# Generate HTML report
pnpm exec playwright merge-reports --reporter=html ./all-blob-reports
pnpm dlx @playwright/test merge-reports --reporter=html ./all-blob-reports
# Generate JSON report separately with explicit output path
PLAYWRIGHT_JSON_OUTPUT_NAME=playwright-report/report.json \
pnpm exec playwright merge-reports --reporter=json ./all-blob-reports
pnpm dlx @playwright/test merge-reports --reporter=json ./all-blob-reports
Comment on lines 159 to +165
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat package.json | grep -A 1 -B 1 '@playwright/test'

Repository: Comfy-Org/ComfyUI_frontend

Length of output: 179

🏁 Script executed:

# Check for catalog definition in pnpm-workspace.yaml or top-level catalog in package.json
if [ -f "pnpm-workspace.yaml" ]; then
  echo "=== pnpm-workspace.yaml ===" 
  grep -A 30 "catalog:" pnpm-workspace.yaml | head -50
fi

# Also check if there's a .pnpmfile.cjs
if [ -f ".pnpmfile.cjs" ]; then
  echo "=== .pnpmfile.cjs ===" 
  cat .pnpmfile.cjs | head -30
fi

# Check pnpm-lock.yaml for the resolved version
echo "=== Checking pnpm-lock.yaml for @playwright/test ===" 
grep -A 3 "'@playwright/test'" pnpm-lock.yaml | head -20

Repository: Comfy-Org/ComfyUI_frontend

Length of output: 1500

🏁 Script executed:

# Read the CI workflow file to see full context
cat -n .github/workflows/ci-tests-e2e.yaml | sed -n '150,180p'

Repository: Comfy-Org/ComfyUI_frontend

Length of output: 1447

🏁 Script executed:

# Find the test shard job definitions in the workflow
cat -n .github/workflows/ci-tests-e2e.yaml | head -160 | tail -100

Repository: Comfy-Org/ComfyUI_frontend

Length of output: 4032

🏁 Script executed:

# Search for how pnpm dlx behaves with unversioned packages
# Also check if the merge-reports job installs dependencies
cat -n .github/workflows/ci-tests-e2e.yaml | sed -n '139,160p'

Repository: Comfy-Org/ComfyUI_frontend

Length of output: 775

🌐 Web query:

Playwright blob report format version compatibility merge-reports

💡 Result:

Short answer

  • Playwright blob reports are ZIP blobs produced by the "blob" reporter and intended to be read/merged by the built-in CLI tool npx playwright merge-reports. [1][2][3]
  • Best practice / compatibility rule: generate and merge blob reports with the same Playwright release (or at least the same minor release series). Mixing blobs produced by much older Playwright versions with a newer merge-reports tool can break or produce unexpected output. When in doubt, use the same Playwright version to create and to merge. [1][2][4]

Details / practical guidance

  • What a blob is: the blob reporter writes report-.zip files that include all test metadata and attachments; merge-reports extracts those blobs and produces HTML/JSON/JUnit/etc. using the current reporter implementation. [2]
  • When merging across OS / environments you may need an explicit merge config (to pick test-root, tags, etc.). [1]
  • Known issues: users have reported sporadic/behavioral problems when merging (empty output, reporters writing to stdout vs file, etc.), which indicates the merge step can be sensitive to Playwright version, reporter choice, and CI workflow details. If you see missing output, ensure blobs are intact, use the same Playwright version, and try piping JSON/JUnit output to a file as a workaround. [4][5]

References

  1. Playwright docs — Test sharding / merging reports (blob + merge-reports). [Playwright Test sharding / merge-reports]. [1]
  2. Playwright docs — Blob reporter details (file name, contents). [Playwright Reporters / Blob]. [2]
  3. Playwright release notes introducing merge-reports (v1.37). [Release notes]. [3]
  4. Playwright GitHub issues reporting merge-reports problems (examples of failures / unexpected behavior). [GitHub issue examples]. [4][5]

If you want, I can:

  • Inspect a sample report-.zip you have and tell you which Playwright version likely produced it, or
  • Give a step-by-step CI workflow that reliably collects and merges blob reports (GitHub Actions example).

Pin @playwright/test version to match the project's installed version to ensure blob report compatibility.

The test shards use Playwright 1.57.0 (from pnpm install --frozen-lockfile in the test jobs), but the merge job uses pnpm dlx @playwright/test without a version pin. Since pnpm dlx fetches the latest available version from npm when no version is specified, mismatches can occur. Blob report format compatibility between different Playwright versions is not guaranteed, and version mismatches can cause merge failures or empty reports.

Pin the version to match the project:

-          pnpm dlx @playwright/test merge-reports --reporter=html ./all-blob-reports
+          pnpm dlx @playwright/test@1.57.0 merge-reports --reporter=html ./all-blob-reports
-          pnpm dlx @playwright/test merge-reports --reporter=json ./all-blob-reports
+          pnpm dlx @playwright/test@1.57.0 merge-reports --reporter=json ./all-blob-reports

Alternatively, add pnpm install --frozen-lockfile to the merge job to use the version from package.json automatically.

🤖 Prompt for AI Agents 
In @.github/workflows/ci-tests-e2e.yaml around lines 159 - 165, The merge step
currently invokes "pnpm dlx @playwright/test merge-reports" without a version,
causing potential Playwright version mismatch with the test shards; update the
merge job to either call the exact installed Playwright version (e.g., replace
"pnpm dlx @playwright/test" with "pnpm dlx @playwright/test@1.57.0" to match the
project's Playwright 1.57.0) or run "pnpm install --frozen-lockfile" in the
merge job before invoking "pnpm dlx @playwright/test merge-reports" so the merge
uses the same locked version and avoids blob report format incompatibilities.


- name: Upload HTML report
uses: actions/upload-artifact@v4
Expand Down