CI: Use custom container for E2E tests #7625
Conversation
dosubot
bot
added
the
size:S
|
Important Review skippedReview was skipped due to path filters ⛔ Files ignored due to path filters (2)
CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including You can disable this status message by setting the Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 WalkthroughWalkthroughReplaces cache-based dist propagation with artifact upload/download; containerizes Playwright E2E and snapshot-update jobs. Setup builds and uploads Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Dev as Developer (push/PR)
participant Setup as Setup Job (runner)
participant Artifacts as GitHub Artifacts
participant Container as Containerized Test Job
participant Comfy as ComfyUI (in-container)
participant Merge as Merge Reports Job
Dev->>Setup: push / open PR
Setup->>Setup: build frontend -> produce `dist/`
Setup->>Artifacts: upload `frontend-dist`
Note over Container,Artifacts: Test jobs run inside container images
Container->>Artifacts: download `frontend-dist`
Container->>Container: copy devtools & install frontend deps
Container->>Comfy: start ComfyUI server (in-container)
Container->>Container: run Playwright tests -> emit blob reports & snapshots
Container->>Artifacts: upload per-shard blob & snapshot artifacts
alt snapshots present
Merge->>Artifacts: download blob reports & snapshots
Merge->>Merge: merge blobs -> generate final HTML/JSON -> upload results
else no snapshots
Merge-->>Merge: detect no artifacts -> exit early
end
Possibly related PRs
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🎨 Storybook Build Status✅ Build completed successfully! ⏰ Completed at: 12/30/2025, 03:30:18 AM UTC 🔗 Links🎉 Your Storybook is ready for review! |
🎭 Playwright Test Results❌ Some tests failed ⏰ Completed at: 12/30/2025, 03:35:20 AM UTC 📈 Summary
📊 Test Reports by Browser
🎉 Click on the links above to view detailed test results for each browser configuration. |
Bundle Size ReportSummary
Category Glance Per-category breakdownApp Entry Points — 3.2 MB (baseline 3.2 MB) • ⚪ 0 BMain entry bundles and manifests
Graph Workspace — 1000 kB (baseline 1000 kB) • ⚪ 0 BGraph editor runtime, canvas, workflow orchestration
Views & Navigation — 6.63 kB (baseline 6.63 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces
Panels & Settings — 300 kB (baseline 300 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens
UI Components — 199 kB (baseline 199 kB) • ⚪ 0 BReusable component library chunks
Data & Services — 12.5 kB (baseline 12.5 kB) • ⚪ 0 BStores, services, APIs, and repositories
Utilities & Hooks — 1.41 kB (baseline 1.41 kB) • ⚪ 0 BHelpers, composables, and utility bundles
Vendor & Third-Party — 9.12 MB (baseline 9.12 MB) • ⚪ 0 BExternal libraries and shared vendor chunks
Other — 3.47 MB (baseline 3.47 MB) • ⚪ 0 BBundles that do not match a named category
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/ci-tests-e2e.yaml (1)
47-66: Update container image to use Playwright v1.57.0 to match project dependency.The project uses Playwright v1.57.0, but the container image
mcr.microsoft.com/playwright:v1.52.0-nobleruns v1.52.0. If the Playwright version in the Docker image does not match the version in the project/tests, Playwright will be unable to locate browser executables. Change the container image tag fromv1.52.0-nobletov1.57.0-noble.Additionally, cache files are created by the runner (UID 1001 on ubuntu-latest) and mounted into the container, but the Playwright container's user 1001 may not have the necessary permissions if that user doesn't exist in the container. Consider using
--user rootor ensure the container image has a user with UID 1001 configured.
♻️ Duplicate comments (1)
.github/workflows/ci-tests-e2e.yaml (1)
96-98: Same permission and version concerns apply here.This job has the same container configuration and cache restoration pattern as
playwright-tests-chromium-sharded, so the permission mismatch and version compatibility concerns from lines 47-66 apply here as well.
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/ci-tests-e2e.yaml(7 hunks)
🧰 Additional context used
🧠 Learnings (6)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Use Playwright tags like `mobile`, `2x` for relevant E2E tests - they are respected by config
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : E2E test files must use pattern `browser_tests/**/*.spec.ts` with Playwright
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Use Playwright tags like `mobile`, `2x` for relevant E2E tests - they are respected by config
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : E2E test files must use pattern `browser_tests/**/*.spec.ts` with Playwright
Applied to files:
.github/workflows/ci-tests-e2e.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup
- GitHub Check: lint-and-format
- GitHub Check: test
- GitHub Check: collect
🔇 Additional comments (1)
.github/workflows/ci-tests-e2e.yaml (1)
9-9: LGTM: Reasonable optimization to skip E2E tests for version bump PRs.This addition prevents running expensive E2E tests on automated version bump PRs, which is a sensible optimization.
Myestery
force-pushed
the
ci/playwright-docker-containers
branch
from
952e5cd to
ef5a55d
Compare
dosubot
bot
added
size:M
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
.github/workflows/ci-tests-e2e.yaml (1)
147-149: Note: merge-reports container usage already flagged.This has already been reviewed in a previous comment. The full Playwright container (with browsers) is not necessary for the
merge-reportsjob, but using it ensures CLI version consistency. A lighter Node.js container would be more efficient if startup time is a concern.
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/ci-tests-e2e.yaml(5 hunks)
🧰 Additional context used
🧠 Learnings (5)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : E2E test files must use pattern `browser_tests/**/*.spec.ts` with Playwright
Applied to files:
.github/workflows/ci-tests-e2e.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: lint-and-format
- GitHub Check: setup
- GitHub Check: test
- GitHub Check: collect
🔇 Additional comments (4)
.github/workflows/ci-tests-e2e.yaml (4)
9-9: LGTM: Skip E2E tests on version bump PRs.Adding
version-bump*to the ignore list is appropriate for automated version bumps that don't require E2E validation.
55-61: LGTM: Correct pattern for container-based jobs with composite actions.The checkout-then-download pattern is necessary because:
- Checkout provides access to
.github/actions(local composite actions)- Workspace download provides built artifacts from the setup job
This is the correct approach for container-based workflows using artifacts.
63-70: LGTM: Efficient test environment setup.The setup correctly:
- Launches the ComfyUI server for testing
- Reuses the built frontend from the workspace artifact
- Avoids redundant installation/build steps
- Relies on container-provided browsers
72-84: LGTM: Proper sharded test execution and report handling.The implementation correctly:
- Uses blob reporter for efficient report merging across shards
- Uploads reports even on test failure (
!cancelled()condition)- Uses appropriate 1-day retention for intermediate blob reports
- Properly names artifacts with shard indices
There was a problem hiding this comment.
For future PRs:
Unused files needing removal:
- .github/actions/setup-playwright/action.yml
Docs needing updates after this PR:
browser_tests/README.md - Remove pnpm exec playwright install chromium --with-depskeep this as it's docs for running locally- CONTRIBUTING.md - Update Playwright test section to mention Docker approach
|
We still have more issues to fix |
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (4)
.github/workflows/ci-tests-e2e.yaml (4)
29-38: Consider optimizing workspace artifact size to reduce CI overhead.Using artifacts instead of cache works correctly with containers, but uploading the entire workspace (except
.git) may create large artifacts if it includesnode_modulesand build outputs. This can slow upload/download times and consume storage quota unnecessarily.If the artifact size exceeds 500MB, consider either:
- Excluding
node_modulesand runningpnpm installin downstream jobs (with cache if possible)- Using a targeted artifact containing only
dist/and files essential for test jobsNote that file permissions are not preserved during artifact uploads.
44-46: Update Playwright container image to match project version.The container image (v1.52.0) lags behind the project's Playwright dependency (v1.57.0). Update the container image from
mcr.microsoft.com/playwright:v1.52.0-nobletomcr.microsoft.com/playwright:v1.57.0-nobleto ensure consistency between CI and development environments.🔎 Proposed fix
container: - image: mcr.microsoft.com/playwright:v1.52.0-noble + image: mcr.microsoft.com/playwright:v1.57.0-noble options: --user 1001
92-94: Update Playwright container image to match project version.Same version mismatch as in the chromium-sharded job. Update to v1.57.0.
🔎 Proposed fix
container: - image: mcr.microsoft.com/playwright:v1.52.0-noble + image: mcr.microsoft.com/playwright:v1.57.0-noble options: --user 1001
149-151: Consider lighter container for merge-reports job.The
merge-reportsjob only runsplaywright merge-reportsCLI commands and doesn't execute tests or require browsers. Using the full Playwright container (which includes browser binaries, ~1.3GB) may be excessive.However, using the same container ensures:
- Consistent Playwright CLI version across all jobs
- Simplified maintenance (single container version to update)
If you prioritize faster job startup and lower resource usage, consider using a lighter Node.js container with just
@playwright/testinstalled. Otherwise, the current approach is acceptable for consistency.💡 Alternative: Use lightweight Node container
merge-reports: needs: [playwright-tests-chromium-sharded] runs-on: ubuntu-latest - container: - image: mcr.microsoft.com/playwright:v1.52.0-noble - options: --user 1001 if: ${{ !cancelled() }} steps: - name: Checkout repository uses: actions/checkout@v5 - # Setup pnpm/node to run playwright merge-reports (browsers pre-installed in container) + # Setup pnpm/node to run playwright merge-reports - name: Setup frontend uses: ./.github/actions/setup-frontendThis would use the default ubuntu-latest runner, which should work since no browsers are needed.
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
.github/actions/setup-comfyui-server/action.yml(3 hunks).github/workflows/ci-tests-e2e.yaml(5 hunks)
🧰 Additional context used
🧠 Learnings (5)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : E2E test files must use pattern `browser_tests/**/*.spec.ts` with Playwright
Applied to files:
.github/workflows/ci-tests-e2e.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: lint-and-format
- GitHub Check: test
- GitHub Check: setup
- GitHub Check: collect
🔇 Additional comments (8)
.github/actions/setup-comfyui-server/action.yml (2)
12-15: LGTM! Container mode input is well-defined.The new
container_modeinput properly distinguishes between host and container execution environments with appropriate defaults.
39-40: LGTM! Conditional Python setup is correct.The conditional logic correctly skips Python setup when running inside a container, as the container provides its own Python environment.
.github/workflows/ci-tests-e2e.yaml (6)
9-9: LGTM! Version bump branch exclusion is appropriate.Adding
version-bump*to branches-ignore prevents unnecessary E2E test runs on automated version bump PRs.
56-61: LGTM! Checkout and workspace download pattern is correct.The sequence of checking out the repository first (to get
.github/actions) then downloading the workspace artifact is the proper approach for container-based jobs.
68-68: LGTM! Container mode flag correctly passed.Setting
container_mode: trueensures the setup action skips redundant Python installation and uses the container's system Python.
102-108: LGTM! Checkout and workspace download pattern is correct.The pattern is consistent with the chromium-sharded job and properly structured.
115-115: LGTM! Container mode flag correctly passed.Consistent with the chromium-sharded job.
44-46: No action needed—--user 1001is the officially recommended pattern for this Playwright container image in GitHub Actions.Playwright's official CI documentation for GitHub Actions uses
--user 1001withmcr.microsoft.com/playwright:v1.57.0-noble. This configuration is verified as standard practice with no documented permission issues.
dosubot
bot
added
size:L
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
.github/actions/setup-comfyui-server/action.yml (1)
55-58: Inconsistent pip invocation style (duplicate concern).This code mixes
python3 -m pip(line 55) withpip3(lines 56-58). As noted in a previous review, usingpython3 -m pipconsistently is more explicit and ensures you're using the pip module associated with the specific Python interpreter—especially important in containerized environments.🔎 Proposed fix for consistency
python3 -m pip install --upgrade pip --break-system-packages || python3 -m pip install --upgrade pip -pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu --break-system-packages || pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu -pip3 install -r requirements.txt --break-system-packages || pip3 install -r requirements.txt -pip3 install wait-for-it --break-system-packages || pip3 install wait-for-it +python3 -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu --break-system-packages || python3 -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu +python3 -m pip install -r requirements.txt --break-system-packages || python3 -m pip install -r requirements.txt +python3 -m pip install wait-for-it --break-system-packages || python3 -m pip install wait-for-it
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
.github/actions/setup-comfyui-server/action.yml(2 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
.github/actions/setup-comfyui-server/action.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup
- GitHub Check: lint-and-format
- GitHub Check: test
- GitHub Check: collect
🔇 Additional comments (3)
.github/actions/setup-comfyui-server/action.yml (3)
12-15: LGTM! Container mode input is well-designed.The
container_modeinput enables dual-mode operation while maintaining backward compatibility with existing workflows.
39-40: LGTM! Conditional Python setup is appropriate.Correctly skips the setup-python action when running in container mode where system Python is available.
65-65: LGTM! Correct use ofpython3.Explicitly using
python3ensures the correct interpreter is invoked in the container environment.
dosubot
bot
added
size:M
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
.github/actions/setup-comfyui-server/action.yml (2)
70-78: Usepython3 -m pipconsistently.Line 75 uses
python3 -m pipwhile lines 76-78 usepip3. For consistency and to ensure the correct interpreter's pip is used, usepython3 -m pipthroughout.🔎 Proposed fix
- name: Install Python requirements (host mode) if: ${{ inputs.container_mode != 'true' }} shell: bash working-directory: ComfyUI run: | python3 -m pip install --upgrade pip - pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu - pip3 install -r requirements.txt - pip3 install wait-for-it + python3 -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu + python3 -m pip install -r requirements.txt + python3 -m pip install wait-for-it
45-49: Use Python Playwright container image and remove this step.This step will fail because:
- User 1001 (non-root) cannot execute
apt-getcommands—permission denied- The workflow uses
mcr.microsoft.com/playwright:v1.57.0-noble, which is the Node.js-based image without PythonThe Python-enabled Playwright image includes Python and pip pre-installed. Update the workflow to use
mcr.microsoft.com/playwright/python:v1.57.0-nobleand remove this step entirely.🔎 Required fixes in both files
In
.github/workflows/ci-tests-e2e.yaml, update the container image:container: - image: mcr.microsoft.com/playwright:v1.57.0-noble + image: mcr.microsoft.com/playwright/python:v1.57.0-nobleApply this change to all three container specifications (lines 44-45, 91-92, and 147-148).
In
.github/actions/setup-comfyui-server/action.yml, remove the apt-get step:- - name: Install pip (container mode) - if: ${{ inputs.container_mode == 'true' }} - shell: bash - run: | - apt-get update && apt-get install -y python3-pip python3-venv -.github/workflows/ci-tests-e2e.yaml (2)
91-92: Use Python-enabled Playwright container.Same issue as lines 44-46: use
mcr.microsoft.com/playwright/python:v1.57.0-nobleinstead.
147-148: Use Python-enabled Playwright container.Same issue as previous container specifications: use
mcr.microsoft.com/playwright/python:v1.57.0-noble.Note: The merge-reports job doesn't require browsers or Python—it only runs
playwright merge-reports. Using the Python Playwright image works but is heavier than necessary. Consider this addressed for consistency, or optimize later with a lighter Node.js-only image if startup time becomes a concern.
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
.github/actions/setup-comfyui-server/action.yml(2 hunks).github/workflows/ci-tests-e2e.yaml(5 hunks)
🧰 Additional context used
🧠 Learnings (5)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Use Playwright tags like `mobile`, `2x` for relevant E2E tests - they are respected by config
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
.github/actions/setup-comfyui-server/action.yml.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : E2E test files must use pattern `browser_tests/**/*.spec.ts` with Playwright
Applied to files:
.github/workflows/ci-tests-e2e.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup
- GitHub Check: test
- GitHub Check: lint-and-format
- GitHub Check: collect
🔇 Additional comments (12)
.github/actions/setup-comfyui-server/action.yml (6)
12-15: LGTM: Clear input parameter.The
container_modeinput is well-defined with appropriate defaults and documentation.
39-43: LGTM: Clean conditional logic for host mode.The renamed step with conditional execution properly separates host and container workflows.
51-57: LGTM: Standard virtual environment setup.The venv creation and PATH configuration are correct. This will work once the Python Playwright container image is used.
59-68: LGTM: Consistent pip usage within venv.Using
pipdirectly after activating the virtual environment is correct and more concise thanpython -m pip.
80-87: LGTM: Proper venv activation before server start.The server startup correctly activates the virtual environment and uses the venv's Python executable.
89-95: LGTM: Host mode server startup is correct.Using
python3directly in host mode is appropriate since the setup-python action installs it aspython3..github/workflows/ci-tests-e2e.yaml (6)
9-9: LGTM: Skip E2E tests for version bumps.Adding
version-bump*to the ignore list is a sensible optimization for automated version bump PRs.
30-38: LGTM: Artifact-based workspace sharing works with containers.Switching from cache to artifacts is necessary for container-based workflows. The entire workspace upload ensures all dependencies are available.
54-60: LGTM: Checkout before download ensures actions are available.The explicit checkout step is necessary to access
.github/actionsbefore downloading the workspace artifact.
67-67: LGTM: Correct container_mode usage.Passing
container_mode: trueactivates the container-specific setup path in the action.
100-106: LGTM: Consistent checkout-then-download pattern.Same correct approach as the chromium-sharded job.
113-113: LGTM: Correct container_mode usage.Consistent with the chromium-sharded job.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
.github/workflows/ci-tests-e2e.yaml (1)
89-90: Add security option to run container as non-root user.This container specification has the same security issue as lines 42-43: missing the
--user 1001option mentioned in the PR summary.🔎 Proposed fix
container: image: mcr.microsoft.com/playwright/python:v1.57.0-noble + options: --user 1001
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
.github/actions/setup-comfyui-server/action.yml(2 hunks).github/workflows/ci-tests-e2e.yaml(6 hunks)
🧰 Additional context used
🧠 Learnings (6)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Use Playwright tags like `mobile`, `2x` for relevant E2E tests - they are respected by config
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
.github/workflows/ci-tests-e2e.yaml.github/actions/setup-comfyui-server/action.yml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : E2E test files must use pattern `browser_tests/**/*.spec.ts` with Playwright
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-16T17:41:58.464Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-16T17:41:58.464Z
Learning: Prerequisites for repository setup: Node.js >= 24, Git repository, available ports for dev server and storybook
Applied to files:
.github/workflows/ci-tests-e2e.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: lint-and-format
- GitHub Check: setup
- GitHub Check: collect
- GitHub Check: test
🔇 Additional comments (11)
.github/actions/setup-comfyui-server/action.yml (5)
12-15: LGTM: Container mode input enables flexible execution.The
container_modeinput cleanly separates container and host execution paths. The default of'false'maintains backward compatibility for existing callers.
39-43: LGTM: Host-only Python setup is correctly gated.The conditional execution ensures Python setup only runs on host runners where it's needed, avoiding redundant setup in containers that already provide Python.
45-53: Verify the security implications of--break-system-packages.The
--break-system-packagesflag bypasses PEP 668 protection for externally-managed Python environments. While necessary in some containerized setups, this approach:
- Allows pip to modify system-managed packages
- Could lead to dependency conflicts or version mismatches
- Is appropriate for ephemeral CI containers but should be intentional
Consider whether a virtual environment approach would be more robust, or confirm that this flag is the intended solution for the Playwright Python container environment.
55-63: LGTM: Host mode requirements installation is correctly configured.The host mode path correctly uses
python3 -m pipwithout--break-system-packages, which is appropriate for non-containerized environments where Python is managed byactions/setup-python.
70-70: LGTM: Explicitpython3command improves reliability.Using
python3instead ofpythonensures the correct interpreter is invoked consistently across both container and host execution modes..github/workflows/ci-tests-e2e.yaml (6)
9-9: LGTM: Skipping E2E tests for version bump PRs is sensible.Version bump PRs typically only update version numbers and don't require full E2E test coverage, reducing unnecessary CI overhead.
30-36: LGTM: Artifact-based distribution is cleaner than cache-based approach.Uploading only the built
dist/directory is more efficient than the full workspace. The 1-day retention is appropriate for CI artifacts that are only needed during the workflow run.
52-58: LGTM: Artifact-based workflow requires checkout before download.The addition of checkout and artifact download steps correctly establishes the workspace needed for containerized test execution. The path mapping (
frontend-dist→dist/) aligns with the setup job's artifact upload.
65-65: LGTM: Container mode flag correctly activates container-specific setup.Passing
container_mode: trueenables the appropriate Python installation logic with--break-system-packagesflags in the composite action.
66-67: LGTM: Step name accurately reflects its reduced scope.The rename to "Setup nodejs, pnpm" correctly indicates that frontend building is handled in the setup job, and this step only installs runtime dependencies.
141-152: LGTM: Merge-reports job correctly avoids unnecessary container overhead.Running the merge-reports job directly on
ubuntu-latestwithout a container is the right choice. It only needs the Playwright CLI (via pnpm) to merge blob reports and doesn't require browsers or Python, making the lightweight approach more efficient.
Myestery
force-pushed
the
ci/playwright-docker-containers
branch
from
94ec109 to
df91d2a
Compare
|
All alerts resolved. Learn more about Socket for GitHub. This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored. |
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/pr-update-playwright-expectations.yaml(4 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Use Playwright tags like `mobile`, `2x` for relevant E2E tests - they are respected by config
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : E2E test files must use pattern `browser_tests/**/*.spec.ts` with Playwright
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
.github/workflows/pr-update-playwright-expectations.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: lint-and-format
- GitHub Check: collect
- GitHub Check: test
- GitHub Check: setup
🔇 Additional comments (7)
.github/workflows/pr-update-playwright-expectations.yaml (7)
66-73: LGTM: Artifact-based distribution is appropriate for container workflows.The switch from cache to artifacts with 1-day retention is correct since cache doesn't work reliably inside containers.
91-95: LGTM: Artifact download correctly replaces cache restoration.The artifact-based approach properly corresponds to the upload in the setup job and works reliably in container environments.
115-139: Good practices: Git configuration and debug output for container environments.The
git config --global --add safe.directoryconfiguration (line 123) is essential for container environments where the repository is owned by a different user. The extensive debug output will help troubleshoot container-specific issues.
224-232: LGTM: Robust handling of missing artifacts.The conditional check properly handles the case where no shards produced snapshot changes, providing clear feedback.
243-251: LGTM: Clean early exit when no artifacts to merge.The guard correctly handles the scenario where no shards produced changes, preventing unnecessary processing and potential errors.
97-102: No action needed—thecontainer_mode: trueparameter properly handles Python executable access in container execution environments.
103-105: The setup-frontend action already properly implements dependency caching. It uses actions/setup-node with cache: 'pnpm', which relies on actions/cache under the hood for caching global packages data. GitHub cache works in containers; caches are found and restored properly in containerized workflows. Dependency installation will benefit from caching across runs.Likely an incorrect or invalid review comment.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/ci-tests-e2e.yaml.github/workflows/pr-update-playwright-expectations.yaml
🧰 Additional context used
🧠 Learnings (16)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-21T06:04:12.562Z
Learning: Applies to browser_tests/**/*.spec.ts : Use E2E tests in `browser_tests/**/*.spec.ts` with Playwright framework
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
.github/workflows/ci-tests-e2e.yaml.github/workflows/pr-update-playwright-expectations.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml.github/workflows/pr-update-playwright-expectations.yaml
📚 Learning: 2025-12-21T06:04:12.562Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-21T06:04:12.562Z
Learning: Applies to browser_tests/**/*.spec.ts : Use E2E tests in `browser_tests/**/*.spec.ts` with Playwright framework
Applied to files:
.github/workflows/ci-tests-e2e.yaml.github/workflows/pr-update-playwright-expectations.yaml
📚 Learning: 2025-11-24T19:48:03.270Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: tests-ui/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:48:03.270Z
Learning: Applies to tests-ui/**/*.test.{js,ts,jsx,tsx} : Write tests for new features
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-13T05:54:35.779Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:17-22
Timestamp: 2025-12-13T05:54:35.779Z
Learning: In browser_tests tests for the Comfy-Org/ComfyUI_frontend repository, the `comfyPage.loadWorkflow()` method already handles all necessary synchronization and waiting. No additional `await comfyPage.nextFrame()` call is needed before taking screenshots after loading a workflow.
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-21T06:04:12.562Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-21T06:04:12.562Z
Learning: Run quality gates before committing: `pnpm lint`, `pnpm typecheck`, `pnpm knip`, and ensure relevant tests pass; never use `--no-verify` to bypass
Applied to files:
.github/workflows/ci-tests-e2e.yaml.github/workflows/pr-update-playwright-expectations.yaml
📚 Learning: 2025-11-24T19:47:56.371Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: src/lib/litegraph/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:56.371Z
Learning: Be sure to typecheck when you're done making a series of code changes using `pnpm typecheck`
Applied to files:
.github/workflows/ci-tests-e2e.yaml.github/workflows/pr-update-playwright-expectations.yaml
📚 Learning: 2025-11-24T19:48:03.270Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: tests-ui/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:48:03.270Z
Learning: Check tests-ui/README.md for test guidelines
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Check assets/ directory for test data when writing tests
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-11-24T19:48:03.270Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: tests-ui/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:48:03.270Z
Learning: Applies to tests-ui/**/*.test.{js,ts,jsx,tsx} : Use existing test utilities rather than writing custom utilities
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-13T05:34:22.209Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:7-15
Timestamp: 2025-12-13T05:34:22.209Z
Learning: In Playwright tests in the Comfy-Org/ComfyUI_frontend repository, the `toPass()` assertion method implements incremental backoff during retries, so increasing the timeout value (e.g., to 5000ms) is sufficient for handling async operations without being overly aggressive.
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-21T06:04:12.562Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-21T06:04:12.562Z
Learning: Applies to browser_tests/**/*.spec.ts : Do not use `waitForTimeout` in Playwright tests; use Locator actions and retrying assertions instead
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-16T17:41:58.464Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-16T17:41:58.464Z
Learning: Prerequisites for repository setup: Node.js >= 24, Git repository, available ports for dev server and storybook
Applied to files:
.github/workflows/ci-tests-e2e.yaml
📚 Learning: 2025-12-11T12:25:24.164Z
Learnt from: christian-byrne
Repo: Comfy-Org/ComfyUI_frontend PR: 7358
File: src/components/dialog/content/signin/SignUpForm.vue:45-54
Timestamp: 2025-12-11T12:25:24.164Z
Learning: The project has CI automation that handles code formatting (pnpm format) automatically, so manual formatting suggestions should not be provided in code reviews for the Comfy-Org/ComfyUI_frontend repository.
Applied to files:
.github/workflows/pr-update-playwright-expectations.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: deploy-and-comment
- GitHub Check: Socket Security: Pull Request Alerts
- GitHub Check: test
- GitHub Check: setup
- GitHub Check: collect
- GitHub Check: lint-and-format
🔇 Additional comments (13)
.github/workflows/pr-update-playwright-expectations.yaml (6)
66-73: LGTM! Artifact upload configuration is correct.The switch from cache to artifact-based propagation is properly implemented with appropriate retention period.
79-86: Container configuration looks correct.The container uses a properly versioned image (0.0.8), and the credentials are correctly configured for GHCR access. The
packages: readpermission is appropriate for pulling the container image.
103-107: LGTM! Server startup and dependency installation.Using the composite action for server startup and
--frozen-lockfilefor reproducible builds follows best practices.
117-148: Robust snapshot staging implementation.The script properly:
- Uses
set -euo pipefailfor strict error handling- Configures
git safe.directoryfor container environments- Handles both modified and untracked files
- Guards against missing files with
[ -f "$file" ] || continue- Provides clear shard-specific output
190-198: Good defensive check for missing artifacts.Properly handles the case where no shards produced snapshot changes, preventing workflow failures.
209-217: Consistent artifact existence check.This mirrors the check in the listing step, ensuring the merge step gracefully handles the no-changes case.
.github/workflows/ci-tests-e2e.yaml (7)
25-32: LGTM! Updated comment and artifact configuration.The comment now correctly states that containerized jobs will install dependencies without cache. The artifact configuration is appropriate.
39-46: Container configuration is correct and consistent.The container uses the same versioned image (0.0.8) as other test jobs, with proper GHCR credentials. The
packages: readpermission allows pulling the container image.
53-65: LGTM! Test job setup is well-structured.The sequence of checkout → download artifact → start server → install deps follows a logical order and uses the composite action for server startup.
67-70: Comment accurately describes test execution environment.The comment correctly notes that browsers are pre-installed in the container, which is the key benefit of using the custom CI container.
87-94: Consistent container configuration across test jobs.The non-sharded test job uses identical container configuration as the sharded tests, ensuring environment consistency.
100-117: LGTM! Non-sharded test setup mirrors sharded configuration.Same setup pattern as sharded tests ensures consistent test execution environment.
138-149: Merge-reports job correctly runs without container.This job only needs the Playwright CLI for
merge-reports, so running on the standard ubuntu-latest runner withsetup-frontendis the right approach. This avoids the overhead of pulling the full container image when browsers aren't needed.
| # Copy changed files to staging directory | ||
| mkdir -p /tmp/changed_snapshots_shard | ||
|
|
||
| # Copy only changed files, preserving directory structure | ||
| # Strip 'browser_tests/' prefix to avoid double nesting | ||
| echo "Copying changed files to staging directory..." | ||
| while IFS= read -r file; do | ||
| # Skip paths that no longer exist (e.g. deletions) | ||
| if [ ! -f "$file" ]; then | ||
| echo " → (skipped; not a file) $file" | ||
| continue | ||
| fi | ||
| # Remove 'browser_tests/' prefix | ||
| [ -f "$file" ] || continue | ||
| file_without_prefix="${file#browser_tests/}" | ||
| # Create parent directories | ||
| mkdir -p "/tmp/changed_snapshots_shard/$(dirname "$file_without_prefix")" | ||
| # Copy file | ||
| cp "$file" "/tmp/changed_snapshots_shard/$file_without_prefix" | ||
| echo " → $file_without_prefix" | ||
| done <<< "$changed_files" |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider adding error handling for cp command.
While the script uses set -euo pipefail, the cp command on line 147 could fail silently if the destination directory creation fails. The mkdir -p on line 146 should handle this, but consider adding explicit error messages.
🔎 Proposed improvement
while IFS= read -r file; do
[ -f "$file" ] || continue
file_without_prefix="${file#browser_tests/}"
- mkdir -p "/tmp/changed_snapshots_shard/$(dirname "$file_without_prefix")"
- cp "$file" "/tmp/changed_snapshots_shard/$file_without_prefix"
+ dest_dir="/tmp/changed_snapshots_shard/$(dirname "$file_without_prefix")"
+ mkdir -p "$dest_dir" || { echo "Failed to create $dest_dir"; exit 1; }
+ cp "$file" "/tmp/changed_snapshots_shard/$file_without_prefix" || { echo "Failed to copy $file"; exit 1; }
done <<< "$changed_files"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Copy changed files to staging directory | |
| mkdir -p /tmp/changed_snapshots_shard | |
| # Copy only changed files, preserving directory structure | |
| # Strip 'browser_tests/' prefix to avoid double nesting | |
| echo "Copying changed files to staging directory..." | |
| while IFS= read -r file; do | |
| # Skip paths that no longer exist (e.g. deletions) | |
| if [ ! -f "$file" ]; then | |
| echo " → (skipped; not a file) $file" | |
| continue | |
| fi | |
| # Remove 'browser_tests/' prefix | |
| [ -f "$file" ] || continue | |
| file_without_prefix="${file#browser_tests/}" | |
| # Create parent directories | |
| mkdir -p "/tmp/changed_snapshots_shard/$(dirname "$file_without_prefix")" | |
| # Copy file | |
| cp "$file" "/tmp/changed_snapshots_shard/$file_without_prefix" | |
| echo " → $file_without_prefix" | |
| done <<< "$changed_files" | |
| # Copy changed files to staging directory | |
| mkdir -p /tmp/changed_snapshots_shard | |
| while IFS= read -r file; do | |
| [ -f "$file" ] || continue | |
| file_without_prefix="${file#browser_tests/}" | |
| dest_dir="/tmp/changed_snapshots_shard/$(dirname "$file_without_prefix")" | |
| mkdir -p "$dest_dir" || { echo "Failed to create $dest_dir"; exit 1; } | |
| cp "$file" "/tmp/changed_snapshots_shard/$file_without_prefix" || { echo "Failed to copy $file"; exit 1; } | |
| done <<< "$changed_files" |
🤖 Prompt for AI Agents
.github/workflows/pr-update-playwright-expectations.yaml around lines 141 to
148: the cp command may fail (e.g., if destination creation unexpectedly fails)
without a clear error message; after each cp "$file"
"/tmp/changed_snapshots_shard/$file_without_prefix" check its exit status and on
failure print a descriptive error to stderr including the source and destination
paths and exit non‑zero (for example, use a conditional or || handler to echo
"Failed to copy $file to /tmp/changed_snapshots_shard/$file_without_prefix" >&2
and exit 1).
DrJKL
added
the
New Browser Test Expectations
|
Updating Playwright Expectations |
github-actions
bot
removed
the
New Browser Test Expectations
christian-byrne
added
needs-backport
## Summary Use a pre-built container image with all dependencies for Playwright E2E tests, eliminating ~130s setup time per shard. ## Changes - Use `ghcr.io/comfy-org/comfyui-ci-container:0.0.8` container for test jobs - Container includes: Playwright browsers, Node.js, pnpm, Python, ComfyUI backend (v0.5.1), all Python deps - Simplified setup: just copy devtools and start server (no cloning, no pip install, no browser setup) - Add `version-bump*` to `branches-ignore` to skip E2E tests for version bump PRs - Replace cache with artifacts (cache doesn't work inside containers) ## Benefits - ~130s faster per shard (no ComfyUI clone, no pip install, no browser download) - Consistent environment across all test jobs - Simpler workflow configuration ## Container Image Repository: https://github.com/comfy-org/comfyui-ci-container Image: `ghcr.io/comfy-org/comfyui-ci-container:0.0.8` ## Test plan - [x] Verify CI workflow runs with container - [x] Verify Playwright tests pass - [x] Verify snapshot updates work correctly --------- Co-authored-by: github-actions <[email protected]>
github-actions
bot
removed
the
needs-backport
## Summary Use a pre-built container image with all dependencies for Playwright E2E tests, eliminating ~130s setup time per shard. ## Changes - Use `ghcr.io/comfy-org/comfyui-ci-container:0.0.8` container for test jobs - Container includes: Playwright browsers, Node.js, pnpm, Python, ComfyUI backend (v0.5.1), all Python deps - Simplified setup: just copy devtools and start server (no cloning, no pip install, no browser setup) - Add `version-bump*` to `branches-ignore` to skip E2E tests for version bump PRs - Replace cache with artifacts (cache doesn't work inside containers) ## Benefits - ~130s faster per shard (no ComfyUI clone, no pip install, no browser download) - Consistent environment across all test jobs - Simpler workflow configuration ## Container Image Repository: https://github.com/comfy-org/comfyui-ci-container Image: `ghcr.io/comfy-org/comfyui-ci-container:0.0.8` ## Test plan - [x] Verify CI workflow runs with container - [x] Verify Playwright tests pass - [x] Verify snapshot updates work correctly --------- Co-authored-by: github-actions <[email protected]>
Backport of #7625 to `core/1.36` Automatically created by backport workflow. ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-7982-backport-core-1-36-CI-Use-custom-container-for-E2E-tests-2e76d73d36508150a21de5f6f1e0a49f) by [Unito](https://www.unito.io) Co-authored-by: Johnpaul Chiwetelu <[email protected]> Co-authored-by: github-actions <[email protected]>
Backport of #7625 to `cloud/1.36` Automatically created by backport workflow. ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-7983-backport-cloud-1-36-CI-Use-custom-container-for-E2E-tests-2e76d73d365081608f74da4b8add041e) by [Unito](https://www.unito.io) Co-authored-by: Johnpaul Chiwetelu <[email protected]> Co-authored-by: github-actions <[email protected]>
Summary
Use a pre-built container image with all dependencies for Playwright E2E tests, eliminating ~130s setup time per shard.
Changes
ghcr.io/comfy-org/comfyui-ci-container:0.0.8container for test jobsversion-bump*tobranches-ignoreto skip E2E tests for version bump PRsBenefits
Container Image
Repository: https://github.com/comfy-org/comfyui-ci-container
Image:
ghcr.io/comfy-org/comfyui-ci-container:0.0.8Test plan