Fix animated webp test and remove screenshot#7632
Merged
AustinMroz merged 2 commits intomainfrom Dec 19, 2025
Merged
Conversation
dosubot
bot
added
the
size:S
Contributor
📝 WalkthroughWalkthroughReplaced timing-sensitive, frame-by-frame and screenshot-based animated-webp tests with DOM-based image-count assertions; also added an explicit canvas dirty flag call (app.canvas.setDirty(true)) after wiring loader output to saver. Changes
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🎨 Storybook Build Status✅ Build completed successfully! ⏰ Completed at: 12/19/2025, 04:39:26 AM UTC 🔗 Links🎉 Your Storybook is ready for review! |
🎭 Playwright Test Results❌ Some tests failed ⏰ Completed at: 12/19/2025, 04:48:32 AM UTC 📈 Summary
📊 Test Reports by Browser
🎉 Click on the links above to view detailed test results for each browser configuration. |
Bundle Size ReportSummary
Category Glance Per-category breakdownApp Entry Points — 3.2 MB (baseline 3.2 MB) • ⚪ 0 BMain entry bundles and manifests
Graph Workspace — 996 kB (baseline 996 kB) • ⚪ 0 BGraph editor runtime, canvas, workflow orchestration
Views & Navigation — 6.54 kB (baseline 6.54 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces
Panels & Settings — 297 kB (baseline 297 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens
UI Components — 184 kB (baseline 184 kB) • ⚪ 0 BReusable component library chunks
Data & Services — 12.5 kB (baseline 12.5 kB) • ⚪ 0 BStores, services, APIs, and repositories
Utilities & Hooks — 1.86 kB (baseline 1.86 kB) • ⚪ 0 BHelpers, composables, and utility bundles
Vendor & Third-Party — 8.46 MB (baseline 8.46 MB) • ⚪ 0 BExternal libraries and shared vendor chunks
Other — 3.44 MB (baseline 3.44 MB) • ⚪ 0 BBundles that do not match a named category
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
browser_tests/tests/widget.spec.ts (1)
288-295: Fix inconsistent window object access.Line 291 correctly uses
window['app']to access the app instance, but Line 292 uses the bare identifierapp. Insidepage.evaluate(), all browser globals must be accessed viawindow. This inconsistency may cause aReferenceError.🔎 Proposed fix
([loadId, saveId]) => { // Set the output of the SaveAnimatedWEBP node to equal the loader node's image window['app'].nodeOutputs[saveId] = window['app'].nodeOutputs[loadId] - app.canvas.setDirty(true) + window['app'].canvas.setDirty(true) },
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
browser_tests/tests/widget.spec.ts(2 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (browser_tests/CLAUDE.md)
browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx}: Test user workflows in browser tests
Use Playwright fixtures for browser tests
Follow naming conventions for browser tests
Check assets/ directory for test data when writing tests
Prefer specific selectors in browser tests
Test across multiple viewports
Files:
browser_tests/tests/widget.spec.ts
browser_tests/**/*.spec.ts
📄 CodeRabbit inference engine (AGENTS.md)
browser_tests/**/*.spec.ts: E2E test files must use patternbrowser_tests/**/*.spec.tswith Playwright
Do not use waitForTimeout in Playwright - use Locator actions and retrying assertions
Use Playwright tags like@mobile,@2xfor relevant E2E tests - they are respected by config
Files:
browser_tests/tests/widget.spec.ts
🧠 Learnings (16)
📓 Common learnings
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:17-22
Timestamp: 2025-12-13T05:54:35.779Z
Learning: In browser_tests tests for the Comfy-Org/ComfyUI_frontend repository, the `comfyPage.loadWorkflow()` method already handles all necessary synchronization and waiting. No additional `await comfyPage.nextFrame()` call is needed before taking screenshots after loading a workflow.
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Prefer specific selectors in browser tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Do not write tests dependent on non-behavioral features like utility classes or styles
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Do not write change detector tests - do not just assert that defaults are certain values
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7303
File: src/components/topbar/CurrentUserPopover.test.ts:199-205
Timestamp: 2025-12-10T03:09:19.636Z
Learning: For test files in the Comfy-Org/ComfyUI_frontend repository: When writing tests, prefer selecting elements by accessible properties (text content, aria-label, role, accessible name) over data-testid attributes. This ensures tests verify actual user-facing behavior and accessibility compliance.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Create browser tests as appropriate after code changes
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Do not use waitForTimeout in Playwright - use Locator actions and retrying assertions
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Be parsimonious in testing - do not write redundant tests
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: tests-ui/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:48:03.270Z
Learning: Check tests-ui/README.md for test guidelines
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:7-15
Timestamp: 2025-12-13T05:34:22.209Z
Learning: In Playwright tests in the Comfy-Org/ComfyUI_frontend repository, the `toPass()` assertion method implements incremental backoff during retries, so increasing the timeout value (e.g., to 5000ms) is sufficient for handling async operations without being overly aggressive.
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Write tests for all changes, especially bug fixes to catch future regressions
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Prefer specific selectors in browser tests
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-13T05:54:35.779Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:17-22
Timestamp: 2025-12-13T05:54:35.779Z
Learning: In browser_tests tests for the Comfy-Org/ComfyUI_frontend repository, the `comfyPage.loadWorkflow()` method already handles all necessary synchronization and waiting. No additional `await comfyPage.nextFrame()` call is needed before taking screenshots after loading a workflow.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Do not write tests dependent on non-behavioral features like utility classes or styles
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Be parsimonious in testing - do not write redundant tests
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Do not write change detector tests - do not just assert that defaults are certain values
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-11-24T19:48:03.270Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: tests-ui/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:48:03.270Z
Learning: Applies to tests-ui/**/*.test.{js,ts,jsx,tsx} : Write tests for new features
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Use Playwright tags like `mobile`, `2x` for relevant E2E tests - they are respected by config
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Aim for behavioral coverage of critical and new features
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Test user workflows in browser tests
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-09T03:39:54.501Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7169
File: src/platform/remote/comfyui/jobs/jobTypes.ts:1-107
Timestamp: 2025-12-09T03:39:54.501Z
Learning: In the ComfyUI_frontend project, Zod is on v3.x. Do not suggest Zod v4 standalone validators (z.uuid, z.ulid, z.cuid2, z.nanoid) until an upgrade to Zod 4 is performed. When reviewing TypeScript files (e.g., src/platform/remote/comfyui/jobs/jobTypes.ts) validate against Zod 3 capabilities and avoid introducing v4-specific features; flag any proposal to upgrade or incorporate v4-only validators and propose staying with compatible 3.x patterns.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-13T11:03:11.264Z
Learnt from: christian-byrne
Repo: Comfy-Org/ComfyUI_frontend PR: 7416
File: src/stores/imagePreviewStore.ts:5-7
Timestamp: 2025-12-13T11:03:11.264Z
Learning: In the ComfyUI_frontend repository, lint rules require keeping 'import type' statements separate from non-type imports, even if importing from the same module. Do not suggest consolidating them into a single import statement. Ensure type imports remain on their own line (import type { ... } from 'module') and regular imports stay on separate lines.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T00:40:09.635Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7537
File: src/components/ui/button/Button.stories.ts:45-55
Timestamp: 2025-12-17T00:40:09.635Z
Learning: Prefer pure function declarations over function expressions (e.g., use function foo() { ... } instead of const foo = () => { ... }) for pure functions in the repository. Function declarations are more functional-leaning, offer better hoisting clarity, and can improve readability and tooling consistency. Apply this guideline across TypeScript files in Comfy-Org/ComfyUI_frontend, including story and UI component code, except where a function expression is semantically required (e.g., callbacks, higher-order functions with closures).
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-11T12:25:15.470Z
Learnt from: christian-byrne
Repo: Comfy-Org/ComfyUI_frontend PR: 7358
File: src/components/dialog/content/signin/SignUpForm.vue:45-54
Timestamp: 2025-12-11T12:25:15.470Z
Learning: This repository uses CI automation to format code (pnpm format). Do not include manual formatting suggestions in code reviews for Comfy-Org/ComfyUI_frontend. If formatting issues are detected, rely on the CI formatter or re-run pnpm format. Focus reviews on correctness, readability, performance, accessibility, and maintainability rather than style formatting.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-13T05:34:15.488Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:7-15
Timestamp: 2025-12-13T05:34:15.488Z
Learning: In Playwright tests for the ComfyUI frontend, the toPass() assertion uses incremental backoff during retries. When a test may involve async operations, increasing the timeout (e.g., to 5000 ms) can be sufficient instead of aggressively extending timeouts. Apply this understanding to tests under browser_tests/tests/; if not resolved, review the toPass() backoff behavior and ensure timeouts align with expected async completion without masking issues.
Applied to files:
browser_tests/tests/widget.spec.ts
🧬 Code graph analysis (1)
browser_tests/tests/widget.spec.ts (1)
src/scripts/app.ts (1)
app(1765-1765)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: deploy-and-comment
- GitHub Check: setup
- GitHub Check: test
- GitHub Check: collect
- GitHub Check: lint-and-format
🔇 Additional comments (1)
browser_tests/tests/widget.spec.ts (1)
265-265: Good restoration of the previously skipped test.The test is now active and uses a more reliable DOM-based assertion instead of screenshot comparisons, which aligns well with the PR objectives to fix the flaky animated webp test.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
browser_tests/tests/widget.spec.ts(2 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (browser_tests/CLAUDE.md)
browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx}: Test user workflows in browser tests
Use Playwright fixtures for browser tests
Follow naming conventions for browser tests
Check assets/ directory for test data when writing tests
Prefer specific selectors in browser tests
Test across multiple viewports
Files:
browser_tests/tests/widget.spec.ts
browser_tests/**/*.spec.ts
📄 CodeRabbit inference engine (AGENTS.md)
browser_tests/**/*.spec.ts: E2E test files must use patternbrowser_tests/**/*.spec.tswith Playwright
Do not use waitForTimeout in Playwright - use Locator actions and retrying assertions
Use Playwright tags like@mobile,@2xfor relevant E2E tests - they are respected by config
Files:
browser_tests/tests/widget.spec.ts
🧠 Learnings (20)
📓 Common learnings
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:17-22
Timestamp: 2025-12-13T05:54:35.779Z
Learning: In browser_tests tests for the Comfy-Org/ComfyUI_frontend repository, the `comfyPage.loadWorkflow()` method already handles all necessary synchronization and waiting. No additional `await comfyPage.nextFrame()` call is needed before taking screenshots after loading a workflow.
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Create browser tests as appropriate after code changes
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Write tests for all changes, especially bug fixes to catch future regressions
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Be parsimonious in testing - do not write redundant tests
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Do not write tests dependent on non-behavioral features like utility classes or styles
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Aim for behavioral coverage of critical and new features
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Prefer specific selectors in browser tests
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-11-24T19:48:03.270Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: tests-ui/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:48:03.270Z
Learning: Applies to tests-ui/**/*.test.{js,ts,jsx,tsx} : Write tests for new features
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to tests-ui/**/*.test.ts : Do not write change detector tests - do not just assert that defaults are certain values
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-13T05:54:35.779Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:17-22
Timestamp: 2025-12-13T05:54:35.779Z
Learning: In browser_tests tests for the Comfy-Org/ComfyUI_frontend repository, the `comfyPage.loadWorkflow()` method already handles all necessary synchronization and waiting. No additional `await comfyPage.nextFrame()` call is needed before taking screenshots after loading a workflow.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Use Playwright tags like `mobile`, `2x` for relevant E2E tests - they are respected by config
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Follow naming conventions for browser tests
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : Do not use waitForTimeout in Playwright - use Locator actions and retrying assertions
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-13T05:34:15.488Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7415
File: browser_tests/tests/mobileBaseline.spec.ts:7-15
Timestamp: 2025-12-13T05:34:15.488Z
Learning: In Playwright tests for the ComfyUI frontend, the toPass() assertion uses incremental backoff during retries. When a test may involve async operations, increasing the timeout (e.g., to 5000 ms) can be sufficient instead of aggressively extending timeouts. Apply this understanding to tests under browser_tests/tests/; if not resolved, review the toPass() backoff behavior and ensure timeouts align with expected async completion without masking issues.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-11-24T19:47:22.909Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: browser_tests/CLAUDE.md:0-0
Timestamp: 2025-11-24T19:47:22.909Z
Learning: Applies to browser_tests/**/*.{e2e,spec}.{ts,tsx,js,jsx} : Use Playwright fixtures for browser tests
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T01:01:03.609Z
Learnt from: CR
Repo: Comfy-Org/ComfyUI_frontend PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T01:01:03.609Z
Learning: Applies to browser_tests/**/*.spec.ts : E2E test files must use pattern `browser_tests/**/*.spec.ts` with Playwright
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-12T23:02:37.473Z
Learnt from: Myestery
Repo: Comfy-Org/ComfyUI_frontend PR: 7422
File: .github/workflows/pr-update-playwright-expectations.yaml:131-135
Timestamp: 2025-12-12T23:02:37.473Z
Learning: In the `.github/workflows/pr-update-playwright-expectations.yaml` workflow in the Comfy-Org/ComfyUI_frontend repository, the snapshot update process is intentionally scoped to only add and update snapshot images. Deletions of snapshot files are handled explicitly outside this workflow and should not be suggested as part of this automation.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-09T03:39:54.501Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7169
File: src/platform/remote/comfyui/jobs/jobTypes.ts:1-107
Timestamp: 2025-12-09T03:39:54.501Z
Learning: In the ComfyUI_frontend project, Zod is on v3.x. Do not suggest Zod v4 standalone validators (z.uuid, z.ulid, z.cuid2, z.nanoid) until an upgrade to Zod 4 is performed. When reviewing TypeScript files (e.g., src/platform/remote/comfyui/jobs/jobTypes.ts) validate against Zod 3 capabilities and avoid introducing v4-specific features; flag any proposal to upgrade or incorporate v4-only validators and propose staying with compatible 3.x patterns.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-13T11:03:11.264Z
Learnt from: christian-byrne
Repo: Comfy-Org/ComfyUI_frontend PR: 7416
File: src/stores/imagePreviewStore.ts:5-7
Timestamp: 2025-12-13T11:03:11.264Z
Learning: In the ComfyUI_frontend repository, lint rules require keeping 'import type' statements separate from non-type imports, even if importing from the same module. Do not suggest consolidating them into a single import statement. Ensure type imports remain on their own line (import type { ... } from 'module') and regular imports stay on separate lines.
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-17T00:40:09.635Z
Learnt from: DrJKL
Repo: Comfy-Org/ComfyUI_frontend PR: 7537
File: src/components/ui/button/Button.stories.ts:45-55
Timestamp: 2025-12-17T00:40:09.635Z
Learning: Prefer pure function declarations over function expressions (e.g., use function foo() { ... } instead of const foo = () => { ... }) for pure functions in the repository. Function declarations are more functional-leaning, offer better hoisting clarity, and can improve readability and tooling consistency. Apply this guideline across TypeScript files in Comfy-Org/ComfyUI_frontend, including story and UI component code, except where a function expression is semantically required (e.g., callbacks, higher-order functions with closures).
Applied to files:
browser_tests/tests/widget.spec.ts
📚 Learning: 2025-12-11T12:25:15.470Z
Learnt from: christian-byrne
Repo: Comfy-Org/ComfyUI_frontend PR: 7358
File: src/components/dialog/content/signin/SignUpForm.vue:45-54
Timestamp: 2025-12-11T12:25:15.470Z
Learning: This repository uses CI automation to format code (pnpm format). Do not include manual formatting suggestions in code reviews for Comfy-Org/ComfyUI_frontend. If formatting issues are detected, rely on the CI formatter or re-run pnpm format. Focus reviews on correctness, readability, performance, accessibility, and maintainability rather than style formatting.
Applied to files:
browser_tests/tests/widget.spec.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup
- GitHub Check: test
- GitHub Check: lint-and-format
- GitHub Check: collect
🔇 Additional comments (1)
browser_tests/tests/widget.spec.ts (1)
296-298: Excellent fix: DOM-based assertion is more reliable.The change from screenshot-based testing to DOM element counting effectively addresses the flaky test issue with animated WebP frames. The chained locator approach is clean, and both past review concerns have been resolved:
- ✅
awaitis now present for the async assertion- ✅ Deprecated
>>selector syntax replaced with modern chained locators
| ([loadId, saveId]) => { | ||
| // Set the output of the SaveAnimatedWEBP node to equal the loader node's image | ||
| window['app'].nodeOutputs[saveId] = window['app'].nodeOutputs[loadId] | ||
| app.canvas.setDirty(true) |
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Prefer consistent global accessor pattern.
Within the same page.evaluate block, line 291 uses window['app'] while line 292 uses app. Both work, but consistent style improves readability.
🔎 Proposed fix for consistency
- app.canvas.setDirty(true)
+ window['app'].canvas.setDirty(true)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| app.canvas.setDirty(true) | |
| window['app'].canvas.setDirty(true) |
🤖 Prompt for AI Agents
In browser_tests/tests/widget.spec.ts around line 292, the code uses
app.canvas.setDirty(true) while the previous line in the same page.evaluate
block uses window['app']; make the accessor consistent by replacing the plain
global reference with the same window['app'] pattern (i.e., call
window['app'].canvas.setDirty(true)) so both lines use the same global accessor
style.
DrJKL
approved these changes
Dec 19, 2025
Collaborator
Author
|
Test failure is unrelated. |
Contributor
The bug that motivated this test case was one in which animated WEBP would not animate in the LoadImage (it would just sit on the first frame forever), so this may be slightly inaccurate. There's definitely a better way to test, "did the frame change at some point across some period of time" but no idea how to do that without it being inherently flaky. |
Collaborator
Author
|
🤔 I'll make a note to return in the future. Perhaps 3 screenshots taken at half the animation interval and then testing that not all 3 frames are equal? |
Yourz
pushed a commit
that referenced
this pull request
Dec 24, 2025
Remove screenshot expectation and restore test. We care that it's displaying an image not what frame of the image is being displayed. ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-7632-Fix-animated-webp-test-and-remove-screenshot-2ce6d73d365081a59d38dc9d82fa1a51) by [Unito](https://www.unito.io)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove screenshot expectation and restore test.
We care that it's displaying an image not what frame of the image is being displayed.
┆Issue is synchronized with this Notion page by Unito