test: fix flaky timeout in channel-management beforeEach navigation - #41385
test: fix flaky timeout in channel-management beforeEach navigation#41385KevLehman wants to merge 4 commits into
Conversation
The beforeEach hook navigates to /home before every test and could burn the entire 60s test timeout when the CI runner is slow: page.goto waits for the 'load' event (observed taking ~30s), and the client boot can get stuck on the loading screen, leaving the Home heading invisible until the test times out (as seen in 'should unignore user1 messages'). Navigate with waitUntil: 'domcontentloaded' so the hook waits on the Home heading instead of full asset loading, bound the first wait to 15s, and recover from a stuck boot with a single reload before waiting again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GYTXKLTT79eSBmX2sz1WeE
|
Looks like this PR is not ready to merge, because of the following issues:
Please fix the issues and try again If you have any trouble, please check the PR guidelines |
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📜 Recent review details⏰ Context from checks skipped due to timeout. (3)
WalkthroughThe e2e home setup now retries ChangesHome navigation reliability
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #41385 +/- ##
===========================================
+ Coverage 68.21% 68.29% +0.07%
===========================================
Files 3968 3968
Lines 154740 154771 +31
Branches 27825 27802 -23
===========================================
+ Hits 105557 105694 +137
+ Misses 44392 44279 -113
- Partials 4791 4798 +7
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
…ation Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GYTXKLTT79eSBmX2sz1WeE
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/meteor/tests/e2e/channel-management.spec.ts`:
- Around line 24-27: Remove the explanatory comment immediately before the
expect retry block in the channel-management test, leaving the retry
implementation unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b2f62762-be3a-49e2-b54b-e1daa2fefb1a
📒 Files selected for processing (2)
apps/meteor/tests/e2e/channel-management.spec.tsapps/meteor/tests/e2e/page-objects/home-channel.ts
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: cubic · AI code reviewer
- GitHub Check: Hacktron Security Check
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx,js}
📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)
**/*.{ts,tsx,js}: Write concise, technical TypeScript/JavaScript with accurate typing in Playwright tests
Avoid code comments in the implementation
Files:
apps/meteor/tests/e2e/page-objects/home-channel.tsapps/meteor/tests/e2e/channel-management.spec.ts
apps/meteor/tests/e2e/page-objects/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)
Utilize existing page objects pattern from
apps/meteor/tests/e2e/page-objects/
Files:
apps/meteor/tests/e2e/page-objects/home-channel.ts
apps/meteor/tests/e2e/**/*.{ts,spec.ts}
📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)
apps/meteor/tests/e2e/**/*.{ts,spec.ts}: Store commonly used locators in variables/constants for reuse
Follow Page Object Model pattern consistently in Playwright tests
Files:
apps/meteor/tests/e2e/page-objects/home-channel.tsapps/meteor/tests/e2e/channel-management.spec.ts
**/*.spec.ts
📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)
**/*.spec.ts: Use descriptive test names that clearly communicate expected behavior in Playwright tests
Use.spec.tsextension for test files (e.g.,login.spec.ts)
Files:
apps/meteor/tests/e2e/channel-management.spec.ts
apps/meteor/tests/e2e/**/*.spec.ts
📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)
apps/meteor/tests/e2e/**/*.spec.ts: All test files must be created inapps/meteor/tests/e2e/directory
Avoid usingpage.locator()in Playwright tests - always prefer semantic locators such aspage.getByRole(),page.getByLabel(),page.getByText(), orpage.getByTitle()
Usetest.beforeAll()andtest.afterAll()for setup/teardown in Playwright tests
Usetest.step()for complex test scenarios to improve organization in Playwright tests
Group related tests in the same file
Utilize Playwright fixtures (test,page,expect) for consistency in test files
Prefer web-first assertions (toBeVisible,toHaveText, etc.) in Playwright tests
Useexpectmatchers for assertions (toEqual,toContain,toBeTruthy,toHaveLength, etc.) instead ofassertstatements in Playwright tests
Usepage.waitFor()with specific conditions instead of hardcoded timeouts in Playwright tests
Implement proper wait strategies for dynamic content in Playwright tests
Maintain test isolation between test cases in Playwright tests
Ensure clean state for each test execution in Playwright tests
Ensure tests run reliably in parallel without shared state conflicts
Files:
apps/meteor/tests/e2e/channel-management.spec.ts
🧠 Learnings (7)
📚 Learning: 2025-12-16T17:29:40.430Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 37834
File: apps/meteor/tests/e2e/page-objects/fragments/admin-flextab-emoji.ts:12-22
Timestamp: 2025-12-16T17:29:40.430Z
Learning: In all page-object files under apps/meteor/tests/e2e/page-objects/, import expect from ../../utils/test (Playwright's async expect) instead of from Jest. Jest's expect is synchronous and incompatible with web-first assertions like toBeVisible, which can cause TypeScript errors.
Applied to files:
apps/meteor/tests/e2e/page-objects/home-channel.ts
📚 Learning: 2026-02-24T19:39:42.247Z
Learnt from: juliajforesti
Repo: RocketChat/Rocket.Chat PR: 38493
File: apps/meteor/tests/e2e/page-objects/fragments/message.ts:7-7
Timestamp: 2026-02-24T19:39:42.247Z
Learning: In RocketChat e2e tests, avoid using data-qa attributes to locate elements. Prefer semantic locators such as getByRole, getByLabel, getByText, getByTitle and ARIA-based selectors. Apply this rule to all TypeScript files under apps/meteor/tests/e2e to improve test reliability, accessibility, and maintainability.
Applied to files:
apps/meteor/tests/e2e/page-objects/home-channel.tsapps/meteor/tests/e2e/channel-management.spec.ts
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In the Rocket.Chat repository, do not reference Biome lint rules in code review feedback. Biome is not used even if biome.json exists; only reference Biome rules if there is explicit, project-wide usage documented. For TypeScript files, review lint implications without Biome guidance unless the project enables Biome rules.
Applied to files:
apps/meteor/tests/e2e/page-objects/home-channel.tsapps/meteor/tests/e2e/channel-management.spec.ts
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In this repository (RocketChat/Rocket.Chat), Biome lint rules are not used even if a biome.json exists. When reviewing TypeScript files (e.g., packages/ui-voip/src/providers/useMediaSession.ts), ensure lint suggestions do not reference Biome-specific rules. Rely on general ESLint/TypeScript lint rules and project conventions instead.
Applied to files:
apps/meteor/tests/e2e/page-objects/home-channel.tsapps/meteor/tests/e2e/channel-management.spec.ts
📚 Learning: 2026-05-06T12:21:44.083Z
Learnt from: juliajforesti
Repo: RocketChat/Rocket.Chat PR: 40256
File: apps/meteor/client/components/CreateDiscussion/CreateDiscussion.tsx:121-149
Timestamp: 2026-05-06T12:21:44.083Z
Learning: Field wrappers in rocket.chat/fuselage-forms (Field, FieldLabel, FieldRow, FieldError, FieldHint) auto-create htmlFor/id associations, aria-describedby, and role="alert" for errors. Do not manually set htmlFor, id, aria-describedby, or role attributes when using these wrappers. This automatic wiring does not apply to plain rocket.chat/fuselage components, which require explicit ID wiring per the accessibility docs. In code reviews, prefer using fuselage-forms wrappers for form fields and verify there is no unnecessary manual ID/aria wiring in files that use these wrappers. If a component uses plain fuselage components, ensure proper id wiring as per docs.
Applied to files:
apps/meteor/tests/e2e/page-objects/home-channel.tsapps/meteor/tests/e2e/channel-management.spec.ts
📚 Learning: 2026-02-24T19:22:48.358Z
Learnt from: juliajforesti
Repo: RocketChat/Rocket.Chat PR: 38493
File: apps/meteor/tests/e2e/omnichannel/omnichannel-send-pdf-transcript.spec.ts:66-67
Timestamp: 2026-02-24T19:22:48.358Z
Learning: In Playwright end-to-end tests (e.g., under apps/meteor/tests/e2e/...), prefer locating elements by translated text (getByText) and ARIA roles (getByRole) over data-qa attributes. If translation values change, update the corresponding test locators accordingly. Never use data-qa locators. This guideline applies to all Playwright e2e test specs in the repository and helps keep tests robust to UI text changes and accessible semantics.
Applied to files:
apps/meteor/tests/e2e/channel-management.spec.ts
📚 Learning: 2026-03-06T18:10:15.268Z
Learnt from: tassoevan
Repo: RocketChat/Rocket.Chat PR: 39397
File: packages/gazzodown/src/code/CodeBlock.spec.tsx:47-68
Timestamp: 2026-03-06T18:10:15.268Z
Learning: In tests (especially those using testing-library/dom/jsdom) for Rocket.Chat components, the HTML <code> element has an implicit ARIA role of 'code'. Therefore, screen.getByRole('code') or screen.findByRole('code') will locate <code> elements even without a role attribute. Do not flag findByRole('code') as invalid in reviews; prefer using the implicit role instead of adding role="code" unless necessary for accessibility.
Applied to files:
apps/meteor/tests/e2e/channel-management.spec.ts
🔇 Additional comments (1)
apps/meteor/tests/e2e/page-objects/home-channel.ts (1)
191-192: LGTM!
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GYTXKLTT79eSBmX2sz1WeE
An unbounded toPass() retries until the overall test timeout, which reproduces the opaque 'timeout exceeded in beforeEach' failure when the client boot is stuck for good. A 30s budget still allows two full navigation attempts and fails with a clear toPass timeout error instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GYTXKLTT79eSBmX2sz1WeE
|
I suppose that something really wrong was happening during this run, and I quite don't agree with this approach. If we're assuming our app can take this amount of time and we're going to wait and add a timeout for every single time such situation happens, we're going to have the most delayed continuous integration over and over again |
|
Agreed that something was genuinely wrong in that run — the server logs show it healthy and idle while the client sat on the loading screen for 55+ seconds, so this looks like the client boot occasionally hanging (same failure mode #41319 hit in On CI speed, this change actually goes the other direction:
So the intent isn't "assume the app can be slow and wait longer" — total wait actually went down (30s bound vs 60s before), and the one retry only exists because a stuck boot is unrecoverable without a fresh navigation. Happy to file an issue for the underlying stuck-boot behavior so it gets root-caused properly, and to adjust the shape of this fix if you'd prefer a different approach. Generated by Claude Code |
|
Closing — the root cause behind this flake was found and fixed in #41399 (ARCH-2272). I traced the failing run (job 87223567263) via the Playwright trace: all 33 boot requests returned 200 in a ~3s burst, then ~58s of complete network silence with the loading spinner still up until the test timeout. The trace console showed the real error — That is exactly the truncation #41399 fixes at the source (the traefik/Go If we still want the retry as defense-in-depth against a genuinely slow boot, worth reopening as a small |
7 participants
Proposed changes (including videos or screenshots)
Fixes a flaky timeout in
channel-management.spec.tswhere the suite's sharedbeforeEachhook could consume the entire 60s test timeout before the test body ever started, e.g. in this run:Diagnosis (from the CI logs):
page.goto('/home')and then waits for the "Home" heading. The timing showsgotospent nearly its full 30s navigation budget waiting for the pageloadevent, and the heading then never appeared in the remaining ~28s — the test timeout killed the wait while it was still pending (hence theTarget page, context or browser has been closederror).This matches the failure mode already documented in #41319 (
preview-public-channelflake): on a busy CI runner the client boot is slow or gets stuck on the loading screen, and no amount of waiting recovers it — only a fresh navigation does.Changes:
expect(async () => { ... }).toPass()— Playwright's built-in retry mechanism, already used elsewhere in the e2e suite — so a stuck boot triggers a freshgotoattempt instead of a single wait exhausting the whole test timeout.waitUntil: 'domcontentloaded'(same pattern used insidebar-menu.spec.ts), so each attempt doesn't spend its budget waiting for full asset loading — what actually matters is the Home heading, which is waited on explicitly with a bounded 15s timeout per attempt.HomeChannel.waitForHome()now accepts an optionaltimeoutparam (backward-compatible for all other callers).No changeset needed — test-only change.
Issue(s)
Steps to test or reproduce
The failure is CI-load dependent and not deterministically reproducible locally. Run the suite to verify no regression:
The suite still passes on the happy path (the retry only kicks in when the Home page fails to render within the bounded wait).
Further comments
Alternatives considered: raising the per-test timeout would hide genuine slow-downs and still fail when the boot is truly stuck; moving
gotoout ofbeforeEach(the #41319 approach) doesn't map here since there are no per-test API mutations racing the boot — every test needs the same fresh navigation; a try/catch with a manualpage.reload()fallback works but hand-rolls whattoPass()already expresses declaratively.🤖 Generated with Claude Code
https://claude.ai/code/session_01GYTXKLTT79eSBmX2sz1WeE
Summary by CodeRabbit