Skip to content

test(webui): cover automation lifecycle E2E - #7059

Merged
think-in-universe merged 6 commits into
mainfrom
issue-7056-automation-lifecycle-e2e
Aug 5, 2026
Merged

think-in-universe merged 6 commits into
mainfrom
issue-7056-automation-lifecycle-e2e

Conversation

@italic-jinxin

Copy link
Copy Markdown
Contributor

Summary

  • Extends the existing Automation WebUI smoke journey from create and rename to the full lifecycle: create → rename → pause → reload → resume → reload → delete.
  • Uses the real Automation APIs without route mocking and confirms every mutation through API read-back using the immutable automation ID.
  • Verifies that paused and scheduled states persist across page reloads and expose the correct UI actions.
  • Confirms deletion in both the UI and API, with failure-safe cleanup to prevent test data from leaking into later scenarios.
  • Keeps the scenario in the existing WebUI smoke, coverage, and nightly Playwright lanes without additional CI configuration.

Linked Issue

Closes #7056

Validation

  • cd tests/e2e && pytest scenarios/test_reborn_webui_v2_smoke.py -k automation -q
    • 4 passed, 38 deselected
  • python -m py_compile tests/e2e/scenarios/test_reborn_webui_v2_smoke.py
  • git diff --check

Test Strategy

User behavior:

A user can create an Automation through Chat, rename it, pause and resume it across reloads, and delete it from the WebUI.

Risk areas:

  • Browser
  • Side effect
  • Persistence
  • Cross-component behavior

Tests added or updated:

  • Browser E2E: Extended the existing Automation smoke scenario.
  • Unit or contract: Not applicable; no production behavior changed.
  • Backend or runtime: Not applicable; the browser scenario exercises the existing served API.
  • Live canary: Not applicable; the flow uses the deterministic test model.

What the tests prove:

Every lifecycle mutation reaches the real API, survives reload where applicable, and is confirmed through API read-back. Deletion is verified by both exact-row UI absence and exact-ID API absence.

Security Impact

None. Test-only change with no authentication, authorization, or secret-handling changes.

Database Impact

None. No schema or migration changes.

Blast Radius

Limited to the existing Automation WebUI smoke E2E scenario.

Rollback Plan

Revert commit 998f6f982 to restore the previous create-and-rename-only coverage.


Review track: A

@italic-jinxin italic-jinxin added size: L 200-499 changed lines risk: low Changes to docs, tests, or low-risk modules contributor: core 20+ merged PRs labels Aug 3, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@railway-app

railway-app Bot commented Aug 3, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-7059 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw ✅ Success (View Logs) Web Aug 5, 2026 at 9:35 am

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-7059 August 3, 2026 12:42 Destroyed
@coderabbitai

coderabbitai Bot commented Aug 3, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ffcbf4d7-f461-4c8b-bac1-0fc2f09f0aa1

📥 Commits

Reviewing files that changed from the base of the PR and between 0ffd686 and a566145.

📒 Files selected for processing (1)
  • tests/CLAUDE.md

📝 Walkthrough

Summary by CodeRabbit

  • Tests
    • Expanded end-to-end coverage for the complete automation lifecycle.
    • Verified creation, renaming, pausing, resuming, persistence after reload, and deletion.
    • Added API verification, filtering checks, failed-run retry coverage, and failure-safe cleanup.
    • Improved automation state polling, absence checks, and timeout diagnostics.
    • Added coverage for pause/resume controls and deletion confirmation dialogs.

Walkthrough

The PR adds stable automation selectors, expands automation polling, and replaces the rename-only smoke test with creation, rename, pause, resume, reload, deletion, API verification, and failure-safe cleanup.

Changes

Automation lifecycle validation

Layer / File(s) Summary
Stable automation selectors
crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.tsx, crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.test.ts, tests/e2e/helpers.py
The automation panel exposes stable identifiers for pause/resume actions, deletion, and scoped delete dialogs. Frontend tests and E2E selectors verify these identifiers.
Automation polling support
tests/e2e/scenarios/test_reborn_webui_v2_smoke.py
Polling supports completed automations, unlimited runs, presence and absence checks, expected states, and timeout diagnostics.
Automation lifecycle scenario
tests/e2e/scenarios/test_reborn_webui_v2_smoke.py, tests/CLAUDE.md
The smoke test covers creation, renaming, pause and resume persistence across reloads, deletion confirmation, API verification, and conditional cleanup. The test documentation records this coverage.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Chat
  participant WebUI
  participant AutomationAPI
  Chat->>AutomationAPI: Create automation
  WebUI->>AutomationAPI: Rename automation
  WebUI->>AutomationAPI: Pause automation
  WebUI->>AutomationAPI: Read paused state after reload
  WebUI->>AutomationAPI: Resume automation
  WebUI->>AutomationAPI: Delete automation
  WebUI->>AutomationAPI: Verify absence
Loading

Suggested reviewers: serrrfirat

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title uses Conventional Commits style and clearly describes the automation lifecycle E2E coverage.
Description check ✅ Passed The description covers the change, linked issue, validation, test strategy, impacts, rollback, and review track; only minor template fields are omitted.
Linked Issues check ✅ Passed The changes satisfy the lifecycle, API read-back, reload persistence, cleanup, unmocked API, and CI-lane requirements in [#7056].
Out of Scope Changes check ✅ Passed The test, selector, helper, and documentation changes directly support the Automation lifecycle E2E objectives, with no unrelated scope identified.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size: XS < 10 changed lines (excluding docs) and removed size: L 200-499 changed lines labels Aug 3, 2026
@italic-jinxin

Copy link
Copy Markdown
Contributor Author

@claude review

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/scenarios/test_reborn_webui_v2_smoke.py`:
- Line 1278: Update the confirmation visibility wait in the delete scenario to
pass the same explicit 15000 ms timeout used by the other browser waits,
preserving the existing expect(confirmation) assertion.
- Around line 213-267: Consolidate the duplicated polling logic from
_wait_for_automation_id and _wait_for_automation_named into one shared poll
helper that accepts a predicate for matching the response data. Move the common
request URL, query parameters, status validation, timeout handling, sleep
cadence, and last-response tracking into that helper, then have both existing
callers provide their automation-specific predicates while preserving their
current return values and timeout assertions.
- Around line 1214-1277: Replace the rendered-name role locators for pause,
resume, delete, and the deletion confirmation dialog in the automation flow with
ID-keyed SEL_V2 selectors or stable test-ID locators. Update each action and
visibility assertion to use the immutable automation_id, while retaining
server-state checks and avoiding dependence on button or dialog label text.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 739c6484-52e5-4667-9e80-7369f75aab93

📥 Commits

Reviewing files that changed from the base of the PR and between 4b71aaa and 998f6f9.

📒 Files selected for processing (1)
  • tests/e2e/scenarios/test_reborn_webui_v2_smoke.py

Comment thread tests/e2e/scenarios/test_reborn_webui_v2_smoke.py Outdated
Comment thread tests/e2e/scenarios/test_reborn_webui_v2_smoke.py Outdated
Comment thread tests/e2e/scenarios/test_reborn_webui_v2_smoke.py Outdated
@ironloopai

ironloopai Bot commented Aug 3, 2026 •

Copy link
Copy Markdown
Contributor

🔎 Review · PR #7059

⚫ Cancelled · Target changed

The target changed before this Run could finish.

Automatic · PR opened · attempt 0 of 3 · cancelled after <1s

Run details
  • Repository: nearai/ironclaw
  • Base: main at 4b71aaa
  • Head: issue-7056-automation-lifecycle-e2e at 998f6f9
  • Created: Aug 3, 2026, 12:47 PM UTC
  • Updated: Aug 3, 2026, 12:47 PM UTC
  • Run: c8906963-1dc7-4985-b186-5dddfadcb844

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-7059 August 3, 2026 13:08 Destroyed
@github-actions github-actions Bot added size: S 10-49 changed lines and removed size: XS < 10 changed lines (excluding docs) labels Aug 3, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.test.ts (1)

233-258: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add coverage for the delete-dialog wrapper's stable selectors.

The "deletes only after confirming the shared dialog" test checks ConfirmDialog props directly through componentProps, but it does not assert the new wrapper div's data-testid="automation-delete-dialog" and data-automation-id attributes added in automation-detail-panel.tsx. tests/e2e/helpers.py's automation_delete_dialog_for selector depends on that wrapper. Use nativeProps(rendered, "div") (already used for <form> in this file) to assert those attributes here, so a regression is caught by this fast component test instead of only by the E2E lifecycle test.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.test.ts`
around lines 233 - 258, Extend the test “AutomationDetailPanel deletes only
after confirming the shared dialog” to inspect the rendered delete-dialog
wrapper via nativeProps(rendered, "div"), and assert its data-testid is
"automation-delete-dialog" and data-automation-id is "automation-alpha"
alongside the existing ConfirmDialog assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In
`@crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.test.ts`:
- Around line 233-258: Extend the test “AutomationDetailPanel deletes only after
confirming the shared dialog” to inspect the rendered delete-dialog wrapper via
nativeProps(rendered, "div"), and assert its data-testid is
"automation-delete-dialog" and data-automation-id is "automation-alpha"
alongside the existing ConfirmDialog assertions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 5fa45d4a-a940-4efe-a424-cd5c1e0aa677

📥 Commits

Reviewing files that changed from the base of the PR and between 998f6f9 and 650ec7e.

📒 Files selected for processing (4)
  • crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.test.ts
  • crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.tsx
  • tests/e2e/helpers.py
  • tests/e2e/scenarios/test_reborn_webui_v2_smoke.py

@italic-jinxin
italic-jinxin force-pushed the issue-7056-automation-lifecycle-e2e branch from 650ec7e to 462da6f Compare August 3, 2026 13:23
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-7059 August 3, 2026 13:23 Destroyed
@think-in-universe
think-in-universe added this pull request to the merge queue Aug 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 4, 2026
@italic-jinxin italic-jinxin self-assigned this Aug 5, 2026
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-7059 August 5, 2026 08:24 Destroyed
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/scenarios/test_reborn_webui_v2_smoke.py`:
- Around line 1270-1273: Update tests/CLAUDE.md to document the materially
expanded test_reborn_v2_automation_lifecycle_persists_from_ui scenario, revise
its coverage-map entry from rename-only coverage to the full automation
lifecycle, and maintain the §6 scenario counts to reflect the change.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 08bf4219-4c25-4591-a449-88f7928a9689

📥 Commits

Reviewing files that changed from the base of the PR and between c51a573 and 0ffd686.

📒 Files selected for processing (4)
  • crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.test.ts
  • crates/ironclaw_webui/frontend/src/pages/automations/components/automation-detail-panel.tsx
  • tests/e2e/helpers.py
  • tests/e2e/scenarios/test_reborn_webui_v2_smoke.py

Comment thread tests/e2e/scenarios/test_reborn_webui_v2_smoke.py
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-7059 August 5, 2026 09:27 Destroyed
@github-actions github-actions Bot added the scope: docs Documentation label Aug 5, 2026
@think-in-universe
think-in-universe added this pull request to the merge queue Aug 5, 2026
Merged via the queue into main with commit d8b5dd7 Aug 5, 2026
40 checks passed
@think-in-universe
think-in-universe deleted the issue-7056-automation-lifecycle-e2e branch August 5, 2026 10:40
l3ocifer pushed a commit to l3ocifer/frick-ironclaw that referenced this pull request Sep 3, 2026
* test(webui): cover automation lifecycle E2E

* test(webui): stabilize automation lifecycle E2E

* ci: map E2E paths to dedicated workflow

* chore(ci): drop superseded E2E planner mapping

* docs(test): update automation lifecycle coverage map

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-7059 — a5661456 Deployed Aug 5, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules scope: docs Documentation size: S 10-49 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Automation Lifecycle E2E

2 participants