Skip to content

fix(e2e): stabilize offline extensions scenario - #6343

Merged
serrrfirat merged 1 commit into
mainfrom
codex/fix-reborn-playwright-offline-navigation
Jul 20, 2026
Merged

serrrfirat merged 1 commit into
mainfrom
codex/fix-reborn-playwright-offline-navigation

Conversation

@serrrfirat

Copy link
Copy Markdown
Collaborator

Summary

  • make the offline extensions scenario return a deterministic active LLM-provider snapshot
  • wait for the provider state to render before taking Chromium offline
  • preserve the original assertions that both extension catalog requests are attempted and the retry flow recovers

Root cause

Scheduled run https://github.com/nearai/ironclaw/actions/runs/29717067979 failed only in the legacy-settings-extensions shard. The offline scenario started on Settings without synchronizing the first-run provider query. When that query resolved with no active provider before the Extensions click, the onboarding gate redirected the SPA to Welcome instead of mounting Extensions. The test then waited for an alert that could never render. The aggregate Reborn Playwright job failed because that shard failed.

Verification

  • focused scenario: 1 passed in 3.09s
  • full test_reborn_webui_v2_legacy_extensions.py: 43 passed in 19.39s
  • git diff --check

@ironloopai

ironloopai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown
Contributor

🔎 IronLoop Review Status

Head: 162cdc311480c2b7f4f2bcfad31383dd355c2d10
Result: Reviewer output needs human attention or validation.
Next: Review the flagged rows before merging.
Updated: 2026-07-20T09:34:50.688Z

Current reviewers:

Reviewer State Verdict Findings Last update
ironloop/common-reviewer (reviewer) Completed Needs validation 0 blocking findings / 0 notes; needs validation 2026-07-20T09:34:50.679Z
Reviewer summaries
Reviewer Detail
ironloop/common-reviewer (reviewer) Needs validation; 0 blocking findings; No actionable defect found in the focused E2E stabilization. Runtime validation is still required.
Recent activity
Time Reviewer State Detail
2026-07-20T09:31:18.802Z ironloop/common-reviewer (reviewer) Queued Accepted review request for head 162cdc3.
2026-07-20T09:31:18.802Z ironloop/common-reviewer (reviewer) Queued Waiting for this reviewer lane to become available.
2026-07-20T09:31:19.012Z ironloop/common-reviewer (reviewer) Started Reviewer worker started.
2026-07-20T09:31:22.853Z ironloop/common-reviewer (reviewer) Workspace ready Prepared isolated checkout (merge_ref) at 46e5372.
2026-07-20T09:34:50.679Z ironloop/common-reviewer (reviewer) Result captured Needs validation; 0 blocking findings.
2026-07-20T09:34:50.679Z ironloop/common-reviewer (reviewer) Completed Review completed and terminal status was persisted.
Available commands
  • @ironloopai help
  • @ironloopai agents
  • @ironloopai review
  • @ironloopai review --agent <agent>
Run metadata

Admission: webhook accepted the request and IronLoop persisted reviewer state before this projection.

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6343 July 20, 2026 09:31 Destroyed
@github-actions github-actions Bot added size: XS < 10 changed lines (excluding docs) risk: low Changes to docs, tests, or low-risk modules contributor: core 20+ merged PRs labels Jul 20, 2026
@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • Tests
    • Enhanced end-to-end coverage for offline extension catalog behavior.
    • Added validation that LLM provider information loads correctly before testing offline settings behavior.
    • Improved verification of the unavailable catalog message and related request attempts.

Walkthrough

The legacy extensions E2E test now mocks the LLM providers endpoint, waits for its response during settings navigation, and verifies the OpenAI provider card before continuing with offline extension catalog assertions.

Changes

Legacy extensions E2E flow

Layer / File(s) Summary
Mock and validate provider setup
tests/e2e/scenarios/test_reborn_webui_v2_legacy_extensions.py
The test mocks /api/webchat/v2/llm/providers, waits for the response on /settings, and asserts the OpenAI provider card is visible before testing offline catalog behavior.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • nearai/ironclaw#5375: Adds related E2E coverage that mocks LLM endpoints for provider-management flows.

Suggested reviewers: ilblackdragon

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title uses conventional-commit style and matches the test-only fix to stabilize the offline extensions scenario.
Description check ✅ Passed Mostly complete: summary, root cause, and verification are present, but several template sections like change type, security impact, and rollback are omitted.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the end-to-end test test_reborn_legacy_extensions_offline_attempts_catalog_requests to mock the LLM providers API endpoint (/api/webchat/v2/llm/providers) with a mock OpenAI provider. It registers this route handler, waits for the response during page navigation, and asserts that the corresponding LLM provider card is visible. There are no review comments, and I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@ironloopai ironloopai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ IronLoop Review: reviewer

Review at a glance

Verdict Blocking Notes Inline Head
⚠️ Needs validation 0 0 0 162cdc311480

Head: 162cdc311480c2b7f4f2bcfad31383dd355c2d10
Next: Human review or validation is required before merging.

Run details

Status: Current
Needs human: no
Needs validation: yes

Summary

No actionable defect found in the focused E2E stabilization. Runtime validation is still required.

Findings

None.

Developer follow-up

After fixing this feedback:

  1. Push the fix to this PR branch.
  2. Re-run this reviewer with @ironloopai review --agent reviewer if you only changed this reviewer's findings.
  3. Re-run all reviewers with @ironloopai review when the fix may affect multiple areas.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/scenarios/test_reborn_webui_v2_legacy_extensions.py`:
- Around line 716-740: Update handle_llm_providers to pass the provider payload
through the json parameter of route.fulfill, removing the manual json.dumps
conversion and redundant content_type argument while preserving the existing
response structure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2c80ff6c-3ad4-4163-9610-0bdcdd52b4bd

📥 Commits

Reviewing files that changed from the base of the PR and between c291098 and 162cdc3.

📒 Files selected for processing (1)
  • tests/e2e/scenarios/test_reborn_webui_v2_legacy_extensions.py

Comment on lines +716 to +740
async def handle_llm_providers(route):
await route.fulfill(
status=200,
content_type="application/json",
body=json.dumps(
{
"providers": [
{
"id": "openai",
"description": "OpenAI API",
"adapter": "open_ai_completions",
"base_url": "https://api.openai.test/v1",
"default_model": "mock-model",
"builtin": True,
"api_key_set": True,
"api_key_required": True,
"base_url_required": False,
"accepts_api_key": True,
}
],
"active": {"provider_id": "openai", "model": "mock-model"},
}
),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

🧩 Analysis chain

🌐 Web query:

Does Playwright Python's Route.fulfillaccept ajson keyword argument?

💡 Result:

Yes, Playwright Python's Route.fulfill method accepts a json keyword argument [1][2]. This argument was added in version 1.29 and allows you to provide a dictionary that will be serialized as a JSON response [1][2]. When this argument is used, Playwright automatically sets the content-type header to application/json if it has not been set otherwise [1][2].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

file='tests/e2e/scenarios/test_reborn_webui_v2_legacy_extensions.py'

printf '\n== imports and json usage ==\n'
grep -nE '^(import json|from json|import .*json|from .* import .*json)' "$file" || true
printf '\n'
grep -n '\bjson\.' "$file" || true

printf '\n== surrounding lines ==\n'
sed -n '700,760p' "$file"

Repository: nearai/ironclaw

Length of output: 3172


Use json= in route.fulfill here. Route.fulfill already serializes dict payloads and sets the JSON content type, so json.dumps(...) plus content_type="application/json" is redundant.

🧰 Tools
🪛 ast-grep (0.44.1)

[info] 719-737: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
{
"providers": [
{
"id": "openai",
"description": "OpenAI API",
"adapter": "open_ai_completions",
"base_url": "https://api.openai.test/v1",
"default_model": "mock-model",
"builtin": True,
"api_key_set": True,
"api_key_required": True,
"base_url_required": False,
"accepts_api_key": True,
}
],
"active": {"provider_id": "openai", "model": "mock-model"},
}
)
Note: [CWE-116] Improper Encoding or Escaping of Output.

(use-jsonify)

🪛 Ruff (0.15.21)

[warning] 716-716: Missing return type annotation for private function handle_llm_providers

Add return type annotation: None

(ANN202)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/scenarios/test_reborn_webui_v2_legacy_extensions.py` around lines
716 - 740, Update handle_llm_providers to pass the provider payload through the
json parameter of route.fulfill, removing the manual json.dumps conversion and
redundant content_type argument while preserving the existing response
structure.

@railway-app

railway-app Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-6343 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw ✅ Success (View Logs) Web Jul 20, 2026 at 9:47 am

@github-actions

Copy link
Copy Markdown
Contributor

Coverage ratchet

Ratchet mode: ENFORCING

RATCHET PASS: global
  observed: 86.21% (319818 / 370969 lines)
  floor:    85.3% (tolerance 0.5pp -> effective floor 84.8%)
  denominator: 370969 lines now vs 320188 at floor capture (+50781 lines, +15.86%) — material change (>5%)

⚠️ 2 Reborn crate(s) have 0 int-tier coverage (target: 0) — ironclaw_prompt_envelope, ironclaw_scripts

Reborn integration-tier coverage

Line coverage (Reborn crates): 86.21% — 319818 / 370969 lines

Per-crate breakdown (65 crates, lowest-covered first)
Crate Line % Covered / Total
ironclaw_prompt_envelope 0% 0 / 88
ironclaw_scripts 0% 0 / 345
ironclaw_runtime_policy 33.84% 89 / 263
ironclaw_event_projections 43.31% 673 / 1554
ironclaw_observability 61.54% 16 / 26
ironclaw_authorization 62.46% 604 / 967
ironclaw_dispatcher 62.88% 83 / 132
ironclaw_mcp 64.89% 595 / 917
ironclaw_triggers 65.44% 2142 / 3273
ironclaw_filesystem 67.78% 3957 / 5838
ironclaw_channel_host 68.65% 219 / 319
ironclaw_memory 69.2% 773 / 1117
ironclaw_reborn_migration 71.64% 1551 / 2165
ironclaw_trust 72.88% 661 / 907
ironclaw_wasm_limiter 74.6% 47 / 63
ironclaw_reborn_event_store 74.67% 958 / 1283
ironclaw_extractors 74.72% 538 / 720
ironclaw_capabilities 75.72% 2096 / 2768
ironclaw_projects 76.48% 400 / 523
ironclaw_reborn_cli 77% 10247 / 13307
ironclaw_llm 78.36% 20306 / 25915
ironclaw_product_context 78.57% 11 / 14
ironclaw_telegram_extension 80.18% 4842 / 6039
ironclaw_wasm_product_adapters 80.36% 1448 / 1802
ironclaw_process_sandbox 80.65% 671 / 832
ironclaw_first_party_extensions 81.06% 5965 / 7359
ironclaw_memory_native 81.17% 3195 / 3936
ironclaw_events 81.95% 1594 / 1945
ironclaw_network 82.98% 673 / 811
ironclaw_reborn_identity 83.59% 433 / 518
ironclaw_processes 83.76% 939 / 1121
ironclaw_secrets 83.79% 2548 / 3041
ironclaw_wasm 84.44% 1069 / 1266
ironclaw_auth 84.81% 3233 / 3812
ironclaw_product_workflow 84.91% 11031 / 12992
ironclaw_reborn_config 85.2% 2055 / 2412
ironclaw_run_state 85.61% 458 / 535
ironclaw_channel_delivery 85.79% 1383 / 1612
ironclaw_common 86.13% 1714 / 1990
ironclaw_threads 86.93% 4708 / 5416
ironclaw_turns 86.95% 14934 / 17175
ironclaw_slack_v2_adapter 87.3% 1491 / 1708
ironclaw_skills 87.58% 4470 / 5104
ironclaw_product_adapter_registry 88.06% 531 / 603
ironclaw_product_adapters 88.1% 3384 / 3841
ironclaw_reborn_traces 88.2% 11946 / 13544
ironclaw_hooks 88.35% 10075 / 11404
ironclaw_host_api 88.65% 4389 / 4951
ironclaw_host_runtime 88.7% 18140 / 20451
ironclaw_reborn_openai_compat 89.21% 3778 / 4235
ironclaw_webui 89.33% 7700 / 8620
ironclaw_extensions 89.38% 2971 / 3324
ironclaw_runner 89.65% 17453 / 19469
ironclaw_telegram_v2_adapter 89.7% 2717 / 3029
ironclaw_reborn_composition 90.09% 74860 / 83091
ironclaw_approvals 90.18% 1598 / 1772
ironclaw_conversations 90.39% 3123 / 3455
ironclaw_event_streams 90.82% 1009 / 1111
ironclaw_resources 91.65% 4476 / 4884
ironclaw_loop_host 92.28% 15997 / 17336
ironclaw_attachments 93.06% 630 / 677
ironclaw_agent_loop 94.95% 9416 / 9917
ironclaw_safety 95.09% 3682 / 3872
ironclaw_outbound 95.52% 3451 / 3613
ironclaw_first_party_extension_ports 95.62% 3672 / 3840

This table itself is informational and never gates the PR on its own — not the percentage, not the per-crate holes, not the 0-coverage callout. A separate coverage ratchet (dry-run until enforce=true; see tests/integration/coverage-floor.toml) can fail the build on specific configured floors.

Exemptions (3 entry/entries excluded from the accounting above)
Module / Crate Reason Issue
crate: ironclaw_embeddings v1-only: consumed only by root ironclaw (src/app.rs, src/tools/builtin/memory.rs, src/workspace/mod.rs, src/config/{mod,embeddings}.rs); no crates/* dependents. Covered by "Tests (Legacy)". #5657
crate: ironclaw_gateway v1-only: consumed only by root ironclaw (src/channels/web/platform/static_files.rs, src/channels/web/handlers/frontend.rs); no crates/* dependents. Covered by "Tests (Legacy)". #5657
crate: ironclaw_tui v1-only: consumed only by root ironclaw (src/main.rs, src/channels/tui.rs); no crates/* dependents. Crate's own doc comment confirms it bridges INTO v1, not Reborn. Covered by "Tests (Legacy)". #5657

@serrrfirat
serrrfirat merged commit 792da03 into main Jul 20, 2026
65 checks passed
@serrrfirat
serrrfirat deleted the codex/fix-reborn-playwright-offline-navigation branch July 20, 2026 10:36
@coderabbitai coderabbitai Bot mentioned this pull request Aug 3, 2026
12 tasks done

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-6343 — 162cdc31 Deployed Jul 20, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules size: XS < 10 changed lines (excluding docs)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant