Skip to content

fix(test-infra): repair the #6520 audit fallout — coverage-lane crash, blind auth suites, weakened guards - #6609

Merged
BenKurrek merged 7 commits into
mainfrom
claude/pr6520-testing-infra-repairs
Jul 24, 2026
Merged

BenKurrek merged 7 commits into
mainfrom
claude/pr6520-testing-infra-repairs

Conversation

@BenKurrek

Copy link
Copy Markdown
Collaborator

Summary

  • Repairs the testing-infrastructure fallout identified by the deep post-merge audit of fix(reborn): make extension readiness and channel delivery generic #6520 (the parts not already fixed by fix(live-qa): operator extension-configuration values as a sequence #6602/test(playwright): reconcile suite to the merged #6520 lifecycle and setup contracts #6603):
  • Coverage-lane crash (live on main): reborn_integration_extension_delivery SIGABRTs with a stack overflow under llvm-cov instrumentation since the fix(reborn): make extension readiness and channel delivery generic #6520 merge commit, killing Coverage (default) and Coverage (all-features) on every main push. The 400-line pairing-attribution journey's future overflows the 2 MiB test-thread stack when instrumentation inflates its frames — boxed via the same Box::pin + _impl pattern its sibling telegram_update_becomes_a_turn_and_a_coordinated_reply already uses.
  • E2E Coverage red (live on main): test_private_tool_installs_full_path fails with {"field":"client_action_id","validation_code":"missing_field"} — the fix(reborn): make extension readiness and channel delivery generic #6520 install contract requires the client gesture id and this scenario was never reconciled. Now sends client_action_id per the SPA contract (extensions-api.ts), same as the test(playwright): reconcile suite to the merged #6520 lifecycle and setup contracts #6603-reconciled specs.
  • Six integration suites ran in no PR lane: scripts/ci/reborn-coverage-int-tier-tests.sh discovered suites with find -maxdepth 1, which cannot see domain-folder bins — so all six tests/integration/auth/ suites (oauth_connect, oauth_popup_journeys, oauth_refresh, auth_gate, auth_failure, reopen_resume_through_gate) ran only in the push-to-main coverage workflow (itself crashed by the bug above; four of the six ran nowhere at all post-merge). Discovery is now registration-driven — every workspace [[test]] whose path sits under tests/integration/ is selected — so a suite cannot be registered without also being selected, in any directory shape. Also made the selector bash-3.2-portable (mapfile → plain string), so the guardrail runs on macOS dev machines; that fixes 12 previously-failing section-D harness assertions locally.
  • Weakened lifecycle guards re-armed: scenario_remove_then_absent_cross_thread's phase-4 guard checked absence of "installation_phase":"setup_needed" while fix(reborn): make extension readiness and channel delivery generic #6520's edit made the pre-remove state active — a stale search projection would read active and pass. The guard now asserts the wire contract directly: the installation_phase key is omitted entirely for a caller with no visible installation. And the retired Activate action's structural successor — an existing member's idempotent install retry reconciling setup_needed → active — had zero integration-tier coverage (setup_needed was never positively observed at that tier); a new scenario drives it on the shared store as a distinct actor.
  • Dead auth-canary config cleanup: AUTH_LIVE_FORCE_GOOGLE_REFRESH and the deliberate-expiry note documented a flow fix(reborn): make extension readiness and channel delivery generic #6520 deleted (expire_secret_in_db); the scopes example drifted from GOOGLE_SCOPE_DEFAULT.

Change Type

  • Bug fix
  • New feature
  • Refactor
  • Documentation
  • CI/Infrastructure
  • Security
  • Dependencies

Linked Issue

Related: post-merge audit of #6520; follow-ups #6602/#6603 fixed the canary wire shape and the Playwright shards, this PR covers the remaining regressions. No pre-existing issue tracks them.

Validation

  • cargo fmt --all -- --check
  • cargo clippy -p ironclaw_reborn_integration_tests --tests -- -D warnings (default) and --all-features — the only crate with Rust changes; both lanes clean
  • cargo build -p ironclaw
  • Relevant tests pass: see per-fix evidence below
  • cargo test --features integration — not applicable: no database-backed behavior changed (test/CI/docs-only diff)
  • Manual testing: local red→green reproductions for both live main breakages (below)
  • review-pr / pr-shepherd --fix — not run (agent session; full command evidence inline)

Test Strategy

User behavior: none changed — this PR touches tests, CI selection, and docs only. The production crates are untouched.

Risk areas:

  • Model behavior
  • Browser
  • Side effect
  • Persistence
  • Security or permissions
  • External provider
  • Cross-component behavior (CI lane selection; integration-suite coverage)

Tests added or updated:

  • Unit or contract: test-reborn-coverage.sh section D — new case D6 pins domain-folder-bin selection and unregistered-sibling exclusion; D1–D5 pass unchanged against the rewritten selector.
  • Reborn integration: scenario_existing_member_reinstall_reconciles_to_active (new, group_extensions, distinct actor via with_actor_id; positively pins cross-thread setup_needed, then the same-member idempotent-install reconciliation to active with no remove); scenario_remove_then_absent_cross_thread guard re-armed to the key-absence wire contract; extension_delivery.rs overflowing journey boxed (_impl extraction, in-file precedent).
  • Recorded fixture: none.
  • Browser E2E: test_reborn_private_tool_installs.py reconciled to the fix(reborn): make extension readiness and channel delivery generic #6520 install gesture contract.
  • Backend or runtime: none (no production code changed).
  • Live canary: none (docs-only canary changes; the product-side refresh-under-expiry proof remains a tracked follow-up, noted in ACCOUNTS.md).

What the tests prove:

  • The coverage lanes can run reborn_integration_extension_delivery under llvm-cov again (the exact previously-crashing invocation passes locally).
  • The private-tool-installs journey passes against the merged install contract through the real serve binary.
  • Any registered integration suite is selected by the coverage lanes regardless of directory shape, and the six auth suites are green before being wired in.
  • A remove that fails to propagate to the search projection can no longer pass the cross-thread guard (any surviving installation_phase fails it).
  • The Activate-successor reconciliation path (existing member, credential completed out-of-band, idempotent re-install → active) is pinned at the integration tier, cross-thread, on the shared store.

Commands run (local, all exit 0 unless noted):

  • cargo llvm-cov --no-report -p ironclaw_reborn_integration_tests --test reborn_integration_extension_delivery — red first (reproduced main's crash: thread 'unbound_telegram_actor_pairs_via_web_minted_code_then_turns_attribute_to_the_paired_user::case_1_libsql' has overflowed its stack, SIGABRT), then green after the fix (-- --skip case_2_postgres locally: no Postgres service; CI provides one — 18 passed / 0 failed).
  • cargo test --test reborn_integration_extension_delivery -- --skip case_2_postgres — green uninstrumented.
  • cargo test --test reborn_group_extensions — 15 passed / 0 failed, including the new scenario and the re-armed guard.
  • cargo test --test reborn_integration_oauth_connect --test reborn_integration_oauth_popup_journeys --test reborn_integration_oauth_refresh --test reborn_integration_auth_gate --test reborn_integration_auth_failure --test reborn_integration_reopen_resume_through_gate — 71 passed / 0 failed (proven green before wiring into CI lanes).
  • tests/e2e: pytest scenarios/test_reborn_private_tool_installs.py — red on main (the standing E2E Coverage failure), green with the fix through the real ironclaw serve binary.
  • bash scripts/ci/test-reborn-coverage.sh — all 14 section-D assertions pass (D1–D6); the 12 remaining failures are the pre-existing section-C fake-gh cases, identical on the pristine tree (which additionally fails all 12 D assertions there, because the old mapfile selector cannot run under macOS bash 3.2 at all).
  • bash scripts/ci/reborn-coverage-int-tier-tests.sh — emits 56 suites (was 50), including the six auth suites; every emitted name cross-checked against Cargo.toml.
  • shellcheck on both edited scripts — clean (the two remaining findings in the harness are pre-existing and identical at base).
  • bash scripts/pre-commit-safety.sh — pass.

Security Impact

None. No production code changed. The auth suites newly running in PR CI only increase enforcement of existing OAuth/gate behavior.

Reborn Trust-Boundary Checklist

N/A — no Reborn production/runtime/DB code changed (tests, CI scripts, and docs only).

Database Impact

None.

Blast Radius

CI coverage-lane selection (reborn-coverage-int-tier-tests.sh consumers: reborn-coverage-lane-run.sh, its 5-lane matrix in reborn-tests.yml), the Coverage (default)/Coverage (all-features)/E2E Coverage jobs on main, the reborn_group_extensions and reborn_integration_extension_delivery suites, and the auth-canary docs. Lane-shape note: the six auth suites join the four flat modulo-partitions (~1.5 suites/lane more); they run 0.01–4.2 s each uninstrumented locally, so lane duration impact is small.

Rollback Plan

Revert the PR; every change is test/CI/docs-scoped and independently revertible per commit. Reverting the selector commit alone returns the auth suites to their pre-PR blind spot (and re-breaks local D-section harness runs on macOS); reverting the extension_delivery commit re-crashes the main coverage lanes.

Review Follow-Through

Known follow-ups deliberately NOT in this PR (from the same audit): a restoration-tracking issue for the 15 quarantined live-canary replay fixtures; a product-side Google refresh-under-expiry canary proof (the harness-side deletion is now documented in ACCOUNTS.md); the ironclaw_product readiness-derivation duplication cluster; a PR/merge-queue trigger for the nightly-only Playwright suite; and a CI lane for the live-QA/canary Python unit suites plus a Python lint pass (ruff F811 would have caught #6520's duplicate Playwright helper). The live-QA harness unit-suite repair (test_run_live_qa.py, red on main) is in flight separately with the operator's local changes and is intentionally untouched here.


Review track: C (CI/Infrastructure)

🤖 Generated with Claude Code

https://claude.ai/code/session_01YGtnyQSPh8ouwypXioXWTc

BenKurrek and others added 5 commits July 23, 2026 22:36
…rflow under llvm-cov

Since the #6520 merge commit, main's Coverage (default) and
Coverage (all-features) jobs die in reborn_integration_extension_delivery:
unbound_telegram_actor_pairs_via_web_minted_code_then_turns_attribute_to_
the_paired_user's ~400-line journey future overflows the 2 MiB test-thread
stack once llvm-cov instrumentation inflates its frames (SIGABRT, 'has
overflowed its stack'). Box the future via the _impl extraction the sibling
telegram_update_becomes_a_turn_and_a_coordinated_reply already uses.

Red first: cargo llvm-cov --no-report -p ironclaw_reborn_integration_tests
--test reborn_integration_extension_delivery reproduced the exact CI crash
locally; green after this change (18 passed instrumented and
uninstrumented; the 2 postgres cases are filtered locally — no service).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGtnyQSPh8ouwypXioXWTc
…-tool-installs

test_private_tool_installs_full_path has failed on main's E2E Coverage lane
since #6520 merged: the install contract now requires the client gesture id
(parse_client_action_id, product_surface_inbound.rs) and this scenario's
_install helper still posted only package_ref, dying with
{"field":"client_action_id","validation_code":"missing_field"}.
Send one id per install gesture via the reborn_webui_harness helper — the
same reconciliation #6603 applied to the extensions-api specs.

Verified locally through the real serve binary: 1 passed (was the standing
red on main).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGtnyQSPh8ouwypXioXWTc
…st]] registration

The selector's find -maxdepth 1 walk could not see domain-folder bins, so
the six tests/integration/auth/ suites (oauth_connect, oauth_popup_journeys,
oauth_refresh, auth_gate, auth_failure, reopen_resume_through_gate) ran in
NO PR or merge-queue lane — their only executor was the push-to-main
coverage workflow, itself crashed since #6520. Discovery now selects every
workspace [[test]] whose path sits under tests/integration/, so a suite
cannot be registered without also being selected, whatever directory shape
it uses; a registered-but-deleted file fails the lane loudly.

Also drops the bash-4 mapfile so the guardrail runs on macOS dev machines
(the old selector could not execute under /bin/bash 3.2 — all 12 section-D
harness assertions failed locally at base), and de-stales the lane-runner's
hardcoded 34/27/7 suite counts.

Regression coverage: test-reborn-coverage.sh D6 pins domain-folder-bin
selection and unregistered-sibling exclusion; D1-D5 pass unchanged. All six
auth suites verified green locally (71 passed / 0 failed) before wiring
them into the lanes; new selector output cross-checked name-by-name against
Cargo.toml (56 suites, was 50).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGtnyQSPh8ouwypXioXWTc
…mber install reconciliation

Two integration-tier weakenings from #6520:

1. scenario_remove_then_absent_cross_thread's phase-4 guard asserted the
   absence of installation_phase:setup_needed — but the same PR seeds a
   credential in phase 1, so the pre-remove state is active and a stale
   search projection would read active and pass the guard. Assert the wire
   contract instead: the installation_phase key is omitted entirely for a
   caller with no visible installation, so ANY surviving phase now fails.
   (Non-vacuity is anchored by scenario_install_then_active positively
   pinning the key for installed entries.)

2. The retired Activate action's structural successor — an existing
   member's idempotent install retry reconciling setup_needed -> active
   (extension_lifecycle.rs's Some(existing) same-caller arm) — had no
   integration coverage, and setup_needed was never positively observed at
   this tier. New scenario drives it on the shared store as a DISTINCT
   actor (with_actor_id): install parks the credential gate, deny leaves
   the membership at setup_needed (observed cross-thread), a seeded
   credential completes setup, and the same member's re-install reconciles
   to active (observed cross-thread) with no remove in between. The
   distinct actor kills scenario-order coupling both ways.

Verified: cargo test --test reborn_group_extensions — 15 passed / 0 failed
including both changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGtnyQSPh8ouwypXioXWTc
…docs

AUTH_LIVE_FORCE_GOOGLE_REFRESH and the deliberate-expiry note documented
the expire_secret_in_db flow #6520 removed (the canary no longer reaches
into persistence); the flag is read nowhere. Point the scopes example at
the full-URL form matching run_live_canary.py's GOOGLE_SCOPE_DEFAULT, and
note that a product-side refresh-under-expiry proof remains a tracked
follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGtnyQSPh8ouwypXioXWTc
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@ironloopai

ironloopai Bot commented Jul 24, 2026 •

Copy link
Copy Markdown
Contributor

🔎 IronLoop Review Status

Head: 344bf0cd28c1b3dd11ad209b980b5a42aec0bf7b
Result: One or more review results were superseded by a newer PR head.
Next: Run @ironloopai review on the latest PR head.
Updated: 2026-07-24T03:27:08.402Z

Current reviewers:

Reviewer State Verdict Findings Last update
ironloop/common-reviewer (reviewer) Superseded N/A N/A 2026-07-24T02:55:51.095Z
Reviewer summaries
Reviewer Detail
ironloop/common-reviewer (reviewer) Superseded by a newer PR head. New head: 4e2314f. Previous verdict: Changes requested.
Recent activity
Time Reviewer State Detail
2026-07-24T02:37:26.639Z ironloop/common-reviewer (reviewer) Queued Accepted review request for head 2fe72a1.
2026-07-24T02:37:26.639Z ironloop/common-reviewer (reviewer) Queued Waiting for this reviewer lane to become available.
2026-07-24T02:37:26.980Z ironloop/common-reviewer (reviewer) Started Reviewer worker started.
2026-07-24T02:37:29.733Z ironloop/common-reviewer (reviewer) Workspace ready Prepared isolated checkout (merge_ref) at fce2104.
2026-07-24T02:40:51.618Z ironloop/common-reviewer (reviewer) Result captured Changes requested; 1 blocking finding.
2026-07-24T02:40:51.618Z ironloop/common-reviewer (reviewer) Completed Review completed and terminal status was persisted.
2026-07-24T02:55:51.095Z ironloop/common-reviewer (reviewer) Superseded A newer PR head replaced this review (4e2314f).
Available commands
  • @ironloopai help
  • @ironloopai agents
  • @ironloopai review
  • @ironloopai review --agent <agent>
Run metadata

Admission: webhook accepted the request and IronLoop persisted reviewer state before this projection.

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6609 July 24, 2026 02:37 Destroyed
@github-actions github-actions Bot added scope: docs Documentation size: M 50-199 changed lines risk: low Changes to docs, tests, or low-risk modules labels Jul 24, 2026
@coderabbitai

coderabbitai Bot commented Jul 24, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 7d7fbf12-dca5-498d-829f-6aac30ee939f

📥 Commits

Reviewing files that changed from the base of the PR and between 4e2314f and 344bf0c.

📒 Files selected for processing (2)
  • scripts/ci/reborn-coverage-int-tier-tests.sh
  • scripts/ci/test-reborn-coverage.sh

📝 Walkthrough

Summary by CodeRabbit

  • Improvements
    • Updated the Google live canary example configuration to use full Gmail/Calendar scope URLs and clarified access-token refresh behavior (and removed the prior deliberate-expiry option).
    • Refreshed coverage lane documentation to match the current lane partitioning approach.
  • Tests / CI
    • Improved integration test discovery to run only explicitly registered integration binaries.
    • Added a domain-folder bin discovery regression case.
    • Added a new group extension reinstall/reconciliation activation scenario and stabilized a WebChat install test request via client_action_id.
  • Bug Fixes
    • Cross-thread extension search now omits installation_phase after removal.

Walkthrough

Changes

Reborn test infrastructure and lifecycle coverage

Layer / File(s) Summary
Manifest-driven integration discovery
scripts/ci/reborn-coverage-int-tier-tests.sh, scripts/ci/test-reborn-coverage.sh, scripts/ci/reborn-coverage-lane-run.sh
Integration targets are extracted from registered Cargo.toml entries, D6 covers nested registered bins and excludes unregistered siblings, and lane comments describe current partitioning.
Google canary configuration and runbook
scripts/auth_live_canary/config.example.env, scripts/live-canary/ACCOUNTS.md
Google scopes use full URLs, forced refresh configuration is removed, and documentation describes preflight token refresh.
Extension install test harness updates
tests/e2e/scenarios/test_reborn_private_tool_installs.py, tests/integration/extension_delivery.rs
Install requests include stable client_action_id, and the Telegram pairing test delegates through a storage-aware helper.
Group extension lifecycle scenarios
tests/integration/group_extensions/*
Coverage adds setup-needed-to-active reinstall reconciliation and asserts removed installations omit installation_phase.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

  • nearai/ironclaw issue 6456 — Directly covers manifest-driven discovery of nested Reborn integration tests.

Possibly related PRs

Suggested reviewers: henrypark133, think-in-universe

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title uses Conventional Commits style and accurately summarizes the CI/test-infra fixes in the changeset.
Description check ✅ Passed The description matches the template and fills the required sections with concrete summary, validation, test strategy, and impact details.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the contributor: core 20+ merged PRs label Jul 24, 2026

@ironloopai ironloopai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ IronLoop Review: reviewer

Review at a glance

Verdict Blocking Notes Inline Head
❌ Changes requested 1 0 1 2fe72a16c7e9

Head: 2fe72a16c7e928f6e6ccf82c63f5817037c7008c
Next: Fix the blocking findings, push the PR branch, then re-run this reviewer.

Run details

Status: Current
Needs human: no
Needs validation: no

Summary

The registration-driven selector works for the current 56 targets, but its parser silently drops valid Cargo test registrations when path precedes name, undermining the PR’s coverage-selection guarantee.

Findings

Blocking: 1 / Notes: 0

Blocking findings

1. ❌ [MEDIUM] Make manifest discovery independent of key order

Location: scripts/ci/reborn-coverage-int-tier-tests.sh:48-50
Cargo TOML key order is not semantic, but this emits a target immediately when it sees path, only if name has already appeared. A valid [[test]] with path = "tests/integration/..." before name = "reborn_integration_..." is silently omitted (and can make discovery report no suites), so that suite receives no coverage-lane execution. Collect both fields for the whole block and emit at its boundary, then add a reversed-order fixture to the harness.

Developer follow-up

After fixing this feedback:

  1. Push the fix to this PR branch.
  2. Re-run this reviewer with @ironloopai review --agent reviewer if you only changed this reviewer's findings.
  3. Re-run all reviewers with @ironloopai review when the fix may affect multiple areas.

sub(/^name = "/, "", name)
sub(/"$/, "", name)
}
in_test && /^path = "tests\/integration\// && name != "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes name precedes path, although Cargo treats TOML key order as irrelevant. A valid test block with path first is silently skipped, returning the coverage blind spot this selector is meant to prevent. Collect both fields per [[test]] block before deciding whether to emit it, and add a reversed-order fixture.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4e2314f — the parser now buffers each [[test]] stanza and emits at the stanza boundary (next table header or EOF), so key order no longer matters. Harness case D6 gained a path-before-name stanza that failed against the old parser (verified red) and passes now; real-repo selector output is byte-identical (56 suites).

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/ci/reborn-coverage-int-tier-tests.sh`:
- Around line 40-52: The TOML registration scan in
scripts/ci/reborn-coverage-int-tier-tests.sh lines 40-52 depends on name
preceding path; buffer both fields within each [[test]] stanza and emit the name
when the stanza closes, regardless of key order. Add the requested D6 fixture
and selection assertion in scripts/ci/test-reborn-coverage.sh lines 933-937 to
cover path-before-name registrations.

In `@scripts/ci/test-reborn-coverage.sh`:
- Around line 933-937: Add a second valid [[test]] stanza to the D6 Cargo.toml
fixture with path before name, and update the corresponding assertions to verify
this entry is emitted. Keep the existing name-before-path fixture and assertions
intact.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a939cbab-6e82-441f-ac8e-178ebe4e10d6

📥 Commits

Reviewing files that changed from the base of the PR and between 6140ef2 and 2fe72a1.

📒 Files selected for processing (10)
  • scripts/auth_live_canary/config.example.env
  • scripts/ci/reborn-coverage-int-tier-tests.sh
  • scripts/ci/reborn-coverage-lane-run.sh
  • scripts/ci/test-reborn-coverage.sh
  • scripts/live-canary/ACCOUNTS.md
  • tests/e2e/scenarios/test_reborn_private_tool_installs.py
  • tests/integration/extension_delivery.rs
  • tests/integration/group_extensions/main.rs
  • tests/integration/group_extensions/scenario_existing_member_reinstall_reconciles_to_active.rs
  • tests/integration/group_extensions/scenario_remove_then_absent_cross_thread.rs

Comment thread scripts/ci/reborn-coverage-int-tier-tests.sh Outdated
Comment thread scripts/ci/test-reborn-coverage.sh
@github-actions

github-actions Bot commented Jul 24, 2026 •

Copy link
Copy Markdown
Contributor

Coverage ratchet

Ratchet mode: ENFORCING

RATCHET PASS: global
  observed: 86.28% (313353 / 363168 lines)
  floor:    86.27% (tolerance 0.5pp -> effective floor 85.77%)
  denominator: 363168 lines now vs 354049 at floor capture (+9119 lines, +2.58%) — not a material change

⚠️ 2 Reborn crate(s) have 0 int-tier coverage (target: 0) — ironclaw_prompt_envelope, ironclaw_scripts

Reborn integration-tier coverage

Line coverage (Reborn crates): 86.28% — 313353 / 363168 lines

Per-crate breakdown (59 crates, lowest-covered first)
Crate Line % Covered / Total
ironclaw_prompt_envelope 0% 0 / 88
ironclaw_scripts 0% 0 / 345
ironclaw_event_projections 43.31% 673 / 1554
ironclaw_observability 61.54% 16 / 26
ironclaw_telegram_v2_adapter 62.35% 631 / 1012
ironclaw_authorization 62.98% 609 / 967
ironclaw_dispatcher 64.17% 77 / 120
ironclaw_memory 69.2% 773 / 1117
ironclaw_trust 73.21% 664 / 907
ironclaw_capabilities 73.72% 2744 / 3722
ironclaw_filesystem 73.91% 4600 / 6224
ironclaw_wasm_limiter 74.6% 47 / 63
ironclaw_extractors 74.72% 538 / 720
ironclaw_mcp 76.2% 775 / 1017
ironclaw_projects 76.48% 400 / 523
ironclaw_reborn_cli 78.03% 10539 / 13506
ironclaw_telegram_extension 78.59% 962 / 1224
ironclaw_llm 78.73% 20989 / 26659
ironclaw_wasm 79.72% 735 / 922
ironclaw_process_sandbox 80.46% 671 / 834
ironclaw_memory_native 81.38% 3203 / 3936
ironclaw_first_party_extensions 82.26% 6624 / 8053
ironclaw_events 82.47% 1604 / 1945
ironclaw_processes 83.16% 933 / 1122
ironclaw_host_api 83.71% 9031 / 10789
ironclaw_reborn_identity 83.8% 450 / 537
ironclaw_secrets 83.8% 2550 / 3043
ironclaw_reborn_config 84.17% 1962 / 2331
ironclaw_auth 85.31% 4026 / 4719
ironclaw_common 85.53% 2252 / 2633
ironclaw_run_state 85.61% 458 / 535
ironclaw_triggers 85.92% 2783 / 3239
ironclaw_network 85.97% 913 / 1062
ironclaw_product 86.2% 18675 / 21664
ironclaw_reborn_event_store 86.51% 1251 / 1446
ironclaw_hooks 86.58% 9930 / 11469
ironclaw_threads 87.2% 4851 / 5563
ironclaw_extensions 87.38% 3775 / 4320
ironclaw_skills 87.58% 4470 / 5104
ironclaw_turns 88.08% 14316 / 16254
ironclaw_reborn_traces 88.13% 11986 / 13600
ironclaw_reborn_composition 88.28% 57401 / 65019
ironclaw_slack_extension 88.47% 1934 / 2186
ironclaw_host_runtime 88.59% 18559 / 20950
ironclaw_reborn_openai_compat 89.06% 3629 / 4075
ironclaw_extension_host 89.12% 3743 / 4200
ironclaw_approvals 90.18% 1598 / 1772
ironclaw_resources 90.85% 4477 / 4928
ironclaw_webui 90.93% 9038 / 9939
ironclaw_conversations 91.1% 3202 / 3515
ironclaw_event_streams 91.24% 1063 / 1165
ironclaw_runner 91.29% 17081 / 18711
ironclaw_loop_host 91.88% 16423 / 17874
ironclaw_attachments 93.06% 630 / 677
ironclaw_outbound 94.48% 4020 / 4255
ironclaw_agent_loop 94.88% 9837 / 10368
ironclaw_safety 95.15% 3749 / 3940
ironclaw_first_party_extension_ports 95.62% 3672 / 3840
ironclaw_runtime_policy 96.55% 811 / 840

This table itself is informational and never gates the PR on its own — not the percentage, not the per-crate holes, not the 0-coverage callout. A separate coverage ratchet (dry-run until enforce=true; see tests/integration/coverage-floor.toml) can fail the build on specific configured floors.

Exemptions (3 entry/entries excluded from the accounting above)
Module / Crate Reason Issue
crate: ironclaw_embeddings v1-only: consumed only by root ironclaw (src/app.rs, src/tools/builtin/memory.rs, src/workspace/mod.rs, src/config/{mod,embeddings}.rs); no crates/* dependents. Covered by "Tests (Legacy)". #5657
crate: ironclaw_gateway v1-only: consumed only by root ironclaw (src/channels/web/platform/static_files.rs, src/channels/web/handlers/frontend.rs); no crates/* dependents. Covered by "Tests (Legacy)". #5657
crate: ironclaw_tui v1-only: consumed only by root ironclaw (src/main.rs, src/channels/tui.rs); no crates/* dependents. Crate's own doc comment confirms it bridges INTO v1, not Reborn. Covered by "Tests (Legacy)". #5657

@railway-app

railway-app Bot commented Jul 24, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-6609 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw ✅ Success (View Logs) Web Jul 24, 2026 at 2:56 am

Review finding (ironloopai blocking + CodeRabbit major, PR #6609): the
selector emitted a name only when it appeared BEFORE the stanza's path
line, so a valid Cargo registration written path-first would be silently
skipped — recreating the blind-spot class this selector exists to prevent.
Cargo treats TOML key order as irrelevant; the parser now buffers each
[[test]] stanza and emits at the stanza boundary (next table header or
EOF).

Red first: harness case D6 extended with a path-before-name stanza failed
against the old parser, passes now; all 14 section-D assertions green and
the real-repo output is byte-identical (56 suites).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGtnyQSPh8ouwypXioXWTc
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6609 July 24, 2026 02:55 Destroyed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/ci/reborn-coverage-int-tier-tests.sh`:
- Around line 56-61: Update the TOML parsing patterns in
scripts/ci/reborn-coverage-int-tier-tests.sh at lines 56-61 to allow optional
whitespace around the name/path keys and equals sign, while preserving
quoted-value extraction. Add a compact no-space Cargo.toml assignment to the D6
fixture in scripts/ci/test-reborn-coverage.sh at lines 927-945 to verify the
parser includes such registrations.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b14b23d9-c338-484f-8e15-e05b2e106393

📥 Commits

Reviewing files that changed from the base of the PR and between 2fe72a1 and 4e2314f.

📒 Files selected for processing (2)
  • scripts/ci/reborn-coverage-int-tier-tests.sh
  • scripts/ci/test-reborn-coverage.sh

Comment thread scripts/ci/reborn-coverage-int-tier-tests.sh Outdated
…regexes

Second formatting-dependence finding on the selector (CodeRabbit, PR #6609):
the awk accepted only 'key = "value"' spacing, so a legal compact
'name="..."' stanza would be silently dropped. Rather than harden the regex
one format at a time, parse Cargo.toml with Python's stdlib tomllib — the
lane already hard-depends on python3 (scripts/ci/lib/reborn_coverage_lcov.py)
and ubuntu-latest + macOS both ship >=3.11 — so the selector accepts exactly
what Cargo accepts, closing the whole class (key order, spacing, comments).

Red first: D6 extended with a compact trailing-comment stanza failed against
the regex parser, passes now; all 14 section-D assertions green; real-repo
output byte-identical (56 suites); shellcheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGtnyQSPh8ouwypXioXWTc
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6609 July 24, 2026 03:27 Destroyed
@BenKurrek
BenKurrek merged commit 6d0da53 into main Jul 24, 2026
61 checks passed
@BenKurrek
BenKurrek deleted the claude/pr6520-testing-infra-repairs branch July 24, 2026 03:42
BenKurrek added a commit that referenced this pull request Jul 24, 2026
…ne (#6660)

* fix(coverage): declare RUST_MIN_STACK on the push-to-main coverage lane

`Code Coverage` has been red on every push to main for 20+ consecutive
commits, aborting with `has overflowed its stack` / `fatal runtime error:
stack overflow` (SIGABRT, exit 101) — on a *different* test each time as
unrelated PRs shifted which future sat deepest:

  096e8f8  unbound_telegram_actor_pairs_via_web_minted_code_… (extension_delivery)
  8f4d832  duplicate_and_restart_replay_converge_exactly_once::case_1 (extension_ingress)
  d06bde9  extension_install_survives_independent_reopen (durable)

Root cause is a workflow gap, not test depth. libtest gives each test
thread a 2 MiB stack. `reborn-tests.yml` splits this package's suites
across two jobs and gives each the headroom it needs —
`reborn-integration-coverage` carries 8 MiB (llvm-cov inflates the
integration harness's async frames; #6609) and `root-reborn-parity-tests`
carries 64 MiB (reborn_qa_smoke_scenarios_e2e drives whole turns on the
libtest stack, ~10 MiB uninstrumented). `coverage.yml` runs
`cargo llvm-cov --workspace`, i.e. BOTH tiers in one job, and declared
neither. Set it to the union's requirement, 64 MiB.

This also explains why the per-test fixes did not converge: the depth
lives in shared harness code (group build -> submit_turn -> composition),
so #6609's `Box::pin` lowered one test below the ceiling and the
next-deepest test simply became the new failure.

The controlled comparison at d06bde9: `Reborn integration coverage (1)`
ran reborn_integration_durable instrumented with RUST_MIN_STACK=8388608
and passed, while `Coverage (all-features)`/`Coverage (default)` ran the
same suite under the same instrumentation with no setting and SIGABRT'd.
Same code, same instrumentation — only the stack size differed.

Regression coverage: tests/coverage_lane_stack_headroom.rs pins the
invariant on both workflows, sized per tier (whole-workspace lanes need
64 MiB; integration-tier-only lanes need 8 MiB). Verified red before this
change (`coverage.yml:coverage … declares no job-level RUST_MIN_STACK`)
and green after. Mutation-tested three ways: a below-floor value, a
whole-workspace lane set to the integration tier's 8 MiB, and dropping
reborn-tests.yml's own value each fail the guard. Non-vacuity assertions
keep a renamed job or reworded `run:` line from silently emptying the scan.

Note: coverage.yml triggers only on `push: branches: [main]`, so this
PR's own CI cannot exercise the fixed lane — it is validated by the
guard test plus the CI evidence above, and proven by the next push to main.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ci): route the coverage-headroom guard into the Reborn root test lanes

The guard added in the previous commit never ran: `root-reborn-parity-tests`
gates on `has_reborn_tests`, and `classify-test-scope.sh`'s
`is_reborn_test_path` matches root suites by the `tests/reborn_*` prefix.
`tests/coverage_lane_stack_headroom.rs` did not match, so a PR touching only
it and a workflow classified as `has_reborn_tests=false` and skipped every
Reborn test lane — the guard was dead weight on exactly the PR shape it
exists to police (a workflow edit).

Caught on PR CI for this branch: `Reborn root tests` reported `skipping`.

Rename to `tests/reborn_coverage_lane_stack_headroom.rs`, matching the
convention every other root suite already uses. Verified with the real
staged file set: the classifier now reports `has_reborn_tests=true`, and the
suite passes under its new target name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-6609 — 344bf0cd Deployed Jul 24, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules scope: docs Documentation size: M 50-199 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant