Repository navigation
test(e2e): inventory provider capability coverage - #6526
Merged
Merged
Conversation
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Contributor
🔎 IronLoop Review StatusHead: Current reviewers:
Reviewer summaries
Recent activity
Available commands
Run metadataAdmission: webhook accepted the request and IronLoop persisted reviewer state before this projection. |
📝 WalkthroughWalkthroughAdds a manifest-backed provider capability inventory, typed Google operation cases with provider readback, exact-name Emulate trace filtering, and two completeness tests. The inventory scenario is wired into the WebUI v2 smoke workflow and documented in the E2E coverage guidance. ChangesProvider capability coverage
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant E2ESmoke
participant Reborn
participant MockLLM
participant Emulate
participant ProviderOracle
E2ESmoke->>Reborn: execute provider operation case
Reborn->>MockLLM: request inline trace response
MockLLM-->>Reborn: capability_info and provider tool call
Reborn->>Emulate: execute provider tool
Emulate-->>Reborn: operation result
Reborn-->>ProviderOracle: completed preview and replay metadata
ProviderOracle->>Emulate: verify provider state
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
railway-app
Bot
temporarily deployed
to
ironclaw-ci-preview / ironclaw-pr-6526
July 22, 2026 22:02
Destroyed
github-actions
Bot
added
scope: ci
![ironloopai[bot]](https://avatars.githubusercontent.com/in/4116490?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ IronLoop Review: reviewer
Review at a glance
| Verdict | Blocking | Notes | Inline | Head |
|---|---|---|---|---|
| ✅ Approved | 0 | 0 | 0 | f98251cb6dba |
Head: f98251cb6dbab2d3741af7d2484f60c53b216d54
Next: No reviewer action needed.
Run details
Status: Current
Needs human: no
Needs validation: no
Summary
Approved. Reviewed the focused stacked-layer change (1 commit; 7 files; 309 additions, 63 deletions) across the CI wiring, inventory source, helper, and replay/full-path consumers. The manifest inventory, trace evidence gate, and executable full-path selection are consistent.
Findings
None.
Developer follow-up
After fixing this feedback:
- Push the fix to this PR branch.
- Re-run this reviewer with
@ironloopai review --agent reviewerif you only changed this reviewer's findings. - Re-run all reviewers with
@ironloopai reviewwhen the fix may affect multiple areas.
9 of 21 tasks
|
🚅 Deployed to the ironclaw-pr-6526 environment in ironclaw-ci-preview
|
* test(e2e): add typed provider operation cases * test(e2e): address provider operation review
78 of 84 tasks
…rage-inventory # Conflicts: # tests/e2e/scenarios/test_reborn_qa_trace_full_path.py
railway-app
Bot
temporarily deployed
to
ironclaw-ci-preview / ironclaw-pr-6526
July 23, 2026 07:49
Destroyed
Contributor
Coverage ratchetReborn integration-tier coverageLine coverage (Reborn crates): 86.32% — 309596 / 358644 lines Per-crate breakdown (62 crates, lowest-covered first)
This table itself is informational and never gates the PR on its own — not the percentage, not the per-crate holes, not the 0-coverage callout. A separate coverage ratchet (dry-run until enforce=true; see tests/integration/coverage-floor.toml) can fail the build on specific configured floors. Exemptions (3 entry/entries excluded from the accounting above)
|
This was referenced Jul 23, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Change Type
Linked Issue
Part of #6524
Validation
cargo fmt --all -- --check— Not applicable: no Rust files changed by this PR.cargo clippy --all --benches --tests --examples --all-features -- -D warnings— Not applicable: no Rust files changed by this PR.cargo build— Rebornironclawbinary built successfully after merging currentmain.cargo test --features integrationif database-backed or integration behavior changed — Not applicable: no database or product integration behavior changed.review-prorpr-shepherd --fixwas run before requesting review — iterativeautoreviewcompleted with no actionable findings before the merge-conflict update.Test Strategy
User behavior: Developers get an actionable CI failure when the shipped provider surface expands without an explicit test classification, instead of relying on harvested traces to discover coverage gaps later.
Risk areas:
Tests added or updated:
test_provider_capability_inventory.pyproves exact manifest/inventory equality, non-overlapping classifications, waiver metadata, and executable evidence for tested operations.What the tests prove: Every statically shipped first-party provider capability is classified exactly once; adding or renaming one fails CI until its coverage status is owned. A capability cannot be marked tested merely by editing metadata: it must have either a harvested full-path call or a typed operation case, and the same registry drives full-path execution.
Commands run:
Results:
51 passed in 1.37s61 files102 passed in 79.02smain)Security Impact
None. This is test metadata and validation only; it does not alter production permissions, credentials, provider traffic, file access, tool execution, or sandbox policy.
Reborn Trust-Boundary Checklist
N/A: test-harness-only change; no trust-bearing production types, runtime behavior, serialization, queues, errors, or sandbox boundaries changed.
Database Impact
None.
Blast Radius
Limited to the harvested-QA/Emulate E2E lane. Incorrect manifest parsing or classification could fail that lane; product binaries and runtime behavior are unchanged. Runtime impact is lightweight inventory validation plus four typed cases inside the existing single-build provider process.
Rollback Plan
Revert the inventory and provider-operation commits to restore trace-local Emulate classification. No data migration or compatibility step is required.
Review Follow-Through
The 92 waivers are intentionally visible debt owned by #6524, not claims of coverage. Follow-up provider-operation PRs should move capabilities from the waiver list to
testedonly when their executable Emulate/full-path evidence lands.Review track: C (CI/test infrastructure)