Repository navigation
docs(testing): document epic confidence workflows - #6928
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
github-actions
Bot
added
scope: docs
📝 WalkthroughSummary by CodeRabbit
WalkthroughUpdated the testing playbook with CI lane mappings, expanded whole-path journey coverage and mutation guidance, and instructions for generating and retrieving product-surface coverage reports. ChangesTesting playbook guidance
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
risk: low
|
🚅 Deployed to the ironclaw-pr-6928 environment in ironclaw-ci-preview
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/internal/testing-playbook.md`:
- Around line 180-189: Update the docs validation guidance in the testing
playbook: remove the required docs/mint validation unless a valid docs/mint
project/config is added, and document the repository’s actual docs validation
command instead. Ensure the instructions specify running the command from the
correct directory and align with the authoritative workflow contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9bcf143e-cb16-4086-abd1-18a8e3d972c3
📒 Files selected for processing (1)
docs/internal/testing-playbook.md
| The authoritative workflow contract, required-check names, and scheduling | ||
| details live in [`.github/workflows/README.md`](../../.github/workflows/README.md). | ||
| Its "Known accepted gaps" section names deterministic and informational checks | ||
| that are deliberately not merge-gating. Re-derive the current lanes before | ||
| changing CI: | ||
|
|
||
| ```bash | ||
| rg -n "pull_request:|merge_group:|push:|schedule:|workflow_dispatch:" \ | ||
| .github/workflows | ||
| ``` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
cd docs
mint dev
mint broken-linksRepository: nearai/ironclaw
Length of output: 233
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Repo files matching CLAUDE/AGENTS/rules:\n'
git ls-files | rg '(^|/)(CLAUDE\.md|AGENTS\.md|\.claude/)' || true
printf '\nRelevant guideline lines:\n'
for f in CLAUDE.md AGENTS.md .claude/rules; do
[ -f "$f" ] && { printf '\n-- %s --\n' "$f"; rg -n "docs/\*\*|\*/testing-playbook|mint dev|mint broken-links" "$f" || true; }
done
printf '\nChanged/internal doc file reference:\n'
git diff --name-only 2>/dev/null | rg '^docs/internal/testing-playbook\.md$' || trueRepository: nearai/ironclaw
Length of output: 5407
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'AGENTS.md doc guideline section:\n'
nl -ba AGENTS.md | sed -n '1,120p'
printf '\nCLAUDE.md doc guideline section:\n'
nl -ba CLAUDE.md | sed -n '1,160p'
printf '\nCitation snippets:\n'
rg -n "docs/internal/|docs/\*\*|Test documentation changes|mint dev|mint broken-links|broken-links" AGENTS.md CLAUDE.md .claude/rules -S || trueRepository: nearai/ironclaw
Length of output: 226
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'AGENTS.md doc guideline section:\n'
awk '{printf "%6d\t%s\n", NR, $0}' AGENTS.md | sed -n '1,120p'
printf '\nCLAUDE.md doc guideline section:\n'
awk '{printf "%6d\t%s\n", NR, $0}' CLAUDE.md | sed -n '1,160p'
printf '\nCitation snippets:\n'
rg -n "docs/internal/|docs/\*\*|Test documentation changes|mint dev|mint broken-links|broken-links" AGENTS.md CLAUDE.md .claude/rules -S || trueRepository: nearai/ironclaw
Length of output: 19298
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'AGENTS.md doc guideline lines 150-190:\n'
awk '{printf "%6d\t%s\n", NR, $0}' AGENTS.md | sed -n '150,190p'
printf '\nMint-related files:\n'
git ls-files | rg '(^|/)mint(\.lock|\.json|mintlify\.json)?$|docs/package\.json|docs/mint' || trueRepository: nearai/ironclaw
Length of output: 2714
Run the required docs/mint validation.
docs/**/* requires mint dev and mint broken-links run from docs/. The repository currently has no docs/mint project/config, so this path has failing guardrail contract. Either add the Mint setup or replace this guidance with the actual docs validation command.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/internal/testing-playbook.md` around lines 180 - 189, Update the docs
validation guidance in the testing playbook: remove the required docs/mint
validation unless a valid docs/mint project/config is added, and document the
repository’s actual docs validation command instead. Ensure the instructions
specify running the command from the correct directory and align with the
authoritative workflow contract.
Source: Coding guidelines
🔎 Review · PR #6928
Submitted review →Reviewed the complete trusted base-to-head comparison. The documentation additions accurately reflect the repository’s CI lanes, journey/state-machine registries, regression enforcement, mutation tooling, and product-surface report workflow. No actionable issues found. Automatic · PR opened · attempt 1 of 3 · completed in 1m 29s Run details
|
![ironloopai[bot]](https://avatars.githubusercontent.com/in/4116490?s=60&v=4){kind=small}
There was a problem hiding this comment.
🔍 Review complete · PR #6928
✅ No actionable findings
Reviewed the complete trusted base-to-head comparison. The documentation additions accurately reflect the repository’s CI lanes, journey/state-machine registries, regression enforcement, mutation tooling, and product-surface report workflow. No actionable issues found.
Validation and technical details
- Inspected the full diff from refs/ironloop/base (457088c) to refs/ironloop/head (e0056d5).
- Cross-checked CI lane claims and accepted gaps against .github/workflows/README.md and relevant workflow definitions.
- Verified referenced journey, state-machine, regression-check, mutation-audit, and product-surface tooling exists and matches the documented interfaces.
- Confirmed all relative Markdown links in docs/internal/testing-playbook.md resolve.
- git diff --check completed successfully.
- Runtime execution of the Python coverage gates was unavailable because the checkout lacks its E2E virtual environment and dependencies (pytest/httpx); static inspection confirmed the documented commands match CI and script entry points.
- Base:
main - Head:
codex/docs-epic-testing-playbookate0056d5 - Run:
4e73670a-ef1a-4246-86fd-20c864d7e91b
This branch was successfully deployed
1 participant
Summary
Change Type
Linked Issue
Related #6524
Validation
cargo fmt --all -- --check— Not applicable: Markdown-only change.cargo clippy --all --benches --tests --examples --all-features -- -D warnings— Not applicable: no Rust or production code changed.cargo build— Not applicable: no compiled source changed.cargo test --features integrationif database-backed or integration behavior changed — Not applicable: no runtime or database behavior changed.git diff --check, and ran the mutation-harness self-tests.review-prorpr-shepherd --fixwas run before requesting review — structured Sol 5.6 medium review found one overstated merge-queue guarantee; it was corrected, and the final review returned no actionable findings.Test Strategy
User behavior:
Engineers can use one playbook to select the correct test seam, understand when evidence runs, extend journey/lifecycle coverage, promote failures into regressions, audit assertion strength, and generate the current product-surface matrix locally.
Risk areas:
Tests added or updated:
What the tests prove:
Commands run:
Security Impact
None. This changes engineer-facing documentation only and does not alter permissions, networking, secrets, file access, tool execution, or sandbox policy.
Reborn Trust-Boundary Checklist
serde(default)fields fail closed or have migration tests: N/A — no serialization changed.Database Impact
None. No schema, migration, PostgreSQL, or libSQL behavior changed.
Blast Radius
Documentation readers selecting Reborn test evidence and interpreting CI lanes. The main risk is a stale or overstated command/guarantee; concrete paths, commands, generated reports, and accepted CI gaps were verified against the PR head.
Pending workstreams #6883, #6884, and #6889 retain ownership of guidance for files and enforcement that have not landed on
main; this PR does not create broken links to their branch-only artifacts.Rollback Plan
Revert commit
e0056d5c1. No runtime or data rollback is required.Review Follow-Through
The first structured review identified an absolute statement that contradicted the CI contract's accepted post-merge-only Windows and benchmark checks. The table now scopes the guarantee to queue-covered checks and explicitly directs engineers to the accepted-gap section. Final structured review was clean.
Review track: A (docs/tests/chore)