Skip to content

docs: add engineer-facing TDD testing playbook - #6411

Merged
serrrfirat merged 1 commit into
mainfrom
codex/testing-playbook
Jul 21, 2026
Merged

serrrfirat merged 1 commit into
mainfrom
codex/testing-playbook

Conversation

@serrrfirat

@serrrfirat serrrfirat commented Jul 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • add an SWE1-friendly, outside-in TDD playbook for Ironclaw engineers
  • explain four practical test types: unit/contract, hermetic feature, surface, and live canary
  • include concrete repository examples, code snippets, commands, and where each test belongs
  • cover hermetic Reborn integration, caller-path side effects, recorded fixtures, browser E2E, backend/runtime integration, and live canaries
  • link the playbook from CONTRIBUTING.md so contributors can find it during development

Change Type

  • Bug fix
  • New feature
  • Refactor
  • Documentation
  • CI/Infrastructure
  • Security
  • Dependencies

Linked Issue

None.

Validation

  • cargo fmt --all -- --check
  • cargo clippy --all --benches --tests --examples --all-features -- -D warnings
  • cargo build
  • Relevant tests pass: documentation-only change; no code tests run
  • cargo test --features integration if database-backed or integration behavior changed
  • Manual testing: restored playbook matches the original 456-line version byte-for-byte; git diff --check and the trailing-whitespace check passed
  • If a coding agent was used and supports it, review-pr or pr-shepherd --fix was run before requesting review

Security Impact

None. Documentation-only change.

Reborn Trust-Boundary Checklist

N/A. This documents existing testing practices and does not change Reborn behavior, types, or trust boundaries.

Database Impact

None.

Blast Radius

Documentation only: docs/internal/testing-playbook.md and a discoverability link in CONTRIBUTING.md.

Rollback Plan

Revert commit fc187695d.

Review Follow-Through

The complete original playbook was restored after feedback that the shortened maintainability rewrite removed too much practical integration and live-canary guidance.


Review track: A (docs/tests/chore)

@ironloopai

ironloopai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown
Contributor

🔎 IronLoop Review Status

Head: fc187695d9ab31b7cf224df58aff110f339d3ced
Result: One or more review results were superseded by a newer PR head.
Next: Run @ironloopai review on the latest PR head.
Updated: 2026-07-21T09:00:59.776Z

Current reviewers:

Reviewer State Verdict Findings Last update
ironloop/common-reviewer (reviewer) Superseded N/A N/A 2026-07-21T08:56:16.232Z
Reviewer summaries
Reviewer Detail
ironloop/common-reviewer (reviewer) Superseded by a newer PR head. New head: 3dd6659. Previous verdict: Approved.
Recent activity
Time Reviewer State Detail
2026-07-21T08:45:21.574Z ironloop/common-reviewer (reviewer) Queued Accepted review request for head 4ecae9d.
2026-07-21T08:45:21.574Z ironloop/common-reviewer (reviewer) Queued Waiting for this reviewer lane to become available.
2026-07-21T08:45:21.651Z ironloop/common-reviewer (reviewer) Started Reviewer worker started.
2026-07-21T08:45:24.851Z ironloop/common-reviewer (reviewer) Workspace ready Prepared isolated checkout (merge_ref) at e41c82f.
2026-07-21T08:47:42.818Z ironloop/common-reviewer (reviewer) Result captured Approved; 0 blocking findings.
2026-07-21T08:47:42.818Z ironloop/common-reviewer (reviewer) Completed Review completed and terminal status was persisted.
2026-07-21T08:56:16.232Z ironloop/common-reviewer (reviewer) Superseded A newer PR head replaced this review (3dd6659).
Available commands
  • @ironloopai help
  • @ironloopai agents
  • @ironloopai review
  • @ironloopai review --agent <agent>
Run metadata

Admission: webhook accepted the request and IronLoop persisted reviewer state before this projection.

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6411 July 21, 2026 08:45 Destroyed
@github-actions github-actions Bot added scope: docs Documentation size: XS < 10 changed lines (excluding docs) risk: low Changes to docs, tests, or low-risk modules contributor: core 20+ merged PRs labels Jul 21, 2026
@coderabbitai

coderabbitai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds an Ironclaw TDD playbook covering test selection, placement, workflow, repository examples, and PR checklists, with a new reference from CONTRIBUTING.md.

Changes

Testing guidance

Layer / File(s) Summary
Test types and selection
docs/internal/testing-playbook.md
Defines four test categories and decision questions for choosing coverage based on observable behavior and integration boundaries.
TDD workflow and test placement
docs/internal/testing-playbook.md
Documents the test-first workflow, maps behaviors to repository locations, and provides examples for unit, integration, replay, E2E, runtime, and canary testing.
Contributor workflow and PR checklist
CONTRIBUTING.md, docs/internal/testing-playbook.md
Links the development workflow to the playbook and adds a routine-creation example, PR test card, new-file checklist, and testing rules.

Estimated code review effort: 1 (Trivial) | ~5 minutes

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title follows Conventional Commits style and accurately summarizes the documentation-only change.
Description check ✅ Passed The description covers the required template sections with summary, change type, validation, impact, rollback, and follow-through.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ironloopai ironloopai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ IronLoop Review: reviewer

Review at a glance

Verdict Blocking Notes Inline Head
✅ Approved 0 0 0 4ecae9dee343

Head: 4ecae9dee34313460e009d7973f3787e22e9cb35
Next: No reviewer action needed.

Run details

Status: Current
Needs human: no
Needs validation: no

Summary

Reviewed the complete normal-shape documentation diff (2 files, 460 added lines). The playbook link, repository paths, test commands, and cited examples align with current repository guidance and implementations.

Findings

None.

Developer follow-up

After fixing this feedback:

  1. Push the fix to this PR branch.
  2. Re-run this reviewer with @ironloopai review --agent reviewer if you only changed this reviewer's findings.
  3. Re-run all reviewers with @ironloopai review when the fix may affect multiple areas.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/internal/testing-playbook.md`:
- Around line 338-340: Update the persistence guidance near the statement about
multiple backends to remove the exception that allows production behavior to
omit libSQL or PostgreSQL. Require both backends for new persistence features,
or reference the repository’s documented approved-exception policy if one
exists.
- Around line 447-448: Update the final test rule in the testing playbook to
require both unit or contract coverage and a hermetic feature test only for
production-wired Reborn or cross-component changes. Align the wording with the
behavior- and risk-based layer selection described in the referenced workflow
sections, without imposing both test types on all production changes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d70ec8f1-fcf4-44a4-aeee-82787933d24c

📥 Commits

Reviewing files that changed from the base of the PR and between 400001e and 4ecae9d.

📒 Files selected for processing (2)
  • CONTRIBUTING.md
  • docs/internal/testing-playbook.md

Comment thread docs/internal/testing-playbook.md
Comment thread docs/internal/testing-playbook.md

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the "Ironclaw TDD Playbook for Engineers" (docs/internal/testing-playbook.md) and updates CONTRIBUTING.md to reference it. The playbook outlines the testing strategy, detailing the four test types (unit/contract, hermetic feature, surface, and live canary), a step-by-step workflow, test locations, and a pull request test card template. The reviewer suggested referring to the root configuration file as the "workspace Cargo.toml" instead of "root Cargo.toml" for consistency with other documentation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +237 to +238
For a distinct scenario, add `tests/integration/<scenario>.rs` and register the
flat test binary in the root `Cargo.toml` as `reborn_integration_<scenario>`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For consistency with tests/integration/CLAUDE.md (which uses the term "workspace Cargo.toml"), consider referring to the root configuration file as the workspace Cargo.toml.

@serrrfirat
serrrfirat force-pushed the codex/testing-playbook branch from 4ecae9d to 3dd6659 Compare July 21, 2026 08:56
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6411 July 21, 2026 08:56 Destroyed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/internal/testing-playbook.md`:
- Around line 194-195: Update the production-facing persistence guidance in the
testing playbook to explicitly require coverage for both supported libSQL and
PostgreSQL paths, removing the conditional wording; only reference an
approved-exception policy if one already exists.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: fc904783-fc6e-460d-8277-41cd20e66ce1

📥 Commits

Reviewing files that changed from the base of the PR and between 4ecae9d and 3dd6659.

📒 Files selected for processing (2)
  • CONTRIBUTING.md
  • docs/internal/testing-playbook.md

Comment thread docs/internal/testing-playbook.md Outdated
Comment on lines +194 to +195
- For production-facing persistence, cover supported libSQL and PostgreSQL
paths as required by the owning contract.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Require both persistence backends explicitly.

CONTRIBUTING.md Lines 133-135 requires new persistence features to support both PostgreSQL and libSQL. “As required by the owning contract” weakens that invariant and can legitimize incomplete backend coverage. Require both paths, or link an explicit approved-exception policy.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/internal/testing-playbook.md` around lines 194 - 195, Update the
production-facing persistence guidance in the testing playbook to explicitly
require coverage for both supported libSQL and PostgreSQL paths, removing the
conditional wording; only reference an approved-exception policy if one already
exists.

@railway-app

railway-app Bot commented Jul 21, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-6411 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw ✅ Success (View Logs) Web Jul 21, 2026 at 9:01 am

@serrrfirat
serrrfirat force-pushed the codex/testing-playbook branch from 3dd6659 to fc18769 Compare July 21, 2026 09:00
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-6411 July 21, 2026 09:01 Destroyed
@serrrfirat
serrrfirat merged commit 45b6891 into main Jul 21, 2026
42 checks passed
@serrrfirat
serrrfirat deleted the codex/testing-playbook branch July 21, 2026 09:21
@coderabbitai coderabbitai Bot mentioned this pull request Jul 21, 2026
2 of 21 tasks
@serrrfirat serrrfirat mentioned this pull request Jul 21, 2026
18 of 29 tasks

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-6411 — fc187695 Deployed Jul 21, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules scope: docs Documentation size: XS < 10 changed lines (excluding docs)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant