docs(#5670): add fixture authoring section to behaviour-testing guide - #5673
Conversation
Add a "Fixture authoring" section documenting the requirement that every BT scenario dispatching an agent stage must include a write_fixture row emitting output/agent-result.json with content conforming to the stage's result schema. The section includes: - A checklist for new scenarios (identify role, create fixture, add write_fixture row, add assertion, verify downstream validation) - A fixture inventory table listing existing fixtures and their target schemas - Guidance on downstream CLI validation beyond JSON schema (e.g. full-length hex SHAs) - A copy-and-adapt example using the fork-bash-routing scenario Note: pre-commit could not run in sandbox (network restriction). The post-script runs pre-commit authoritatively on the runner. Closes #5670
|
🤖 Finished Review · ✅ Success · Started 12:02 PM UTC · Completed 12:13 PM UTC |
Site previewPreview: https://29c9fc07-site.fullsend-ai.workers.dev Commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
ReviewFindingsLow
Labels: PR adds documentation for behaviour test fixture authoring, touching e2e testing guidance |
|
🤖 Finished Retro · ✅ Success · Started 7:10 AM UTC · Completed 7:24 AM UTC |
Retro: PR #5673 — docs(#5670): add fixture authoring section to behaviour-testing guideWorkflow overviewThis PR was the product of a fully autonomous retro→triage→code→review pipeline. The retro agent on PR fullsend-ai/fullsend#5654 identified a documentation gap that caused a first-attempt BT failure (missing
Cost: $2.02 for the code agent session. The code agent spent 3m11s in the Claude session (45% of wall time); the remaining 55% was infrastructure overhead (sandbox creation, Go build from source, post-script). What went well
Evidence for existing issues (no new proposals needed)
ConclusionThis workflow operated cleanly end-to-end. The retro→triage→code→review pipeline produced a correct docs change on the first attempt with no human intervention before the final approval. All observations are covered by existing open issues — no new proposals filed. |
Summary
docs/guides/dev/behaviour-testing.mddocumenting that every BT scenario dispatching an agent stage must include awrite_fixturerow emittingoutput/agent-result.jsonwith schema-valid contentContext
The code agent's first attempt on PR #5654 omitted the
write_fixturerow foragent-result.json, causing a harness validation failure. The existing behaviour-testing guide documented thewrite_fixturesyntax but did not explain the requirement foroutput/agent-result.jsonor which schemas fixtures must conform to. This section fills that gap for both agents and human contributors.Testing
e2e/behaviour/fixtures/Closes #5670
Post-script verification
agent/5670-fixture-authoring-docs)a3dd597c77dc90dc183f844b5aa9badea57a7bd4..HEAD)