test(#5495): fold wrong-label negative into issue-label positive scenario - #5500
Conversation
…ario Merge the standalone "Wrong label does not trigger issue harness" scenario into the existing "Issue label dispatches issue-only harness" scenario in dispatch.feature. The combined scenario now: 1. Sets up issue-ping harness and creates an issue 2. Applies a wrong label and asserts the harness did not run 3. Applies the correct ready-for-ping label and waits for success 4. Asserts exactly 1 dispatch occurred The ScenarioStart timestamp is safely sequenced: whenIssueLabeled sets it on each call, so the negative check uses the wrong-label timestamp (T1) and the positive check uses the right-label timestamp (T2). The 90s wait in thenHarnessAgentDidNotRun completes before T2 is set, preventing false greens from timestamp reuse. This eliminates one GHA round-trip (~90s) by sharing harness setup and issue creation between the negative and positive assertions. Note: pre-commit could not run in sandbox (network restriction). Closes #5495
|
🤖 Finished Review · ✅ Success · Started 4:56 AM UTC · Completed 5:06 AM UTC |
Site previewPreview: https://70df4ca4-site.fullsend-ai.workers.dev Commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
ReviewFindingsLow
Labels: PR modifies e2e behaviour dispatch feature file |
|
🤖 Finished Retro · ✅ Success · Started 6:39 AM UTC · Completed 6:50 AM UTC |
Retro: PR #5500 — fold wrong-label negative into issue-label positive scenarioVerdict: Clean workflow, no new proposals. This was an efficient, single-iteration agent workflow that completed in ~2 hours from issue creation to merge. The code agent produced a correct, minimal change (+3/−17 lines) that consolidated two e2e scenarios into one, eliminating a ~90s GHA round-trip. Timeline
AnalysisCode agent judgment was sound. The issue warned about Review agent finding was accurate and well-calibrated. The Autonomy readiness signal. The review agent's analysis fully covered what the human reviewer checked — the human approved without adding any findings or requesting changes. This is positive evidence for review agent autonomy on test-only e2e scenario consolidation changes in this repo. Existing issues with new evidence
|
Summary
Fold the standalone "Wrong label does not trigger issue harness" scenario into the "Issue label dispatches issue-only harness" scenario in
dispatch.feature, eliminating one GHA round-trip (~90s) by sharing harness setup and issue creation.Related Issue
Part of #5490 (fewer positive GHA round-trips for behaviour suite).
Changes
dispatched exactly 1 timeassertion to verify no duplicate dispatch from either label eventwhenIssueLabeledsetsScenarioStarton each call, so the negative check uses the wrong-label timestamp and the positive check uses the right-label timestamp, with the 90sthenHarnessAgentDidNotRunwait ensuring clean separationTesting
pkg/behaviourtest/...unit tests passgo vetpassesmake behaviour-test(requires live GitHub infrastructure — must be verified in CI)Closes #5495
Post-script verification
agent/5495-fold-wrong-label-negative)c9c7540c2995d9a294ede00c2444d2eb600e4852..HEAD)