Skip to content

test(#5494): fold fork-PR issue-harness negative into fork-label positive - #5501

Merged
ifireball merged 1 commit into
mainfrom
agent/5494-fold-fork-issue-negative
Jul 23, 2026
Merged

test(#5494): fold fork-PR issue-harness negative into fork-label positive#5501
ifireball merged 1 commit into
mainfrom
agent/5494-fold-fork-issue-negative

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Fold the standalone "Fork PR does not trigger issue-only harness" negative scenario into the existing "Fork PR label dispatches harness" positive scenario, eliminating a ~90s settle wait while maintaining the same test coverage.

Related Issue

Part of #5490.

Changes

  • Register a second harness (fork-issue-ping) with an issue-only trigger (event.entity.kind == "work_item") in the fork-label positive scenario
  • Assert the issue-only harness did not run after the fork harness positive assertions complete
  • Remove the standalone fork↛issue negative scenario that independently opened a fork PR and waited 90s

Testing

  • make lint — pre-commit could not run in sandbox (network); post-script runs authoritative check
  • make go-test on ./pkg/behaviourtest/... — all tests pass
  • go vet ./pkg/behaviourtest/... — clean
  • Fork dispatch feature stays green (requires live e2e run)

Checklist

  • PR title follows Conventional Commits (correct type, ! for breaking changes)
  • Commits are signed off (DCO) — autonomous agent, exempt
  • I wrote this contribution myself and can explain all changes in it

Closes #5494

Post-script verification

  • Branch is not main/master (agent/5494-fold-fork-issue-negative)
  • Secret scan passed (gitleaks — c9c7540c2995d9a294ede00c2444d2eb600e4852..HEAD)
  • PR body secret scan passed (gitleaks — no-git)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

…tive

Register an issue-only harness ("fork-issue-ping") in the fork-label
positive scenario and assert it did not run, replacing the standalone
"Fork PR does not trigger issue-only harness" scenario. This removes
one ~90s settle wait by piggybacking the negative assertion on the
existing fork PR + label dispatch round-trip.

Note: pre-commit could not run in sandbox (network access denied).
The post-script runs an authoritative pre-commit check.

Unrelated test failures in internal/binary, internal/cli, and
internal/fetch are pre-existing (sandbox network restrictions).
All behaviourtest package tests pass.

Closes #5494
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team as a code owner July 23, 2026 04:57
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Jul 23, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 23, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:59 AM UTC · Completed 5:08 AM UTC
Commit: ce866d2 · View workflow run →

@github-actions

Copy link
Copy Markdown

Site preview

Preview: https://85f9a80b-site.fullsend-ai.workers.dev

Commit: ce866d23a14590731a023e3e7eca1600ec99fe79

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review

Copy link
Copy Markdown

Looks good to me


Labels: PR refactors e2e fork-dispatch behaviour tests (component/e2e + component/dispatch) as a maintenance chore (type/chore).

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge component/e2e End-to-end tests component/dispatch Workflow dispatch and triggers type/chore Maintenance and housekeeping tasks labels Jul 23, 2026
@ifireball ifireball added the ok-to-test Allow e2e CI to run after maintainer review (must be re-applied after each push) label Jul 23, 2026
@ifireball
ifireball added this pull request to the merge queue Jul 23, 2026
Merged via the queue into main with commit 14fdc08 Jul 23, 2026
45 of 49 checks passed
@ifireball
ifireball deleted the agent/5494-fold-fork-issue-negative branch July 23, 2026 06:35
@fullsend-ai-retro

fullsend-ai-retro Bot commented Jul 23, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 6:37 AM UTC · Completed 6:51 AM UTC
Commit: ce866d2 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #5501 — test(#5494): fold fork-PR issue-harness negative into fork-label positive

Workflow went well — no proposals filed.

This retro covers a clean, zero-rework agent workflow on PR #5501, a code-agent-authored e2e test consolidation chore (part of epic #5490).

Timeline

Time (UTC) Event
04:34 ifireball creates #5494, triggers /fs-triage
04:37–04:40 Triage agent completes (~5 min), labels ready-to-code + triaged
04:45 ifireball triggers /fs-code with explicit scoping instructions
04:45–04:57 Code agent run 29980341964 — 12 min total, ~9 min agent time. First-attempt success.
04:57 PR #5501 created (1 file, +13/−16 lines)
04:58–05:08 Review agent run 29980884048 — 10 min, $2.23. Dispatched 3/5 sub-agents (correctness, style, intent-coherence; skipped security + docs). Zero actionable findings. Approved, labeled ready-for-merge.
05:28 ifireball adds ok-to-test
06:22 ifireball approves (no comments)
06:35 PR merged via merge queue

Assessment

What went well:

  • Code agent succeeded first-try with a well-scoped, correct change. It even performed a proactive safety check (grepping for DispatchAgent usage to confirm multi-harness registration safety).
  • Review agent made smart triage decisions — skipping irrelevant sub-agents and correctly approving a clean change.
  • The correctness sub-agent identified that the refactored test is actually stronger than the original (verifies entity-kind filtering when the label matches, not just when a fork PR is opened with a different label).
  • Total agent pipeline time: ~28 minutes (triage + code + review). No rework cycles.

Why no proposals:

  • I considered filing an autonomy-readiness tracking issue (human review added zero findings, ~75 min latency). However, #5156 correctly identifies that zero-comment human approvals are not meaningful evidence of agent superiority — the absence of human comments cannot be used to infer depth of human review. This PR (trivially correct, single-file test refactoring) is exactly the pattern Autonomy-readiness methodology produces false evidence when human review has no comments #5156 flags as producing false autonomy evidence.
  • Existing issue #4532 already tracks a similar data point (test-only PR where agent review covered human review).
  • No review quality gaps, no rework, no token waste, and no pipeline inefficiencies were identified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/dispatch Workflow dispatch and triggers component/e2e End-to-end tests ok-to-test Allow e2e CI to run after maintainer review (must be re-applied after each push) ready-for-merge All reviewers approved — ready to merge ready-for-review Triggers review agent dispatch type/chore Maintenance and housekeeping tasks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(behaviour): fold fork-PR↛issue-harness negative into fork-label positive

1 participant