Skip to content

sync: port triage agent changes from scaffold - #8

Merged
ggallen merged 7 commits into
mainfrom
sync-triage-from-scaffold
Jul 1, 2026
Merged

sync: port triage agent changes from scaffold#8
ggallen merged 7 commits into
mainfrom
sync-triage-from-scaffold

Conversation

@ggallen

@ggallen ggallen commented Jul 1, 2026

Copy link
Copy Markdown
Member

Summary

Sync triage agent files with fullsend-ai/fullsend scaffold, bringing the agents repo up to date with 17 commits that landed in the scaffold since the initial fork.

Changes by commit:

  1. Schema: Replace blocked action (single blocked_by URL) with prerequisites (structured existing/create arrays); add question action
  2. Agent prompt: Prerequisites action, existing-PR gate (hard constraint), question action, scoped anti-premature-resolution rule
  3. Post-script: Prerequisites handler with cross-repo issue creation and allowlist validation, question handler, stderr diagnostics, deferred ready-to-code label, GITHUB_WORKSPACE guard
  4. Pre-script: Add question label to triage label reset set
  5. Tests: Full test coverage for prerequisites and question actions
  6. Skill: Generalize issue-labels for issues and PRs (not just issues)
  7. Harness: Migrate runner_envenv.runner/env.sandbox per ADR 0055

Original commits ported:

  • 0fb3793c fix(#1752): defer ready-to-code label until after label_actions
  • 1e725001 feat(#1811): add question action to triage agent
  • 1a2db28a fix: update stale doc references for question control label
  • e492ac78 feat(schema): replace blocked with prerequisites action
  • b2055cb1 feat(triage): replace blocked action with prerequisites in agent prompt
  • 6f79d87a fix(triage): correct label name in agent prompt
  • 3a44b0cc feat(triage): handle prerequisites action in post-script
  • 080368cf fix(triage): update post-triage tests for prerequisites action
  • 11bae491 fix(triage): update schema validation tests for prerequisites action
  • 2933555f fix(#2024): triage agent must not ask reporters implementation questions
  • 8c7bec88 fix(#2237): scope anti-premature-resolution rule to user-facing questions
  • e57f10a7 fix(triage): address review feedback on prerequisites action
  • 12b47a9a fix(#2393): add diagnostic stderr output to post-script failure paths
  • 095039eb fix(#1321): add existing-PR gate to triage agent definition
  • 3ed6080c feat(skill): generalize issue-labels for issues and PRs
  • 51ec19e2 fix(#2458): guard GITHUB_WORKSPACE in post-triage.sh for local execution
  • 3ce10f4d refactor(harness): migrate triage to env.runner/env.sandbox (ADR 0055)

Test plan

  • post-triage-test.sh — all 46 tests pass
  • validate-output-schema-test.sh — all triage-specific tests pass (non-triage schema tests expected to fail since fix/review schemas not in this repo)

ggallen added 7 commits July 1, 2026 10:38
… action

Replace the `blocked` action (single `blocked_by` URL) with `prerequisites`
(structured `existing` and `create` arrays for multiple prerequisite tracking
and cross-repo issue creation). Add `question` action for support questions
that should be answered directly rather than triaged as bugs.

Ported from fullsend-ai/fullsend commits:
- e492ac78 feat(schema): replace blocked with prerequisites action (#401)
- 1e725001 feat(#1811): add question action to triage agent

Signed-off-by: Greg Allen <greg@fullsend.ai>
Signed-off-by: Greg Allen <gallen@redhat.com>
…isting-PR gate

- Replace blocked references with prerequisites action throughout
- Add existing-PR gate (hard constraint): if an open PR already
  addresses the issue, use prerequisites instead of sufficient
- Add question action handling for support questions
- Update section 2c to use prerequisite terminology
- Scope anti-premature-resolution rule to user-facing questions only

Ported from fullsend-ai/fullsend commits:
- b2055cb1 feat(triage): replace blocked with prerequisites in agent prompt
- 6f79d87a fix(triage): correct label name in agent prompt
- 1e725001 feat(#1811): add question action
- 2933555f fix(#2024): agent must not ask reporters implementation questions
- 8c7bec88 fix(#2237): scope anti-premature-resolution rule
- 095039eb fix(#1321): add existing-PR gate
- e57f10a7 fix(triage): address review feedback on prerequisites action

Signed-off-by: Greg Allen <greg@fullsend.ai>
Signed-off-by: Greg Allen <gallen@redhat.com>
…tics

- Add prerequisites case with cross-repo issue creation, allowlist
  validation, and blocked label management
- Add question action case
- Redirect error messages to stderr for all failure paths
- Add diagnostic output (stderr) to help debug post-script failures
- Defer ready-to-code label until after label_actions to avoid
  webhook race conditions
- Guard GITHUB_WORKSPACE for local execution

Ported from fullsend-ai/fullsend commits:
- 0fb3793c fix(#1752): defer ready-to-code label until after label_actions
- 3a44b0cc feat(triage): handle prerequisites action in post-script
- 12b47a9a fix(#2393): add diagnostic stderr output to post-script failure paths
- 51ec19e2 fix(#2458): guard GITHUB_WORKSPACE in post-triage.sh for local execution

Signed-off-by: Greg Allen <greg@fullsend.ai>
Signed-off-by: Greg Allen <gallen@redhat.com>
Include the `question` label in the set of triage labels that are
reset before a new triage run.

Ported from fullsend-ai/fullsend commit 1a2db28a.

Signed-off-by: Greg Allen <greg@fullsend.ai>
Signed-off-by: Greg Allen <gallen@redhat.com>
Update post-triage-test.sh:
- Add gh mock for label list API and issue creation
- Add GITHUB_WORKSPACE setup with config.yaml allowlist fixture
- Replace blocked tests with prerequisites tests (existing and create)
- Add tests for allowed/disallowed issue creation targets
- Add question action tests

Update validate-output-schema-test.sh:
- Replace blocked validation tests with prerequisites tests
- Add question action validation test
- Add prerequisites-both-arrays-empty negative test

Ported from fullsend-ai/fullsend commits:
- 080368cf fix(triage): update post-triage tests for prerequisites action
- 11bae491 fix(triage): update schema validation tests for prerequisites action

Signed-off-by: Greg Allen <greg@fullsend.ai>
Signed-off-by: Greg Allen <gallen@redhat.com>
Broaden the issue-labels skill to work with both issues and pull
requests. Remove the hardcoded control-label blocklist in favor of
pipeline-side filtering. Simplify the constraint language to be
agent-agnostic.

Ported from fullsend-ai/fullsend commit 3ed6080c.

Signed-off-by: Greg Allen <greg@fullsend.ai>
Signed-off-by: Greg Allen <gallen@redhat.com>
Migrate from deprecated `runner_env` to the nested `env.runner` /
`env.sandbox` format per ADR 0055. Add top-level pre_script/post_script
alongside forge-specific ones. Add sandbox env block for GitHub forge
to forward GITHUB_ISSUE_URL and GH_TOKEN into the sandbox.

Ported from fullsend-ai/fullsend commit 3ce10f4d.

Signed-off-by: Greg Allen <greg@fullsend.ai>
Signed-off-by: Greg Allen <gallen@redhat.com>
@ggallen

ggallen commented Jul 1, 2026

Copy link
Copy Markdown
Member Author

Closing — took the wrong approach by overwriting agents repo files with scaffold versions. The agents repo files are already up to date; the scaffold is the one that needs to catch up or be deprecated.

@ggallen ggallen closed this Jul 1, 2026
@ggallen ggallen reopened this Jul 1, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 1, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:44 PM UTC · Completed 2:51 PM UTC
Commit: 97a323f · View workflow run →

@ggallen
ggallen merged commit 8058a42 into main Jul 1, 2026
5 checks passed
@fullsend-ai-retro

fullsend-ai-retro Bot commented Jul 1, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 2:49 PM UTC · Completed 2:56 PM UTC
Commit: 97a323f · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

Review skipped — this PR is already merged.

The /fs-review command only reviews open pull requests.

Posted by fullsend post-review check

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #8 — sync: port triage agent changes from scaffold

Timeline

  1. 14:41:11 — PR opened by ggallen (manual sync, branch sync-triage-from-scaffold). 15 files, 2,246 lines added — ports the full triage agent system from scaffold.
  2. 14:41:25 — Review agent dispatched (run 28525802151).
  3. 14:41:37 — Human comments that the PR took the wrong approach (overwriting agents repo files with scaffold versions).
  4. 14:41:38 — PR closed.
  5. 14:41:49 — PR reopened (11 seconds later).
  6. 14:44:18 — Review bot posts status comment (review started on 97a323f).
  7. 14:44:38 — PR merged by ggallen (~3.5 minutes after open).
  8. 14:51:59 — Review agent completes (~10.5 minutes total, 31 tool calls, 4 sub-agents). Post-script detects PR is already merged and skips posting the review.
  9. 14:49:44 — Retro agent dispatched.

Analysis

Review timing: The review agent did substantial work (31 tool calls, 4 sub-agents, ~7 minutes of agent time) but its output was never posted because the PR merged before the review completed. This is a known class of issue.

No review before merge: The PR was merged without any formal review — no human approval, no bot review. For a manual sync PR of this size, this is a process choice by the maintainer.

Retro debounce worked correctly: The first retro dispatch (on PR reopen) was properly debounced/cancelled, and only the post-merge retro ran.

Existing coverage

All identified improvement areas are already tracked by open issues in fullsend-ai/fullsend:

  • Review agent wasted on merged PRs: #2388, #2606, #1439
  • Post review results on merged PRs as informational: #1373
  • Pre-review skip to save tokens: #2809

No new proposals are warranted — existing issues comprehensively cover the improvement opportunities observed in this workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant