ci: add Qwen Code issue follow-up bot workflow - #3854
Merged
Conversation
yiliang114
changed the title
Collaborator
Author
|
@copilot pls review for me |
Contributor
Here's a review of the changes. All 10 unit tests pass locally.
Strengths:
Issues / suggestions:
No blocking issues — the logic is sound and the guardrails (rollout gate, dry-run flag, no GITHUB_TOKEN fallback for the bot token) are well-designed. |
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
Collaborator
ReviewOverviewAdds The "bot" is essentially a long natural-language prompt fed to Major concerns1. PR description references tests that don't exist in the diffThe "Validation" section claims: …and describes "Unit tests cover obvious test issues, short-gibberish template reports, missing-information reports…" But 2. Prompt-injection / abuse surface, given
|
wenshao
requested changes
May 6, 2026
wenshao
previously requested changes
May 6, 2026
wenshao
left a comment
wenshao
left a comment
Collaborator
There was a problem hiding this comment.
Qwen Code CI/Test (windows-latest, 20.x) is failing (unrelated to these workflow YAML changes).
Additional finding: The PR description references .github/scripts/issue-followup-bot.test.mjs but this file is not in the diff. Please add the test file or update the description.
Collaborator
ReviewOverviewAdds a single new workflow Strengths
Issues / suggestions1. Global concurrency group will queue new-issue responses behind scheduled batchesconcurrency:
group: '${{ github.workflow }}'
cancel-in-progress: falseA scheduled batch can run up to 15 min; any group: '${{ github.workflow }}-${{ github.event.issue.number || github.event.inputs.issue_number || ''scheduled'' }}'This still serializes scheduled runs against each other but lets per-issue events run independently. 2. Unsafe fallback in dispatch dry-run envDISPATCH_DRY_RUN: "${{ github.event.inputs.dry_run || 'false' }}"If DISPATCH_DRY_RUN: "${{ github.event.inputs.dry_run || 'true' }}"The boolean input default is 3.
|
Collaborator
Author
|
Addressed the review feedback in
Validation run: Prettier, YAML parse, actionlint, prepare-script |
Collaborator
OverviewAdds a single workflow file ( Strengths
Issues / SuggestionsBugs / correctness
Workflow / structure
Style / nits
Test coverage
Security considerations
VerdictSolid, well-structured PR. The sandbox approach is the right call for letting a model interact with
Everything else is polish. LGTM with the caveats above. |
- Require explicit `--repo <expected>` on every gh command path; reject any --repo value that does not match REPOSITORY/GITHUB_REPOSITORY so a poisoned issue body cannot redirect bot writes to another repo. - Add OPENAI_BASE_URL to the secret-scrubbing list so an internal proxy URL is not echoed into comments or labels. - Print the resolved DISPATCH_DRY_RUN / ISSUE_OPENED_DRY_RUN / SCHEDULE_DRY_RUN inputs alongside the resolved dry_run state for easier debugging of automatic vs manual paths.
- Fold the repo-match check into validate_issue_edit_args / validate_issue_comment_args; replace the standalone require_explicit_repo with a smaller require_repo_match used only by the read-only paths. - Add an upfront guard that fails fast when expected_repo is unset, and document the positional subcommand match. - Drop the configuration-notice job: it warned on every issues.opened and cron tick when QWEN_ISSUE_FOLLOWUP_BOT_ENABLED was unset, which is the default state. - Remove the redundant BOT_GITHUB_TOKEN re-export at step level (already inherited from the workflow env). - Invert the dry_run resolution so every branch starts from `true` and flips to `false` on explicit opt-in, removing the duplicate assignments. - Collapse the multi-line dry-run debug block into a single state line. - Note in the prompt that global flags and short aliases (`-b`, `-F`) are rejected by the runner so the model only emits long-form gh commands.
Collaborator
Author
Agent-Logs-Url: https://github.com/QwenLM/qwen-code/sessions/1cf8097d-b747-4838-a206-63a11352facc Co-authored-by: yiliang114 <11473889+yiliang114@users.noreply.github.com>
Contributor
Reviewed the latest state (ee8a3c2). One fix applied, two minor notes below. Fixed (ee8a3c2): Reject log messages from Remaining minor notes (no code change needed):
Everything else looks solid: dry-run enforcement, |
Collaborator
Code ReviewOverviewAdds a new GitHub Actions workflow ( Strengths
Issues & SuggestionsCorrectness
Security
Maintainability
Nits
Risk AssessmentLow-to-moderate. The dry-run default plus per-trigger opt-in means merging this is effectively a no-op until repo vars are flipped. The shim design is conservative — write paths are tightly scoped to single-issue label additions and marker comments, with no path to closing, assigning, removing labels, or Recommendation: Approve with the request to (1) commit the shim regression test mentioned in the PR description and (2) decide whether |
DennisYu07
approved these changes
May 7, 2026
DragonnZhang
pushed a commit
that referenced
this pull request
May 8, 2026
* ci: add issue follow-up bot workflow * ci: gate issue follow-up bot rollout * ci: reduce issue follow-up batch size * ci: address issue follow-up bot review * ci: add temporary issue bot canary * ci: fix canary verification * ci: dedupe bot token issue comments * ci: remove temporary issue bot canary * ci: avoid repeated issue bot followups * ci: simplify issue follow-up bot prompt * ci: refine issue follow-up bot flow * ci: harden issue follow-up bot workflow * ci: harden issue follow-up bot rollout * ci: enforce follow-up bot dry-run writes * ci: redact blocked bot command args * ci: lock follow-up bot gh wrapper to current repo - Require explicit `--repo <expected>` on every gh command path; reject any --repo value that does not match REPOSITORY/GITHUB_REPOSITORY so a poisoned issue body cannot redirect bot writes to another repo. - Add OPENAI_BASE_URL to the secret-scrubbing list so an internal proxy URL is not echoed into comments or labels. - Print the resolved DISPATCH_DRY_RUN / ISSUE_OPENED_DRY_RUN / SCHEDULE_DRY_RUN inputs alongside the resolved dry_run state for easier debugging of automatic vs manual paths. * ci: tighten follow-up bot wrapper and trim noise - Fold the repo-match check into validate_issue_edit_args / validate_issue_comment_args; replace the standalone require_explicit_repo with a smaller require_repo_match used only by the read-only paths. - Add an upfront guard that fails fast when expected_repo is unset, and document the positional subcommand match. - Drop the configuration-notice job: it warned on every issues.opened and cron tick when QWEN_ISSUE_FOLLOWUP_BOT_ENABLED was unset, which is the default state. - Remove the redundant BOT_GITHUB_TOKEN re-export at step level (already inherited from the workflow env). - Invert the dry_run resolution so every branch starts from `true` and flips to `false` on explicit opt-in, removing the duplicate assignments. - Collapse the multi-line dry-run debug block into a single state line. - Note in the prompt that global flags and short aliases (`-b`, `-F`) are rejected by the runner so the model only emits long-form gh commands. * ci: fix shim reject logs to include full subcommand context Agent-Logs-Url: https://github.com/QwenLM/qwen-code/sessions/1cf8097d-b747-4838-a206-63a11352facc Co-authored-by: yiliang114 <11473889+yiliang114@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: yiliang114 <11473889+yiliang114@users.noreply.github.com>
xaelistic
pushed a commit
to xaelistic/qwen-code
that referenced
this pull request
Jun 7, 2026
* ci: add issue follow-up bot workflow * ci: gate issue follow-up bot rollout * ci: reduce issue follow-up batch size * ci: address issue follow-up bot review * ci: add temporary issue bot canary * ci: fix canary verification * ci: dedupe bot token issue comments * ci: remove temporary issue bot canary * ci: avoid repeated issue bot followups * ci: simplify issue follow-up bot prompt * ci: refine issue follow-up bot flow * ci: harden issue follow-up bot workflow * ci: harden issue follow-up bot rollout * ci: enforce follow-up bot dry-run writes * ci: redact blocked bot command args * ci: lock follow-up bot gh wrapper to current repo - Require explicit `--repo <expected>` on every gh command path; reject any --repo value that does not match REPOSITORY/GITHUB_REPOSITORY so a poisoned issue body cannot redirect bot writes to another repo. - Add OPENAI_BASE_URL to the secret-scrubbing list so an internal proxy URL is not echoed into comments or labels. - Print the resolved DISPATCH_DRY_RUN / ISSUE_OPENED_DRY_RUN / SCHEDULE_DRY_RUN inputs alongside the resolved dry_run state for easier debugging of automatic vs manual paths. * ci: tighten follow-up bot wrapper and trim noise - Fold the repo-match check into validate_issue_edit_args / validate_issue_comment_args; replace the standalone require_explicit_repo with a smaller require_repo_match used only by the read-only paths. - Add an upfront guard that fails fast when expected_repo is unset, and document the positional subcommand match. - Drop the configuration-notice job: it warned on every issues.opened and cron tick when QWEN_ISSUE_FOLLOWUP_BOT_ENABLED was unset, which is the default state. - Remove the redundant BOT_GITHUB_TOKEN re-export at step level (already inherited from the workflow env). - Invert the dry_run resolution so every branch starts from `true` and flips to `false` on explicit opt-in, removing the duplicate assignments. - Collapse the multi-line dry-run debug block into a single state line. - Note in the prompt that global flags and short aliases (`-b`, `-F`) are rejected by the runner so the model only emits long-form gh commands. * ci: fix shim reject logs to include full subcommand context Agent-Logs-Url: https://github.com/QwenLM/qwen-code/sessions/1cf8097d-b747-4838-a206-63a11352facc Co-authored-by: yiliang114 <11473889+yiliang114@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: yiliang114 <11473889+yiliang114@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
QWEN_ISSUE_FOLLOWUP_BOT_ISSUES_DRY_RUNcontrolsissues.opened, andQWEN_ISSUE_FOLLOWUP_BOT_SCHEDULE_DRY_RUNcontrols scheduled batches. The previousQWEN_ISSUE_FOLLOWUP_BOT_DRY_RUNremains a fallback.ghwrapper around the Qwen action so the model can inspect issues, add labels, and create body-based comments. In dry-run, the wrapper hard-blocks all writes. In write mode, it only permits single-issuegh issue edit --add-labeland single-issuegh issue comment --bodyoperations; other issue edits, comment edit/delete paths, body files,gh api, and known secret values in command arguments are blocked.Validation
ghwrapper simulation covers dry-run write blocking plus add-label/comment allowlist behavior.workflow_dispatchdry-runs on this branch verified that the workflow can be invoked from the PR branch without mutating issues.Reviewer Verification
Qwen Issue Follow-up Botfrom GitHub Actions on this branch withissue_numberset to a known issue,dry_run=true, andscheduled_limit=1.issue_number,dry_run=true, andscheduled_limit=1to verify scheduled candidate selection without writing comments or labels.QWEN_ISSUE_FOLLOWUP_BOT_ENABLED=trueandQWEN_ISSUE_FOLLOWUP_BOT_ISSUES_DRY_RUN=false; leaveQWEN_ISSUE_FOLLOWUP_BOT_SCHEDULE_DRY_RUNunset ortrueif scheduled batches should remain dry-run.Scope / Risk
status/needs-triage.QWEN_CODE_BOT_TOKENor the existingCI_BOT_PATfallback.Testing Matrix
Testing matrix notes:
Example Issue Follow-up Result
A controlled canary run from an earlier revision of this workflow was used to validate the intended issue-facing behavior: the bot adds routing labels and posts a concise marker comment asking for missing information without assigning or closing the issue.
Example issue: #3770