Skip to content

docs: add visual evidence section to PR template #2534

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
qwen-code-ci-bot wants to merge 2 commits into from
+33 −0
Closed

docs: add visual evidence section to PR template #2534

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f04fc07
docs: add visual evidence section to PR template
Mar 20, 2026
15f63d9
ci: add visual evidence check to PR review bot
Mar 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/pull_request_template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
## Visual Evidence (Screenshots / Video)

<!--
A screenshot or short video showing the change in action helps reviewers pick this up faster.

- For UI changes: Add a screenshot or GIF showing the before/after
- For CLI changes: Add a terminal screenshot or screen recording
- For API changes: Add a curl example or API response snippet

If there's nothing visual to show (e.g., refactor, internal change, docs-only), please note that here: **N/A - [reason]**
-->

## TLDR

<!-- Add a brief description of what this pull request changes and why and any important things for reviewers to look at -->
Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/qwen-code-pr-review.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,27 @@ jobs:
If ADDITIONAL_INSTRUCTIONS contains text, prioritize those specific areas or focus points in your review.
Common instruction examples: "focus on security", "check performance", "review error handling", "check for breaking changes"

## Visual Evidence Check (REQUIRED - Run First)

When reviewing a pull request, check if the PR body contains any visual evidence of the change — this could be in a "Screenshots / Video Demo" section, inline images, linked videos, GIFs, or even a Loom/YouTube link anywhere in the body.

**Only leave a comment if ALL of the following are true:**
- There are no images, videos, GIFs, or video links anywhere in the PR body
- There is no "N/A", "no user-facing change", "internal change", "refactor", or similar opt-out language anywhere in the body
- The PR is not a docs-only change (i.e., only `.md` files changed)

**If you do comment:**
- Be friendly and brief: "Thanks for the PR! A screenshot or short video showing the change in action helps reviewers pick this up faster. If there's nothing visual to show (e.g., refactor or internal change), just note that in the description."
- Comment once per PR — do not repeat on subsequent runs
- Do **not** block the PR, request changes, or mark any check as failed

**Implementation:**
1. Run: gh pr view $PR_NUMBER --json body -q .body
2. Check the body for: images (![...], <img src=, .png, .jpg, .gif), video links (loom.com, youtube.com, youtu.be), or screenshot sections

@wenshao wenshao Apr 24, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The check can be bypassed by the PR template boilerplate itself. The template now adds a visible ## Visual Evidence (Screenshots / Video) heading and an HTML comment containing opt-out terms like N/A, refactor, internal change, and docs-only, while these instructions say to accept screenshot sections and opt-out language anywhere in the body. If an author leaves the template untouched, the bot may treat boilerplate as evidence or opt-out text and skip the intended reminder.

Consider stripping HTML comments/template boilerplate before checking, and only accepting actual media/link content or explicit non-comment opt-out text entered by the author.

— gpt-5.5 via Qwen Code /review

@wenshao wenshao Apr 24, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The PR template accepts API evidence such as a curl example or API response snippet, but this detector only recognizes images, videos, GIFs, video links, or screenshot sections as evidence. API-facing PRs that follow the template can still receive an incorrect visual-evidence reminder.

Consider extending the evidence check to treat API evidence as valid, such as curl commands, HTTP request/response snippets, JSON response blocks, or an explicit API response section.

— gpt-5.5 via Qwen Code /review

3. Check for opt-out language: "N/A", "no user-facing change", "internal change", "refactor", "docs-only"
4. Check if only .md files changed: echo "$CHANGED_FILES" | grep -v '\.md$'
5. If all conditions are met, add the comment before your main review

Once you have the information, provide a comprehensive code review by:
1. Writing your review to a file: write_file("review.md", "<your detailed review feedback here>")
2. Posting the review: gh pr comment $PR_NUMBER --body-file review.md --repo $REPOSITORY
Expand Down
Loading