docs: add visual evidence section to PR template - #2534
Conversation
Encourage contributors to include screenshots or video demos in PRs. Helps reviewers understand and validate changes faster. If nothing visual to show (refactor, internal change), contributors should note 'N/A' in the section.
📋 Review SummaryThis PR adds a new "Visual Evidence (Screenshots / Video)" section to the pull request template, encouraging contributors to include visual documentation of their changes. The change is well-structured, provides clear guidance, and includes an appropriate opt-out mechanism for non-visual changes. 🔍 General Feedback
🎯 Specific Feedback🔵 Low
✅ Highlights
|
When reviewing PRs, the bot now checks if the PR body contains visual evidence (screenshots, videos, GIFs, or video links). The bot will leave a friendly comment requesting visual evidence only if: - No images/videos/GIFs in the PR body - No opt-out language (N/A, internal change, refactor, etc.) - Not a docs-only change This helps reviewers understand and validate changes faster.
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
|
|
||
| **Implementation:** | ||
| 1. Run: gh pr view $PR_NUMBER --json body -q .body | ||
| 2. Check the body for: images (![...], <img src=, .png, .jpg, .gif), video links (loom.com, youtube.com, youtu.be), or screenshot sections |
There was a problem hiding this comment.
[Suggestion] The check can be bypassed by the PR template boilerplate itself. The template now adds a visible ## Visual Evidence (Screenshots / Video) heading and an HTML comment containing opt-out terms like N/A, refactor, internal change, and docs-only, while these instructions say to accept screenshot sections and opt-out language anywhere in the body. If an author leaves the template untouched, the bot may treat boilerplate as evidence or opt-out text and skip the intended reminder.
Consider stripping HTML comments/template boilerplate before checking, and only accepting actual media/link content or explicit non-comment opt-out text entered by the author.
— gpt-5.5 via Qwen Code /review
|
|
||
| **Implementation:** | ||
| 1. Run: gh pr view $PR_NUMBER --json body -q .body | ||
| 2. Check the body for: images (![...], <img src=, .png, .jpg, .gif), video links (loom.com, youtube.com, youtu.be), or screenshot sections |
There was a problem hiding this comment.
[Suggestion] The PR template accepts API evidence such as a curl example or API response snippet, but this detector only recognizes images, videos, GIFs, video links, or screenshot sections as evidence. API-facing PRs that follow the template can still receive an incorrect visual-evidence reminder.
Consider extending the evidence check to treat API evidence as valid, such as curl commands, HTTP request/response snippets, JSON response blocks, or an explicit API response section.
— gpt-5.5 via Qwen Code /review
DragonnZhang
left a comment
There was a problem hiding this comment.
LGTM! ✅ Clean addition of the Visual Evidence section to the PR template. The two suggestions (template bypass detection and API evidence recognition) are non-blocking and can be addressed in follow-ups. — Qwen Code /review
|
The PR template has since been reworked (#4496, #3522, and earlier) and now includes an "Evidence (Before & After)" section requesting before/after screenshots, tmux logs, or video, with an N/A opt-out for non-UI changes — which covers the template change proposed here. The diff here also no longer applies cleanly (it targets the old template layout). The remaining piece — having the review bot auto-comment when a PR lacks visual evidence — isn't in the current |
Encourage contributors to include screenshots or video demos in PRs.
What this adds:
Why:
A screenshot or short video showing the change in action helps reviewers pick this up faster. This reduces back-and-forth and speeds up PR review.
Effect: