Skip to content

Conversation

@ralphbean
Copy link

@ralphbean ralphbean commented Oct 31, 2025

Summary

The receiving-code-review skill previously lacked guidance on WHERE to post responses on GitHub, leading to agents posting top-level PR comments when they should reply in specific review comment threads.

Changes

  • Decision tree for threaded vs. top-level comments
  • Correct gh api command for threaded replies
  • Common mistakes section showing wrong approach
  • Test scenario documenting expected behavior

Real-world failure case

PR konflux-ci/architecture#268 comment #3472979081 where an agent using this skill posted to the top-level PR instead of replying in the review thread.

Test plan

  • Review the new "Posting Responses on GitHub" section
  • Verify the API endpoint is correct: /repos/{owner}/{repo}/pulls/{pull_number}/comments/{comment_id}/replies
  • Test scenario provided in TEST-SCENARIO.md

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added comprehensive guidance on posting responses to GitHub code reviews, distinguishing between threaded comment replies and general PR discussions.
    • Included practical command examples, common mistakes, and best practices for structuring effective responses.

@coderabbitai
Copy link

coderabbitai bot commented Oct 31, 2025

Warning

Rate limit exceeded

@ralphbean has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 13 minutes and 54 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 726f348 and 4c20db1.

📒 Files selected for processing (1)
  • skills/receiving-code-review/SKILL.md (1 hunks)

Walkthrough

Added documentation for the receiving-code-review skill covering how to post responses on GitHub. New content distinguishes between replying to specific review threads versus general PR discussions, provides CLI command examples, and includes test documentation outlining validation scenarios for correct response placement behavior.

Changes

Cohort / File(s) Summary
Receiving Code Review Documentation
skills/receiving-code-review/SKILL.md
Added new section "Posting Responses on GitHub" with guidelines distinguishing threaded replies from general PR discussions, includes command examples, rationale subsection "Why threaded replies matter," common mistakes section, and best practices for structuring responses.
Receiving Code Review Test Documentation
skills/receiving-code-review/examples/github-placement-testing.md
New file defining test scenarios and validation criteria for verifying correct GitHub response placement, including baseline failure example, two test cases (threaded vs. top-level responses), example prompts with expected CLI commands, and success criteria checklist.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

  • Content accuracy: Verify CLI command examples and GitHub workflow instructions are correct
  • Consistency: Ensure documentation aligns with the new test scenarios and examples
  • Completeness: Check that both threaded and general discussion response patterns are clearly distinguished and comprehensive

Poem

A rabbit hops through GitHub threads,
With responses placed where wisdom treads,
Threaded or not—we now impart,
How code review replies play their part! 🐰📝

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The pull request title "Add GitHub response placement guidance to receiving-code-review" directly and accurately describes the primary change in the changeset. The title clearly indicates that new guidance documentation has been added to the receiving-code-review skill specifically about proper GitHub response placement (threaded vs. top-level comments). The title is concise, specific, and avoids generic or vague terminology. A teammate scanning the repository history would immediately understand that this PR adds documentation guidance related to GitHub comment placement strategy for the receiving-code-review skill.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
skills/receiving-code-review/SKILL.md (1)

102-111: Add language specification to decision tree code block.

The GitHub API endpoint for replying to review comments is /repos/{owner}/{repo}/pulls/{pull_number}/comments/{comment_id}/replies, which the instructions correctly show. However, the code block lacks a language identifier for proper rendering.

-```
+```yaml
 WHEN responding to code review feedback on GitHub:
 
 IF feedback is from a specific review comment thread:
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 42d44ce and 12241b1.

📒 Files selected for processing (2)
  • skills/receiving-code-review/SKILL.md (1 hunks)
  • skills/receiving-code-review/TEST-SCENARIO.md (1 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.18.1)
skills/receiving-code-review/SKILL.md

102-102: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🔇 Additional comments (1)
skills/receiving-code-review/SKILL.md (1)

115-139: ✅ Correct API guidance and clear examples.

The endpoint path is accurate per GitHub's official API documentation. The decision tree, rationale, and "Common mistake" section effectively teach agents when and how to use threaded replies versus top-level comments. The contrast between the wrong (gh pr comment) and right (gh api .../replies) approaches is particularly helpful.

The receiving-code-review skill previously lacked guidance on WHERE to post
responses on GitHub, leading to agents posting top-level PR comments when
they should reply in specific review comment threads.

This commit adds:
- Decision tree for threaded vs. top-level comments
- Correct gh api command for threaded replies
- Common mistakes section showing wrong approach

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Signed-off-by: Ralph Bean <[email protected]>
@ralphbean ralphbean force-pushed the fix-receiving-code-review-github-placement branch from 726f348 to 4c20db1 Compare October 31, 2025 18:00
@obra
Copy link
Owner

obra commented Oct 31, 2025

Thanks for this. I see that you've got an example of the failure case. Do you have examples of it working right with the fix?

@ralphbean
Copy link
Author

Thanks for this. I see that you've got an example of the failure case. Do you have examples of it working right with the fix?

No, not yet.

@nickolasclarke
Copy link

My PR, which I just updated, handles some of this, and is slightly duplicative. If we like everything across both of these PRs, we should reconcile this a bit. #41

@obra
Copy link
Owner

obra commented Dec 23, 2025

Thanks for this! We trimmed it down to the essential insight and added it in commit 0ca515a:

When replying to inline review comments on GitHub, reply in the comment thread (gh api repos/{owner}/{repo}/pulls/{pr}/comments/{id}/replies), not as a top-level PR comment.

The rest of the gh CLI mechanics are things Claude already knows, but this threading guidance is a useful behavioral nudge. Credited you in the release notes!

@obra obra closed this Dec 23, 2025
obra added a commit that referenced this pull request Dec 24, 2025
When replying to inline review comments, use the thread API rather than
posting top-level PR comments.

Based on feedback from @ralphbean in PR #79.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants