Repository navigation
docs(readme): drop test counts and the zero-incident claim - #1912
Conversation
A test count is a vanity metric. It tells a reader nothing about whether the bot works, it reads as filler, and it goes stale on every PR — the number said ~6,300 while the suites actually totalled 6,955. Replaced with what the reader is deciding on: no paywall, and what the CI gate actually runs. "Zero prod incidents" also had to go, because it stopped being true. Production sat on a stale build for four days after two deploys failed health checks and auto-rolled back unnoticed (#1890). Swapped it for the property that is genuinely worth advertising and is verifiable in scripts/deploy.sh:465: a deploy failing its health checks reverts to the last good build. That is what kept service up during the incident, and it is a real answer to "what happens when a deploy of a self-hosted bot goes wrong".
|
Warning Review limit reached
Next review available in: 8 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Failed to generate code suggestions for PR |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 1 file
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Missed one in Security & Maintenance: "Zero production incidents to date". Same claim as the two already removed, and same problem — it stopped being true with #1890. My grep only matched "Zero prod incidents" and "Zero incidents", so this phrasing slipped through. Caught by cubic. The second half of that line was the honest part and is kept: incidents are tracked as public issues with the root cause written up in the open. That is a stronger signal than a streak, because a streak only holds until it doesn't.
|
Good catch, fixed in I had grepped for Kept the honest half of that line:
That is a stronger signal than a streak anyway. A streak only holds until it doesn't, and #1890 is the proof — whereas "we write up what broke" stays true and is checkable against the issue tracker. Verified nothing else remains: |
|
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Auto-approved: Removes stale test counts and an outdated zero-incident claim from the README; replaces them with accurate, durable details about the test suite and auto-rollback. The diff is bounded to a single documentation file and clearly beneficial.
Re-trigger cubic
#1893 changed the invite permission set without reading decisions/2026-06-18-invite-permission-scope.md, landing on 3165184 (music only). Its prerequisite #1498 is closed, so the ADR's curated set should already have been live. Adopts 3173504: music + ManageMessages (auto-mod cleanup) + ViewAuditLog, which auditHandler needs to attribute moderation cases — without it that attribution was silently degraded on every fresh install. Tracing the call sites found a fifth copy: GuildService.generateBotInviteUrl hardcoded permissions='8' (Administrator), backing the dashboard's "add Lucky to this server" button. cubic then found a sixth in the e2e fixtures, which mocked an Administrator response and so could not have caught either regression. All six now derive from the shared constant. Also corrects docs/TOP_GG_SUBMISSION.md, which specified 36970496 with a breakdown summing to a different number entirely, and drops the vanity and zero-incident claims removed from the README in #1912. Tests pin the bitfield to the ADR and were verified to fail when the value is reverted. Closes #1923.
1 participant
Test counts
A test count says nothing about whether the bot works. It reads as filler, and it goes stale the moment anyone adds a test — the README said
~6,300in five places while the suites actually totalled 6,955 (shared 1398, bot 3188, backend 1350, frontend 1019).Replaced with what a reader is actually deciding on:
~6,300 tests · Zero prod incidentsNo paywall, no premium tierEvery PR runs lint + build + ~6,300 tests + SonarCloud gatesEvery PR runs lint, build, the full test suite, and SonarCloud gates~6,300 unit + integration tests, Playwright e2eJest + Vitest unit/integration, Playwright e2e# Run all tests (~6,300 total)# Run all testsnpm run verify (lint + build + ~6,300 tests)npm run verify (lint + build + tests)The suite still runs on every PR. It just does not need a number attached to be worth having.
The zero-incident claim
Zero prod incidentsandZero incidents: Battle-tested in production with full test coveragealso had to go, because they stopped being true. Production sat on a stale build from 2026-07-16 for four days after two deploys failed their health checks and auto-rolled back without anyone noticing (#1890).Swapped for the property that is genuinely worth advertising and that a self-hoster actually cares about:
Verified in
scripts/deploy.sh:465before claiming it, and it is exactly what kept service up during that incident. It answers "what happens when a deploy of a self-hosted bot goes wrong", which is a better pitch than a coverage boast.Note
The same test-count line was in the Top.gg listing (I had just updated it to
~6,900during the resubmission). Same reasoning applies, so I will strip it there too rather than leave the two out of sync.Summary by cubic
Remove test counts and all “zero incidents” claims from the README. Replace with durable details: “No paywall, no premium tier”, CI gate (lint, build, full test suite, SonarCloud), test stack (
Jest+Vitest,Playwright), “Auto-rollback” on failed deploys, and that incidents are tracked as public GitHub issues with root cause and fix documented.Written for commit 5ca1b6c. Summary will update on new commits.