Skip to content

docs: document gated-marker coverage audit (#61) - #89

Merged
wjduenow merged 2 commits into
devfrom
feature/61-coverage-audit-docs
May 14, 2026
Merged

wjduenow merged 2 commits into
devfrom
feature/61-coverage-audit-docs

Conversation

@wjduenow

@wjduenow wjduenow commented May 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds a ## Pre-release coverage audit section to CONTRIBUTING.md — a one-shot command that runs all gated markers (bigquery, anthropic, cli_subprocess, e2e, wheel_smoke) under --cov in a single invocation, plus guidance on interpreting the default-vs-gated coverage delta.
  • Cross-references the new section from .claude/rules/testing-signal.md § "Known gap: excluded markers".

Docs-only; no code paths touched. Addresses LOW finding L7 from the 2026-05-11 codebase review.

Closes #61

Test plan

  • CONTRIBUTING.md § Pre-release coverage audit renders correctly
  • Cross-reference in .claude/rules/testing-signal.md resolves

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added pre-release coverage audit guidance for maintainers, including instructions for running gated-marker tests with environment variables
    • Explained how to interpret coverage measurement differences before releases

Review Change Stack

Add a "Pre-release coverage audit" section to CONTRIBUTING.md with a
one-shot command that runs all gated markers under --cov, plus guidance
on interpreting the default-vs-gated coverage delta. Cross-reference it
from .claude/rules/testing-signal.md § "Known gap: excluded markers".

Closes #61

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented May 14, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (4)
  • feature/.*
  • bug/.*
  • hotfix/.*
  • feat/.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9806ef80-ae7d-433d-b34b-c7cfb278f4fa

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This docs-only PR adds maintainer guidance for auditing coverage from pytest markers excluded from the default coverage badge, and cross-references that guidance from the testing rules.

Changes:

  • Adds a pre-release gated-marker coverage audit section to CONTRIBUTING.md.
  • Links .claude/rules/testing-signal.md to the new contributing guide section.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
CONTRIBUTING.md Adds the gated-marker coverage audit command and interpretation guidance.
.claude/rules/testing-signal.md Adds a cross-reference to the new pre-release audit documentation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread CONTRIBUTING.md Outdated
Comment thread CONTRIBUTING.md Outdated
@wjduenow

Copy link
Copy Markdown
Owner Author

@CodeRabbit review

@coderabbitai

coderabbitai Bot commented May 14, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Fix the pre-release coverage audit command per Copilot review:
- Add --cov-fail-under=0 so the inherited 80% gate from addopts does not
  make the measurement command exit non-zero on passing gated tests.
- Use --cov-append so the gated run combines with the default run into
  one .coverage file; the "additional coverage" claim is now reproducible
  from the documented command instead of being a standalone gated-only %.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wjduenow

Copy link
Copy Markdown
Owner Author

PR Review Summary

Fixed (2 items)

File Line Issue Commit
CONTRIBUTING.md 43 Command inherited --cov-fail-under=80 from addopts, so it could exit non-zero even when gated tests pass 3d33eba
CONTRIBUTING.md 49 Command produced standalone gated-only coverage, not a combined total — made the "additional coverage" delta non-reproducible 3d33eba

Both fixed in one change: the audit command now uses --cov-append (combines the gated run into the default run's .coverage file so the term report shows the combined total) and --cov-fail-under=0 (overrides the inherited 80% gate — this is a measurement, not a gate). Guidance rephrased to describe the combined-total delta.

False Positives (0 items)

None.

@wjduenow
wjduenow merged commit 295c4d7 into dev May 14, 2026
2 checks passed
@wjduenow
wjduenow deleted the feature/61-coverage-audit-docs branch May 14, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants