Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .claude/rules/testing-signal.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,12 @@ SF_RUN_BQ=1 pytest -m bigquery --no-cov
SF_RUN_BQ=1 GOOGLE_CLOUD_PROJECT=<billing-project> ANTHROPIC_API_KEY=sk-... pytest -m e2e --no-cov
```

For a one-shot **pre-release** measurement of how much coverage the gated paths
contribute (run all gated markers under `--cov` in a single invocation), see
`CONTRIBUTING.md` § "Pre-release coverage audit". A maintainer running that
before each release catches coverage regressions in the gated paths that the
default badge number cannot surface.

**`wheel_smoke` marker (issue #47).** Maintainer-only gate added by issue #47 to verify wheel-build packaging without coupling it to default CI. The single test (`tests/test_wheel_packaging.py`) shells out `python -m build --wheel --outdir <tmp>` (or `uvx --from build pyproject-build` when `build` isn't in the venv), opens the artifact via `zipfile.ZipFile`, and asserts the canonical demo file set appears under `signalforge/_demo/`. Catches `pyproject.toml` `[tool.hatch.build.targets.wheel] include` regressions that editable-install tests cannot — see `python-build.md` § "Shipping package data" for the full pattern.

## End-to-end gated tests (issue #10)
Expand Down
32 changes: 32 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,38 @@ ruff check . && ruff format --check . && pyright && pytest

**Coverage:** see [`docs/codecov-ops.md`](docs/codecov-ops.md) for Codecov setup, badge interpretation, and threshold bumps.

## Pre-release coverage audit

The default `pytest` run — and therefore the coverage badge — measures only the
default marker set. Tests gated behind `bigquery`, `anthropic`, `cli_subprocess`,
`e2e`, and `wheel_smoke` are filtered out by `addopts` (see
`.claude/rules/testing-signal.md` § "Known gap: excluded markers"), so the
real-network and packaging paths are not instrumented in the badge number.

Before cutting a release, run both suites and combine their coverage into one
total to catch regressions in the gated paths:

```bash
# 1. Default coverage (what the badge reports) — writes a fresh .coverage file:
pytest

# 2. Append the gated-marker run to the SAME .coverage data file.
# --cov-append combines with run 1 so the term report shows the COMBINED total.
# --cov-fail-under=0 overrides the 80% gate inherited from addopts — gated
# markers alone never clear it, and this is a measurement, not a gate.
# (bigquery/anthropic/e2e need creds; cli_subprocess/wheel_smoke do not.)
SF_RUN_BQ=1 ANTHROPIC_API_KEY=sk-... GOOGLE_CLOUD_PROJECT=<billing-project> \
pytest -m 'bigquery or anthropic or e2e or cli_subprocess or wheel_smoke' \
--cov=signalforge --cov-append --cov-fail-under=0 --cov-report=term
```

The combined total from step 2 minus the default badge number from step 1 is
the coverage the gated paths add — typically 5–10%. Interpreting the delta: if
the default badge number drops by M% but the combined total holds steady, that
is likely a redistribution (a code path moved behind a gated marker) rather than
a true regression. A drop in the *combined* total is a real regression worth
chasing before the release goes out.

## Test markers

Tests are tagged with `@pytest.mark.{unit, integration, error}` (declared in
Expand Down
Loading