Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
4ae12c2
47: super plan for init-demo subcommand (Austin fixture for PyPI users)
wjduenow May 11, 2026
a937b3e
47: bump plan to devolved + capture beads manifest
wjduenow May 11, 2026
a7c58fa
47-1: bootstrap src/signalforge/_demo/ tree + parity test
wjduenow May 12, 2026
0965527
Merge bead bd_1-scaffolding-t1o.1: US-001 bootstrap _demo/ tree + par…
wjduenow May 12, 2026
6e9aa9f
47-3: public signalforge.demo.copy_demo + DemoError hierarchy
wjduenow May 12, 2026
43c1fd6
47-2: wire wheel packaging include directive + wheel_smoke marker
wjduenow May 12, 2026
ea0002a
Merge bead bd_1-scaffolding-t1o.3: US-003 public signalforge.demo.cop…
wjduenow May 12, 2026
d04bf46
Merge bead bd_1-scaffolding-t1o.2: US-002 wire wheel packaging + whee…
wjduenow May 12, 2026
f7da7ce
47-4: CLI init-demo subcommand + four typed CLI errors
wjduenow May 12, 2026
82b63aa
Merge bead bd_1-scaffolding-t1o.4: US-004 CLI init-demo subcommand + …
wjduenow May 12, 2026
4ba04f6
47-6: subprocess smoke test for signalforge init-demo --help
wjduenow May 12, 2026
976ab7c
47-7: docs — README Quick Start + cli-ops.md § init-demo + CLAUDE.md
wjduenow May 12, 2026
6118e5a
Merge bead bd_1-scaffolding-t1o.6: US-006 subprocess smoke for init-d…
wjduenow May 12, 2026
429b3d6
Merge bead bd_1-scaffolding-t1o.7: US-007 docs README + cli-ops + CLA…
wjduenow May 12, 2026
ce42376
47-5: 5-surface parity test for init-demo + --force
wjduenow May 12, 2026
b8f9be4
Merge bead bd_1-scaffolding-t1o.5: US-005 5-surface parity test
wjduenow May 12, 2026
80630b2
Merge dev into feature/47-init-demo (catch up #48 + #49)
wjduenow May 12, 2026
ca57423
47-8: QG pass 1 — refactor signalforge.demo to subpackage + remediati…
wjduenow May 12, 2026
7524b48
47-8: QG pass 2 — clear "dest exists but not a directory" error path
wjduenow May 12, 2026
25832d8
47-8: QG pass 3 — env_var() rendering in profile loader + logger gate…
wjduenow May 12, 2026
5e2b234
47-8: QG pass 4 — add ProfileEnvVarUnsetError to warehouse __all__ + …
wjduenow May 12, 2026
58e2bbe
47-9: patterns & memory — Hatch include directive + wheel_smoke marke…
wjduenow May 12, 2026
6d4e616
47: cover codecov-flagged gaps in init_demo handler + error constructors
wjduenow May 12, 2026
9a5f9df
47: address PR #78 review feedback (6 fixes + 2 false positives)
wjduenow May 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 19 additions & 7 deletions .claude/rules/cli-layer.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,28 @@ src/signalforge/cli/
# map_exception_to_exit_code, _safe_excepthook, _EXCEPTION_TO_EXIT_CODE,
# progress helpers (should_emit_progress / format_elapsed /
# emit_progress_entry / emit_progress_done)
errors.py # CliError + CliPathError + CliInputError
errors.py # CliError + CliPathError + CliInputError + CliInitDemo*
generate.py # add_parser + cmd_generate (the full pipeline)
init_demo.py # add_parser + cmd_init_demo (copy bundled demo to disk; issue #47)
lint.py # add_parser + cmd_lint (config-only validator)
version.py # add_parser + cmd_version (prints signalforge __version__)
```

Flat layout — one module per subcommand, no nested directories, no `__main__.py`. Mirrors clauditor's CLI shape (16 subcommand modules in clauditor; SignalForge ships three for v0.1). Every subcommand module exports exactly two public symbols: `add_parser(subparsers) -> None` (registers the subparser, no return) and `cmd_<name>(args) -> int` (handler returning the exit code). The top-level `main(argv: list[str] | None = None) -> int` accepts an explicit argv list (defaults to `sys.argv[1:]`) — that's what makes in-process testing trivial: tests call `main([...])` directly, assert on the returned `int` and capsys output, and never spawn a subprocess.
Flat layout — one module per subcommand, no nested directories, no `__main__.py`. Mirrors clauditor's CLI shape (16 subcommand modules in clauditor; SignalForge ships four as of #47). Every subcommand module exports exactly two public symbols: `add_parser(subparsers) -> None` (registers the subparser, no return) and `cmd_<name>(args) -> int` (handler returning the exit code). The top-level `main(argv: list[str] | None = None) -> int` accepts an explicit argv list (defaults to `sys.argv[1:]`) — that's what makes in-process testing trivial: tests call `main([...])` directly, assert on the returned `int` and capsys output, and never spawn a subprocess.

When v0.2 adds a new subcommand (`signalforge doctor`, `signalforge profile`, ...), match the precedent: one new module under `src/signalforge/cli/`, register via `add_parser(subparsers)` from `main()`, return an int from `cmd_<name>(args)`.

## Library-surface pattern: CLI handler wraps a public lib module at the boundary (issue #47)

Issue #47 introduces a new pattern for subcommands that have a useful programmatic surface — `signalforge init-demo` ships both as a CLI subcommand AND as a public Python function `signalforge.demo.copy_demo(dest, *, force=False) -> Path`. The split:

- **`signalforge.demo`** (subpackage) — the public library entry point. Owns its own typed-error hierarchy (`DemoError` base + `DemoPathError`, `DemoDestExistsError`, `DemoDestUnsafeError`, `DemoFixtureMissingError`). Errors carry an optional `remediation` and a `cause` kwarg mirrored from every other `signalforge.*.errors` module's layer-base pattern. The library function returns useful work product (`Path`) — not just side effects — so notebook / script callers have a clean programmatic surface.
- **`signalforge.cli.init_demo`** (CLI module) — thin handler that argparse-parses its inputs, calls into `signalforge.demo.copy_demo(...)` inside the single `try/except Exception` boundary (DEC-016), and wraps every `DemoError` subclass into the matching `CliInitDemo*Error` (tier 2 for input-validation failures, tier 1 for broken-install / filesystem failures). The CLI owns the next-steps message + exit-code mapping; the library function stays clean of CLI concerns.

This is "two-layer error wrapping" — library typed errors are public (catchable by library callers), CLI typed errors are public (registered in `_EXCEPTION_TO_EXIT_CODE`), and the CLI handler does the translation at the boundary. The 7th AST scan walks BOTH `demo/errors.py` AND `cli/errors.py` so missing tier mappings on either side fail loud. Defence-in-depth: the lower-level `DemoError` subclasses are ALSO registered in `_EXCEPTION_TO_EXIT_CODE` with the same tiers as their CLI wrappers, so a v0.2 contributor who adds a new `Demo*Error` subclass and forgets to wire the CLI wrapper still gets a sensible exit code via `map_exception_to_exit_code`'s MRO walk.

When v0.2 adds a similar dual-surface subcommand (e.g., `signalforge fetch-rubrics` library-callable from a notebook, or `signalforge configure` from a CI bootstrap script), follow this pattern: public lib module with its own `errors.py`, CLI handler wraps at the boundary, both layers' errors land in the exit-code mapping.

## Four-tier exit-code taxonomy (DEC-008, DEC-019, DEC-024)

Every `cmd_<name>` handler returns an integer drawn from exactly four values. Ported from clauditor's `llm-cli-exit-code-taxonomy.md` rule; the wording is locked because CI parsers across repos key on the same boundary. **Do NOT invent a fifth category. Do NOT collapse 2 and 3.**
Expand All @@ -34,7 +46,7 @@ Every `cmd_<name>` handler returns an integer drawn from exactly four values. Po

The mapping table lives at `signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE`. `map_exception_to_exit_code(exc)` walks `type(exc).__mro__` against the table so subclasses inherit their parent's tier; an unregistered type (or a bare `Exception`) lands at tier 1 per DEC-016 (the panic path).

The 7th AST scan in `tests/test_audit_completeness.py` (DEC-024) walks every `src/signalforge/*/errors.py` (nine modules: the eight stage layers plus `cli/errors.py`) and asserts every concrete `class <Name>Error(...):` declaration appears in the mapping table. Excluded bases: the nine per-stage abstract bases (`ManifestError`, `WarehouseError`, `SafetyError`, `LLMError`, `LLMHelperError`, `DraftError`, `PruneError`, `GradeError`, `DiffError`, `CliError`). A new typed exception lands without a tier mapping → test fails loud.
The 7th AST scan in `tests/test_audit_completeness.py` (DEC-024) walks every `src/signalforge/*/errors.py` (ten modules as of #47: the eight pipeline stages plus `cli/errors.py` plus the new `demo/errors.py`) and asserts every concrete `class <Name>Error(...):` declaration appears in the mapping table. Excluded bases: the ten per-stage abstract bases (`ManifestError`, `WarehouseError`, `SafetyError`, `LLMError`, `DraftError`, `PruneError`, `GradeError`, `DiffError`, `CliError`, `DemoError`). **NOTE:** `LLMHelperError` is deliberately NOT excluded — it is raised directly in `signalforge.llm.client` (three sites), so it's a concrete leaf for taxonomy purposes and must appear in `_EXCEPTION_TO_EXIT_CODE`. A new typed exception lands without a tier mapping → test fails loud.

See clauditor's `.claude/rules/llm-cli-exit-code-taxonomy.md` for the source rule in the See-Also footer.

Expand Down Expand Up @@ -74,10 +86,10 @@ The exit-code mapping is a `dict[type[BaseException], int]` keyed by exception c
Three load-bearing invariants:

1. **One entry per concrete error class.** Every concrete `class <Name>Error(...):` declaration in any `src/signalforge/*/errors.py` module gets exactly one entry in the table. Adding multiple entries (the same class registered at two tiers) is a typo — the dict semantics keep only the last one and the test won't notice.
2. **Abstract bases land in `_EXCEPTION_MAPPING_EXCLUDED_BASES`, not the table.** The nine per-stage abstract bases (`ManifestError`, `WarehouseError`, `SafetyError`, `LLMError`, `LLMHelperError`, `DraftError`, `PruneError`, `GradeError`, `DiffError`, `CliError` — eight pipeline stages plus the CLI base) are listed in the frozenset constant and excluded by the AST scan. Subclasses inherit via the MRO walk; registering an abstract base directly would make `map_exception_to_exit_code(exc_with_unrelated_concrete_base)` accidentally return the abstract-base tier, defeating the per-class precision.
2. **Abstract bases land in `_EXCEPTION_MAPPING_EXCLUDED_BASES`, not the table.** The ten per-stage abstract bases (`ManifestError`, `WarehouseError`, `SafetyError`, `LLMError`, `DraftError`, `PruneError`, `GradeError`, `DiffError`, `CliError`, `DemoError` — eight pipeline stages plus the CLI base plus the demo-layer base added in #47) are listed in the frozenset constant and excluded by the AST scan. `LLMHelperError` is deliberately NOT in the excluded set despite living one level below `LLMError` — it's raised directly in `signalforge.llm.client` (concrete leaf for taxonomy purposes). Subclasses inherit via the MRO walk; registering an abstract base directly would make `map_exception_to_exit_code(exc_with_unrelated_concrete_base)` accidentally return the abstract-base tier, defeating the per-class precision.
3. **An unregistered concrete class falls to tier 1 via the panic path.** `map_exception_to_exit_code` returns `1` for any class without an MRO match, mirroring `signalforge.cli._helpers._safe_excepthook`'s tier-1 default for bare `Exception`. The 7th AST scan ensures unregistered concretes are caught at test time, not runtime — but the runtime fallback is the safety net.

If v0.2 introduces a new intermediate abstract base (e.g., a `WarehouseTransientError` that sits between `WarehouseError` and the concrete `WarehouseRateLimitError`), add it to `_EXCEPTION_MAPPING_EXCLUDED_BASES` AND document the addition. The exclusion list is a contract surface, not a convenience cache. The same rule applies if a new pipeline subpackage ships its own `errors.py` (a tenth stage, e.g., `signalforge.cache`): the companion test `test_scan_7_discovers_every_per_stage_errors_module` asserts the scan walks exactly nine `errors.py` files, so the count must be bumped in lockstep with the new stage's abstract base getting added to the excluded-bases set.
If v0.2 introduces a new intermediate abstract base (e.g., a `WarehouseTransientError` that sits between `WarehouseError` and the concrete `WarehouseRateLimitError`), add it to `_EXCEPTION_MAPPING_EXCLUDED_BASES` AND document the addition. The exclusion list is a contract surface, not a convenience cache. The same rule applies if a new pipeline subpackage ships its own `errors.py` (an eleventh stage, e.g., `signalforge.cache`): the companion test `test_scan_7_discovers_every_per_stage_errors_module` asserts the scan walks exactly ten `errors.py` files, so the count must be bumped in lockstep with the new stage's abstract base getting added to the excluded-bases set. Issue #47 set the precedent by adding `signalforge.demo` (the 10th `errors.py`) and `DemoError` (the 10th excluded base) in lockstep.

## Multi-surface parity for behaviour changes (QG pass-3 lesson)

Expand Down Expand Up @@ -127,9 +139,9 @@ The CLI is the orchestration layer (NOT a stage-0 reader/parser per the `manifes

## 7th AST scan: every typed exception has an exit-code mapping (DEC-019, DEC-024)

`tests/test_audit_completeness.py::test_every_typed_error_is_in_exit_code_mapping_table` is the 7th AST scan in the project (after the six landed by #4 / #5 / #6 / #7). Walks every `*/errors.py` under `src/signalforge/`, collects each `class <Name>Error(...):` declaration via `ast.ClassDef`, and asserts the class is registered in `signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE`. Excludes the nine per-stage abstract bases (frozenset constant `_EXCEPTION_MAPPING_EXCLUDED_BASES`); subclasses inherit via the MRO walk in `map_exception_to_exit_code`.
`tests/test_audit_completeness.py::test_every_typed_error_is_in_exit_code_mapping_table` is the 7th AST scan in the project (after the six landed by #4 / #5 / #6 / #7). Walks every `*/errors.py` under `src/signalforge/`, collects each `class <Name>Error(...):` declaration via `ast.ClassDef`, and asserts the class is registered in `signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE`. Excludes the ten per-stage abstract bases (frozenset constant `_EXCEPTION_MAPPING_EXCLUDED_BASES`); subclasses inherit via the MRO walk in `map_exception_to_exit_code`.

A companion test `test_scan_7_discovers_every_per_stage_errors_module` asserts the scan walks exactly nine `errors.py` files (one per stage subpackage). A future stage that forgets to ship `errors.py` (or moves the CLI errors to a sibling location) breaks this test loudly.
A companion test `test_scan_7_discovers_every_per_stage_errors_module` asserts the scan walks exactly ten `errors.py` files (one per stage subpackage, including `demo/errors.py` from #47). A future stage that forgets to ship `errors.py` (or moves the CLI errors to a sibling location) breaks this test loudly.

Sanity test `test_exit_code_mapping_has_at_least_one_entry_per_tier` asserts every tier (1, 2, 3) has at least one entry in the table — guards against an accidental mass-rename / deletion.

Expand Down
30 changes: 29 additions & 1 deletion .claude/rules/python-build.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,34 @@ packages = ["src/<package>"]

Do NOT rely on Hatchling auto-discovery for src layout. The wheel will silently produce an empty package if you forget this. Always declare explicitly. (DEC-011, learned the slow way.)

## Shipping package data (non-`.py` files) — explicit `include` directive (issue #47)

Hatchling's default `packages` declaration ships `.py` files under the named tree. Non-`.py` data files (YAML, JSON, SQL, dotfiles like `.gitignore`) are **not** auto-discovered — they need an explicit `include` entry alongside `packages`:

```toml
[tool.hatch.build.targets.wheel]
packages = ["src/signalforge"]
include = ["src/signalforge/_demo"] # ship the bundled demo tree
```

Defence-in-depth: even when the current Hatchling behaviour appears to ship sibling data files under `packages`, the `include` directive is a contract surface that survives Hatchling version drift. Issue #47's empirical investigation found that current Hatchling versions sometimes do auto-include but the behaviour is not contractually documented — the `include` directive makes it explicit and gated by a CI test (see below).

**Maintainer-only `wheel_smoke` gate.** Issue #47 (DEC-003) lands a `@pytest.mark.wheel_smoke` test that shells out `python -m build --wheel` (or `uvx --from build pyproject-build` when `build` isn't in the venv), opens the artifact via `zipfile.ZipFile`, and asserts the canonical file set appears under the expected wheel path. Registration:

```toml
[tool.pytest.ini_options]
markers = [
"wheel_smoke: maintainer-only; builds the wheel via python -m build and inspects the artifact (run with --no-cov)",
]
addopts = "... -m 'not bigquery and not anthropic and not cli_subprocess and not e2e and not wheel_smoke'"
```

Maintainer runs `pytest -m wheel_smoke --no-cov` before declaring a packaging-touching PR ready. The `--no-cov` is required because `--cov-fail-under` in default `addopts` fails marker-specific runs that exercise only a fraction of the codebase (mirrors `pytest -m bigquery --no-cov` and `pytest -m cli_subprocess --no-cov` precedents from `testing-signal.md`).

**Belt-and-braces verification step.** Before merging any PR that touches `[tool.hatch.build.targets.wheel]`, run `python -m build --wheel && unzip -l dist/*.whl | grep <expected-path>` locally and inspect the file list. The wheel_smoke marker catches absence; the manual `unzip -l` catches surprising additions (e.g. cache directories, hidden build artefacts).

**Dotfile inclusion is fragile.** Hatchling's glob behaviour on dotfiles (`.gitignore`, `.env.example`) varies by version. The wheel_smoke test must explicitly assert dotfile presence alongside regular files — a regression that drops only the dotfile would otherwise slip through. Fallback if Hatchling silently strips a dotfile: ship as a non-dot name (e.g. `gitignore.demo`) and rewrite to the dot-name at copy time in the consuming code (issue #47 DEC-006 documents the fallback; not needed in v0.1 because current Hatchling preserves dotfiles under `include`).

## Editable install (zsh-safe)

```bash
Expand All @@ -59,4 +87,4 @@ When CI widens to a Python matrix (likely v0.3, in lockstep with `ci-supply-chai

## Reference

`plans/super/1-project-scaffolding.md` — DEC-002, DEC-004, DEC-011, DEC-014. Issue #46 — Python version reconciliation.
`plans/super/1-project-scaffolding.md` — DEC-002, DEC-004, DEC-011, DEC-014. Issue #46 — Python version reconciliation. `plans/super/47-init-demo.md` — DEC-002 (Hatch `include` for non-`.py` package data), DEC-003 (`wheel_smoke` maintainer-gate pattern), DEC-006 (dotfile inclusion fallback).
7 changes: 5 additions & 2 deletions .claude/rules/testing-signal.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,16 +99,19 @@ Revisit when actual coverage exceeds `<N> + 5` for two consecutive `dev` builds.

### Known gap: excluded markers (DEC-004)

Coverage measures only the default pytest set. Tests gated behind `bigquery`, `anthropic`, `cli_subprocess`, and `e2e` markers are excluded by addopts (`-m 'not bigquery and not anthropic and not cli_subprocess and not e2e'`). Those code paths are exercised via fakes in unit tests; the real-network paths are not instrumented.
Coverage measures only the default pytest set. Tests gated behind `bigquery`, `anthropic`, `cli_subprocess`, `e2e`, and `wheel_smoke` markers are excluded by addopts (`-m 'not bigquery and not anthropic and not cli_subprocess and not e2e and not wheel_smoke'`). Those code paths are exercised via fakes in unit tests; the real-network paths and the actual `python -m build` invocation are not instrumented.

Because `--cov-fail-under` is in `addopts`, marker-specific runs (`pytest -m cli_subprocess`, `pytest -m bigquery`, `pytest -m e2e`) will fail the coverage gate. Use `--no-cov` for these runs:
Because `--cov-fail-under` is in `addopts`, marker-specific runs (`pytest -m cli_subprocess`, `pytest -m bigquery`, `pytest -m e2e`, `pytest -m wheel_smoke`) will fail the coverage gate. Use `--no-cov` for these runs:

```bash
pytest -m cli_subprocess --no-cov
pytest -m wheel_smoke --no-cov
SF_RUN_BQ=1 pytest -m bigquery --no-cov
SF_RUN_BQ=1 GOOGLE_CLOUD_PROJECT=<billing-project> ANTHROPIC_API_KEY=sk-... pytest -m e2e --no-cov
```

**`wheel_smoke` marker (issue #47).** Maintainer-only gate added by issue #47 to verify wheel-build packaging without coupling it to default CI. The single test (`tests/test_wheel_packaging.py`) shells out `python -m build --wheel --outdir <tmp>` (or `uvx --from build pyproject-build` when `build` isn't in the venv), opens the artifact via `zipfile.ZipFile`, and asserts the canonical demo file set appears under `signalforge/_demo/`. Catches `pyproject.toml` `[tool.hatch.build.targets.wheel] include` regressions that editable-install tests cannot — see `python-build.md` § "Shipping package data" for the full pattern.

## End-to-end gated tests (issue #10)

Established by issue #10 (e2e smoke test against `bigquery-public-data`). Apply to any new test that exercises the full pipeline against a real warehouse + a real LLM provider.
Expand Down
Loading
Loading