diff --git a/.claude/rules/cli-layer.md b/.claude/rules/cli-layer.md index 5fb91394..a7383b3b 100644 --- a/.claude/rules/cli-layer.md +++ b/.claude/rules/cli-layer.md @@ -13,16 +13,28 @@ src/signalforge/cli/ # map_exception_to_exit_code, _safe_excepthook, _EXCEPTION_TO_EXIT_CODE, # progress helpers (should_emit_progress / format_elapsed / # emit_progress_entry / emit_progress_done) - errors.py # CliError + CliPathError + CliInputError + errors.py # CliError + CliPathError + CliInputError + CliInitDemo* generate.py # add_parser + cmd_generate (the full pipeline) + init_demo.py # add_parser + cmd_init_demo (copy bundled demo to disk; issue #47) lint.py # add_parser + cmd_lint (config-only validator) version.py # add_parser + cmd_version (prints signalforge __version__) ``` -Flat layout — one module per subcommand, no nested directories, no `__main__.py`. Mirrors clauditor's CLI shape (16 subcommand modules in clauditor; SignalForge ships three for v0.1). Every subcommand module exports exactly two public symbols: `add_parser(subparsers) -> None` (registers the subparser, no return) and `cmd_(args) -> int` (handler returning the exit code). The top-level `main(argv: list[str] | None = None) -> int` accepts an explicit argv list (defaults to `sys.argv[1:]`) — that's what makes in-process testing trivial: tests call `main([...])` directly, assert on the returned `int` and capsys output, and never spawn a subprocess. +Flat layout — one module per subcommand, no nested directories, no `__main__.py`. Mirrors clauditor's CLI shape (16 subcommand modules in clauditor; SignalForge ships four as of #47). Every subcommand module exports exactly two public symbols: `add_parser(subparsers) -> None` (registers the subparser, no return) and `cmd_(args) -> int` (handler returning the exit code). The top-level `main(argv: list[str] | None = None) -> int` accepts an explicit argv list (defaults to `sys.argv[1:]`) — that's what makes in-process testing trivial: tests call `main([...])` directly, assert on the returned `int` and capsys output, and never spawn a subprocess. When v0.2 adds a new subcommand (`signalforge doctor`, `signalforge profile`, ...), match the precedent: one new module under `src/signalforge/cli/`, register via `add_parser(subparsers)` from `main()`, return an int from `cmd_(args)`. +## Library-surface pattern: CLI handler wraps a public lib module at the boundary (issue #47) + +Issue #47 introduces a new pattern for subcommands that have a useful programmatic surface — `signalforge init-demo` ships both as a CLI subcommand AND as a public Python function `signalforge.demo.copy_demo(dest, *, force=False) -> Path`. The split: + +- **`signalforge.demo`** (subpackage) — the public library entry point. Owns its own typed-error hierarchy (`DemoError` base + `DemoPathError`, `DemoDestExistsError`, `DemoDestUnsafeError`, `DemoFixtureMissingError`). Errors carry an optional `remediation` and a `cause` kwarg mirrored from every other `signalforge.*.errors` module's layer-base pattern. The library function returns useful work product (`Path`) — not just side effects — so notebook / script callers have a clean programmatic surface. +- **`signalforge.cli.init_demo`** (CLI module) — thin handler that argparse-parses its inputs, calls into `signalforge.demo.copy_demo(...)` inside the single `try/except Exception` boundary (DEC-016), and wraps every `DemoError` subclass into the matching `CliInitDemo*Error` (tier 2 for input-validation failures, tier 1 for broken-install / filesystem failures). The CLI owns the next-steps message + exit-code mapping; the library function stays clean of CLI concerns. + +This is "two-layer error wrapping" — library typed errors are public (catchable by library callers), CLI typed errors are public (registered in `_EXCEPTION_TO_EXIT_CODE`), and the CLI handler does the translation at the boundary. The 7th AST scan walks BOTH `demo/errors.py` AND `cli/errors.py` so missing tier mappings on either side fail loud. Defence-in-depth: the lower-level `DemoError` subclasses are ALSO registered in `_EXCEPTION_TO_EXIT_CODE` with the same tiers as their CLI wrappers, so a v0.2 contributor who adds a new `Demo*Error` subclass and forgets to wire the CLI wrapper still gets a sensible exit code via `map_exception_to_exit_code`'s MRO walk. + +When v0.2 adds a similar dual-surface subcommand (e.g., `signalforge fetch-rubrics` library-callable from a notebook, or `signalforge configure` from a CI bootstrap script), follow this pattern: public lib module with its own `errors.py`, CLI handler wraps at the boundary, both layers' errors land in the exit-code mapping. + ## Four-tier exit-code taxonomy (DEC-008, DEC-019, DEC-024) Every `cmd_` handler returns an integer drawn from exactly four values. Ported from clauditor's `llm-cli-exit-code-taxonomy.md` rule; the wording is locked because CI parsers across repos key on the same boundary. **Do NOT invent a fifth category. Do NOT collapse 2 and 3.** @@ -34,7 +46,7 @@ Every `cmd_` handler returns an integer drawn from exactly four values. Po The mapping table lives at `signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE`. `map_exception_to_exit_code(exc)` walks `type(exc).__mro__` against the table so subclasses inherit their parent's tier; an unregistered type (or a bare `Exception`) lands at tier 1 per DEC-016 (the panic path). -The 7th AST scan in `tests/test_audit_completeness.py` (DEC-024) walks every `src/signalforge/*/errors.py` (nine modules: the eight stage layers plus `cli/errors.py`) and asserts every concrete `class Error(...):` declaration appears in the mapping table. Excluded bases: the nine per-stage abstract bases (`ManifestError`, `WarehouseError`, `SafetyError`, `LLMError`, `LLMHelperError`, `DraftError`, `PruneError`, `GradeError`, `DiffError`, `CliError`). A new typed exception lands without a tier mapping → test fails loud. +The 7th AST scan in `tests/test_audit_completeness.py` (DEC-024) walks every `src/signalforge/*/errors.py` (ten modules as of #47: the eight pipeline stages plus `cli/errors.py` plus the new `demo/errors.py`) and asserts every concrete `class Error(...):` declaration appears in the mapping table. Excluded bases: the ten per-stage abstract bases (`ManifestError`, `WarehouseError`, `SafetyError`, `LLMError`, `DraftError`, `PruneError`, `GradeError`, `DiffError`, `CliError`, `DemoError`). **NOTE:** `LLMHelperError` is deliberately NOT excluded — it is raised directly in `signalforge.llm.client` (three sites), so it's a concrete leaf for taxonomy purposes and must appear in `_EXCEPTION_TO_EXIT_CODE`. A new typed exception lands without a tier mapping → test fails loud. See clauditor's `.claude/rules/llm-cli-exit-code-taxonomy.md` for the source rule in the See-Also footer. @@ -74,10 +86,10 @@ The exit-code mapping is a `dict[type[BaseException], int]` keyed by exception c Three load-bearing invariants: 1. **One entry per concrete error class.** Every concrete `class Error(...):` declaration in any `src/signalforge/*/errors.py` module gets exactly one entry in the table. Adding multiple entries (the same class registered at two tiers) is a typo — the dict semantics keep only the last one and the test won't notice. -2. **Abstract bases land in `_EXCEPTION_MAPPING_EXCLUDED_BASES`, not the table.** The nine per-stage abstract bases (`ManifestError`, `WarehouseError`, `SafetyError`, `LLMError`, `LLMHelperError`, `DraftError`, `PruneError`, `GradeError`, `DiffError`, `CliError` — eight pipeline stages plus the CLI base) are listed in the frozenset constant and excluded by the AST scan. Subclasses inherit via the MRO walk; registering an abstract base directly would make `map_exception_to_exit_code(exc_with_unrelated_concrete_base)` accidentally return the abstract-base tier, defeating the per-class precision. +2. **Abstract bases land in `_EXCEPTION_MAPPING_EXCLUDED_BASES`, not the table.** The ten per-stage abstract bases (`ManifestError`, `WarehouseError`, `SafetyError`, `LLMError`, `DraftError`, `PruneError`, `GradeError`, `DiffError`, `CliError`, `DemoError` — eight pipeline stages plus the CLI base plus the demo-layer base added in #47) are listed in the frozenset constant and excluded by the AST scan. `LLMHelperError` is deliberately NOT in the excluded set despite living one level below `LLMError` — it's raised directly in `signalforge.llm.client` (concrete leaf for taxonomy purposes). Subclasses inherit via the MRO walk; registering an abstract base directly would make `map_exception_to_exit_code(exc_with_unrelated_concrete_base)` accidentally return the abstract-base tier, defeating the per-class precision. 3. **An unregistered concrete class falls to tier 1 via the panic path.** `map_exception_to_exit_code` returns `1` for any class without an MRO match, mirroring `signalforge.cli._helpers._safe_excepthook`'s tier-1 default for bare `Exception`. The 7th AST scan ensures unregistered concretes are caught at test time, not runtime — but the runtime fallback is the safety net. -If v0.2 introduces a new intermediate abstract base (e.g., a `WarehouseTransientError` that sits between `WarehouseError` and the concrete `WarehouseRateLimitError`), add it to `_EXCEPTION_MAPPING_EXCLUDED_BASES` AND document the addition. The exclusion list is a contract surface, not a convenience cache. The same rule applies if a new pipeline subpackage ships its own `errors.py` (a tenth stage, e.g., `signalforge.cache`): the companion test `test_scan_7_discovers_every_per_stage_errors_module` asserts the scan walks exactly nine `errors.py` files, so the count must be bumped in lockstep with the new stage's abstract base getting added to the excluded-bases set. +If v0.2 introduces a new intermediate abstract base (e.g., a `WarehouseTransientError` that sits between `WarehouseError` and the concrete `WarehouseRateLimitError`), add it to `_EXCEPTION_MAPPING_EXCLUDED_BASES` AND document the addition. The exclusion list is a contract surface, not a convenience cache. The same rule applies if a new pipeline subpackage ships its own `errors.py` (an eleventh stage, e.g., `signalforge.cache`): the companion test `test_scan_7_discovers_every_per_stage_errors_module` asserts the scan walks exactly ten `errors.py` files, so the count must be bumped in lockstep with the new stage's abstract base getting added to the excluded-bases set. Issue #47 set the precedent by adding `signalforge.demo` (the 10th `errors.py`) and `DemoError` (the 10th excluded base) in lockstep. ## Multi-surface parity for behaviour changes (QG pass-3 lesson) @@ -127,9 +139,9 @@ The CLI is the orchestration layer (NOT a stage-0 reader/parser per the `manifes ## 7th AST scan: every typed exception has an exit-code mapping (DEC-019, DEC-024) -`tests/test_audit_completeness.py::test_every_typed_error_is_in_exit_code_mapping_table` is the 7th AST scan in the project (after the six landed by #4 / #5 / #6 / #7). Walks every `*/errors.py` under `src/signalforge/`, collects each `class Error(...):` declaration via `ast.ClassDef`, and asserts the class is registered in `signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE`. Excludes the nine per-stage abstract bases (frozenset constant `_EXCEPTION_MAPPING_EXCLUDED_BASES`); subclasses inherit via the MRO walk in `map_exception_to_exit_code`. +`tests/test_audit_completeness.py::test_every_typed_error_is_in_exit_code_mapping_table` is the 7th AST scan in the project (after the six landed by #4 / #5 / #6 / #7). Walks every `*/errors.py` under `src/signalforge/`, collects each `class Error(...):` declaration via `ast.ClassDef`, and asserts the class is registered in `signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE`. Excludes the ten per-stage abstract bases (frozenset constant `_EXCEPTION_MAPPING_EXCLUDED_BASES`); subclasses inherit via the MRO walk in `map_exception_to_exit_code`. -A companion test `test_scan_7_discovers_every_per_stage_errors_module` asserts the scan walks exactly nine `errors.py` files (one per stage subpackage). A future stage that forgets to ship `errors.py` (or moves the CLI errors to a sibling location) breaks this test loudly. +A companion test `test_scan_7_discovers_every_per_stage_errors_module` asserts the scan walks exactly ten `errors.py` files (one per stage subpackage, including `demo/errors.py` from #47). A future stage that forgets to ship `errors.py` (or moves the CLI errors to a sibling location) breaks this test loudly. Sanity test `test_exit_code_mapping_has_at_least_one_entry_per_tier` asserts every tier (1, 2, 3) has at least one entry in the table — guards against an accidental mass-rename / deletion. diff --git a/.claude/rules/python-build.md b/.claude/rules/python-build.md index 1f18d95a..c29efc6f 100644 --- a/.claude/rules/python-build.md +++ b/.claude/rules/python-build.md @@ -35,6 +35,34 @@ packages = ["src/"] Do NOT rely on Hatchling auto-discovery for src layout. The wheel will silently produce an empty package if you forget this. Always declare explicitly. (DEC-011, learned the slow way.) +## Shipping package data (non-`.py` files) — explicit `include` directive (issue #47) + +Hatchling's default `packages` declaration ships `.py` files under the named tree. Non-`.py` data files (YAML, JSON, SQL, dotfiles like `.gitignore`) are **not** auto-discovered — they need an explicit `include` entry alongside `packages`: + +```toml +[tool.hatch.build.targets.wheel] +packages = ["src/signalforge"] +include = ["src/signalforge/_demo"] # ship the bundled demo tree +``` + +Defence-in-depth: even when the current Hatchling behaviour appears to ship sibling data files under `packages`, the `include` directive is a contract surface that survives Hatchling version drift. Issue #47's empirical investigation found that current Hatchling versions sometimes do auto-include but the behaviour is not contractually documented — the `include` directive makes it explicit and gated by a CI test (see below). + +**Maintainer-only `wheel_smoke` gate.** Issue #47 (DEC-003) lands a `@pytest.mark.wheel_smoke` test that shells out `python -m build --wheel` (or `uvx --from build pyproject-build` when `build` isn't in the venv), opens the artifact via `zipfile.ZipFile`, and asserts the canonical file set appears under the expected wheel path. Registration: + +```toml +[tool.pytest.ini_options] +markers = [ + "wheel_smoke: maintainer-only; builds the wheel via python -m build and inspects the artifact (run with --no-cov)", +] +addopts = "... -m 'not bigquery and not anthropic and not cli_subprocess and not e2e and not wheel_smoke'" +``` + +Maintainer runs `pytest -m wheel_smoke --no-cov` before declaring a packaging-touching PR ready. The `--no-cov` is required because `--cov-fail-under` in default `addopts` fails marker-specific runs that exercise only a fraction of the codebase (mirrors `pytest -m bigquery --no-cov` and `pytest -m cli_subprocess --no-cov` precedents from `testing-signal.md`). + +**Belt-and-braces verification step.** Before merging any PR that touches `[tool.hatch.build.targets.wheel]`, run `python -m build --wheel && unzip -l dist/*.whl | grep ` locally and inspect the file list. The wheel_smoke marker catches absence; the manual `unzip -l` catches surprising additions (e.g. cache directories, hidden build artefacts). + +**Dotfile inclusion is fragile.** Hatchling's glob behaviour on dotfiles (`.gitignore`, `.env.example`) varies by version. The wheel_smoke test must explicitly assert dotfile presence alongside regular files — a regression that drops only the dotfile would otherwise slip through. Fallback if Hatchling silently strips a dotfile: ship as a non-dot name (e.g. `gitignore.demo`) and rewrite to the dot-name at copy time in the consuming code (issue #47 DEC-006 documents the fallback; not needed in v0.1 because current Hatchling preserves dotfiles under `include`). + ## Editable install (zsh-safe) ```bash @@ -59,4 +87,4 @@ When CI widens to a Python matrix (likely v0.3, in lockstep with `ci-supply-chai ## Reference -`plans/super/1-project-scaffolding.md` — DEC-002, DEC-004, DEC-011, DEC-014. Issue #46 — Python version reconciliation. +`plans/super/1-project-scaffolding.md` — DEC-002, DEC-004, DEC-011, DEC-014. Issue #46 — Python version reconciliation. `plans/super/47-init-demo.md` — DEC-002 (Hatch `include` for non-`.py` package data), DEC-003 (`wheel_smoke` maintainer-gate pattern), DEC-006 (dotfile inclusion fallback). diff --git a/.claude/rules/testing-signal.md b/.claude/rules/testing-signal.md index e517ea36..d5b13e1f 100644 --- a/.claude/rules/testing-signal.md +++ b/.claude/rules/testing-signal.md @@ -99,16 +99,19 @@ Revisit when actual coverage exceeds ` + 5` for two consecutive `dev` builds. ### Known gap: excluded markers (DEC-004) -Coverage measures only the default pytest set. Tests gated behind `bigquery`, `anthropic`, `cli_subprocess`, and `e2e` markers are excluded by addopts (`-m 'not bigquery and not anthropic and not cli_subprocess and not e2e'`). Those code paths are exercised via fakes in unit tests; the real-network paths are not instrumented. +Coverage measures only the default pytest set. Tests gated behind `bigquery`, `anthropic`, `cli_subprocess`, `e2e`, and `wheel_smoke` markers are excluded by addopts (`-m 'not bigquery and not anthropic and not cli_subprocess and not e2e and not wheel_smoke'`). Those code paths are exercised via fakes in unit tests; the real-network paths and the actual `python -m build` invocation are not instrumented. -Because `--cov-fail-under` is in `addopts`, marker-specific runs (`pytest -m cli_subprocess`, `pytest -m bigquery`, `pytest -m e2e`) will fail the coverage gate. Use `--no-cov` for these runs: +Because `--cov-fail-under` is in `addopts`, marker-specific runs (`pytest -m cli_subprocess`, `pytest -m bigquery`, `pytest -m e2e`, `pytest -m wheel_smoke`) will fail the coverage gate. Use `--no-cov` for these runs: ```bash pytest -m cli_subprocess --no-cov +pytest -m wheel_smoke --no-cov SF_RUN_BQ=1 pytest -m bigquery --no-cov SF_RUN_BQ=1 GOOGLE_CLOUD_PROJECT= ANTHROPIC_API_KEY=sk-... pytest -m e2e --no-cov ``` +**`wheel_smoke` marker (issue #47).** Maintainer-only gate added by issue #47 to verify wheel-build packaging without coupling it to default CI. The single test (`tests/test_wheel_packaging.py`) shells out `python -m build --wheel --outdir ` (or `uvx --from build pyproject-build` when `build` isn't in the venv), opens the artifact via `zipfile.ZipFile`, and asserts the canonical demo file set appears under `signalforge/_demo/`. Catches `pyproject.toml` `[tool.hatch.build.targets.wheel] include` regressions that editable-install tests cannot — see `python-build.md` § "Shipping package data" for the full pattern. + ## End-to-end gated tests (issue #10) Established by issue #10 (e2e smoke test against `bigquery-public-data`). Apply to any new test that exercises the full pipeline against a real warehouse + a real LLM provider. diff --git a/CLAUDE.md b/CLAUDE.md index fa0bd614..a5ffc0f8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Repository status -v0.1 alpha. Ten issues shipped: +v0.1 alpha. Eleven issues shipped: - **#1 (project scaffolding)** — `pyproject.toml` (Hatchling + src layout), `src/signalforge/__init__.py` with `__version__`, smoke test, ruff + pyright + pytest configs, GitHub Actions CI on PRs into `dev` and pushes to `main`, and `CONTRIBUTING.md`. - **#2 (manifest loader)** — `signalforge.manifest` subpackage: typed `Manifest` / `Model` (Pydantic v2), `load(project_dir, manifest_path=None) -> Manifest`, single-model resolver by `unique_id` or file path, schema-version tolerance v9–v12, symlink-hardened path canonicalisation, soft 200 MB warning. See `docs/manifest-loader-ops.md` for the operational reference. @@ -17,6 +17,8 @@ v0.1 alpha. Ten issues shipped: - **#9 (CLI entrypoint)** — `signalforge.cli` subpackage exposing the `signalforge` console-script entry point (registered via `[project.scripts]` in `pyproject.toml`, wired to `signalforge.cli:main`). Three subcommands: `generate` (the full draft → prune → grade → diff pipeline against a single model), `lint` (config-only validator that loads `signalforge.yml`, the dbt manifest, and the warehouse profile without making any LLM or warehouse call), and `version` (prints `signalforge <__version__>`). Flag set on `generate`: positional `` (unique_id or filename), `--project-dir` (absolute assertion, DEC-027 — must directly contain `dbt_project.yml`; the unflagged default walks up from cwd per DEC-001), `--manifest`, `--profiles-dir`, `--mode {schema-only,aggregate-only,sample}`, `--min-score`, `--write` / `--dry-run` (mutex), `--format {ansi,markdown,json}`, plus the observability triad `--quiet` / `--verbose` / `--no-color`. Four-tier exit-code taxonomy ported from clauditor (`0` success, `1` load/parse failure + panic-path catch, `2` input-validation / post-call invariant, `3` external-dep / fail-closed audit-write durability) wired through `signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE` and gated by the 7th AST scan in `tests/test_audit_completeness.py` — every concrete `*Error` subclass across nine `errors.py` modules either lands in the table or in the `_EXCEPTION_MAPPING_EXCLUDED_BASES` abstract-base allow-list, so a new typed exception without a tier mapping breaks the test loudly. DEC-021 sidecar-then-raise ordering for `GradeBelowThresholdError` (graduated from grade-layer's v0.2 reservation): `cmd_generate` writes the diff sidecar BEFORE raising the threshold error, so the operator's reproducibility artefact is always durable even on tier-2 failure. DEC-015 graduates `signalforge.diff.render_to_text(report, kind)` to the public surface so the CLI doesn't reach into `_renderers`. Eight typed-exception re-exports across `signalforge.draft` / `signalforge.llm` (`LLMOutputAnchorContractError`, `LLMOutputJSONError`, `LLMOutputValidationError`, `LLMResponseAuditWriteError`, `LLMResponseAuditRecordTooLargeError`, `PromptEnvelopeBreachError`, `LLMConnectionError`, `LLMResponseFormatError`) so the CLI's `try / except` ladder keys on the public symbol from each layer's package init, not the private submodule path. No traceback ever leaks (DEC-016): every `cmd_` wraps its pipeline in a single `try / except Exception`, calls `format_error_to_stderr`, returns the mapped exit code; `_safe_excepthook` is installed via `sys.excepthook` for belt-and-braces unless `--verbose` is set (KeyboardInterrupt / SystemExit pass through unchanged). Stderr message shape is `ERROR: ` + optional `↳ Remediation:` footer for tier 1/3 and most tier 2; multi-violation tier 2 (drafter anchor-contract, lint multi-block) gets a header line plus ` - ` bullets via `format_error_to_stderr` as the single sink — typed errors never override `__str__` for the bullet shape. Path canonicalisation flows through `canonicalise_user_path(raw, project_dir)` for every user-supplied path; failures re-raise as `CliPathError` so the CLI's catch surface stays homogeneous. Logger grep-gate extends to `src/signalforge/cli/` (6 dirs as of #9). One subprocess-gated smoke test (`pytest -m cli_subprocess`) catches `[project.scripts]` regressions that in-process `main(argv)` testing can't. See `docs/cli-ops.md` for the operational reference, `plans/super/9-cli-entrypoint.md` for the design, and `.claude/rules/cli-layer.md` for the rules distilled from this ticket. - **#10 (e2e smoke test)** — `tests/fixtures/dbt_project_austin/` ships a minimal dbt project pointing at `bigquery-public-data.austin_bikeshare.bikeshare_trips` (one staging model `stg_bikeshare_trips` aliased directly to the source per Path A — sidesteps the `dbt run` materialisation step in the v0.1 smoke-test path), a hand-crafted `target/manifest.json` (regen via `regenerate.sh` invoking `uvx --from "dbt-bigquery==1.8.*" --with "dbt-core==1.8.*" dbt parse`), a locked `signalforge.yml` (`safety.mode: aggregate-only`, `prune.sample_strategy: materialised`, `grade.min_pass_rate: 0.95` / `min_mean_score: 0.95` / `total_budget_seconds: 600`, `llm.model: claude-sonnet-4-6`), and a paired in-process `tests/manifest/test_austin_fixture_loads.py` + `tests/cli/test_austin_fixture_config.py` that gate the fixture without env vars. The actual end-to-end test `tests/cli/test_e2e_bigquery_smoke.py` is gated by a new `@pytest.mark.e2e` marker (registered in `pyproject.toml`; excluded by default `addopts`) plus a runtime `pytest.skipif` for the env-var triple `SF_RUN_BQ=1`, `ANTHROPIC_API_KEY`, `GOOGLE_CLOUD_PROJECT` — the in-process `main(argv)` invocation asserts no traceback leaks (DEC-016 of `cli-layer.md`), exit code 0, and the durable `.signalforge/diff.json` + `.signalforge/grade.json` sidecars round-trip through typed deserialisers in `tests/cli/_e2e_helpers.py`. Operator-facing walkthrough lives at `docs/e2e-smoke-test.md` (business-language intro, prerequisites, run command, cost ceiling, troubleshooting matrix); the README's `## Trying it out` section is the quickstart that points at the docs file. Maintainer-only command: `pytest -m e2e --no-cov` (mirrors `bigquery` / `anthropic` / `cli_subprocess` gated-marker convention from `testing-signal.md`). The live e2e run also drove three follow-up code changes: an explicit JSON-shape example added to the drafter system prompt (`_PROMPT_VERSION` rotated `1c558064` → `c7d15d59`), `LLMCacheTooSmallError` retired from the public surface in favour of soft-drop-and-log semantics in `signalforge.llm.client` (Anthropic silently no-ops sub-minimum cache markers; the prior hard-fail blocked any caller whose cached block was naturally small — the grade layer's compact rubric was 291 tokens), and the grade-progress count in `signalforge.cli.generate` corrected to compute artifact count from the candidate (the prior `prune_result.kept_count` substitution emitted "0 artifacts" runs whenever prune dropped everything). See `plans/super/10-e2e-bigquery-smoke.md` for the design + decisions and `tests/cli/_e2e_helpers.py` for the typed sidecar-deserialisation helpers. +- **#47 (init-demo)** — `signalforge.demo` + `signalforge.cli.init_demo` subpackages: ships the Austin bikeshare demo dbt project as wheel package-data so a first-run PyPI user (`pip install signalforge-dbt`) can `signalforge init-demo /tmp/sf-austin && signalforge generate ...` without cloning the repo or hand-editing `profiles.yml`. The bundled tree lives under `src/signalforge/_demo/` and ships via an explicit `[tool.hatch.build.targets.wheel] include = ["src/signalforge/_demo"]` directive in `pyproject.toml` (DEC-002 — Hatchling's default `packages` glob picks up `.py` only, so the data files were silently dropped before this ticket; verified empirically by `unzip -l dist/*.whl | grep _demo/`). Public library entry point `signalforge.demo.copy_demo(dest, *, force=False) -> Path` (DEC-012) extracts the tree via `importlib.resources.files("signalforge").joinpath("_demo")` so wheel / editable / zipapp installs all resolve; CLI subcommand `signalforge init-demo []` (DEC-009 flat layout, mirrors `lint` / `version`) wraps it. `--force` triggers an atomic `rmtree + copytree` replace (DEC-001) and refuses `/` / `$HOME` / `Path.cwd()` as a blast-radius guard (`CliInitDemoDestUnsafeError`). Four new CLI typed errors (`CliInitDemoDestExistsError` / `CliInitDemoDestUnsafeError` / `CliInitDemoFixtureMissingError` / `CliInitDemoCopyError`) all register in `_EXCEPTION_TO_EXIT_CODE` (tier 2 / tier 2 / tier 1 / tier 1) and are auto-gated by the 7th AST scan. Plain-text next-steps message printed to stdout on success (DEC-014) names `GOOGLE_CLOUD_PROJECT` + `ANTHROPIC_API_KEY` + the three first-run commands; survives `--no-color` because it carries no colour codes. Test fixture and demo tree are kept in lockstep by a parity test (`tests/test_demo_fixture_parity.py`, DEC-008) that walks both trees and asserts byte-equality except for two documented rewrites (`profiles.yml` ships with `env_var('GOOGLE_CLOUD_PROJECT')`; `.gitignore` is slimmed for the demo audience); `tests/fixtures/dbt_project_austin/regenerate.sh` updates both sides in lockstep (DEC-015). New maintainer-only `@pytest.mark.wheel_smoke` marker (DEC-003 — registered in `pyproject.toml`; excluded by default `addopts`; run via `pytest -m wheel_smoke --no-cov`) shells out `python -m build --wheel` and inspects the artifact via `zipfile.ZipFile` to assert the canonical demo file set + `.gitignore` appear under `signalforge/_demo/`. Subprocess smoke test extends to cover `signalforge init-demo --help` (AC-5) under the existing `@pytest.mark.cli_subprocess` gate. The 5-surface parity test (`tests/cli/test_5_surface_parity_init_demo.py`, US-005 pin) asserts `init-demo` + `--force` appear consistently across argparse help, handler docstring, `docs/cli-ops.md`, and this plan's DEC list. See `plans/super/47-init-demo.md` for the design + decisions and `docs/cli-ops.md` § `signalforge init-demo` for the operational reference. + Design happens in the open on the `dev` branch; v0.1 feature work is complete and the CLI ships the operator-facing surface. ## Public API surface (v0.1 + v0.2 additions) @@ -31,6 +33,7 @@ Design happens in the open on the `dev` branch; v0.1 feature work is complete an - `signalforge.grade.grade_artifacts`, `GradingReport`, `GradingResult`, `GradeConfig`, `load_grade_config`, `Criterion`, `Rubric`, `GradeThresholds`, `DEFAULT_RUBRIC`, `GradeEvent`, the typed literal (`GradeOutputViolationType`), and the `GradeError` hierarchy (nine classes). Documented in `docs/grade-ops.md`. - `signalforge.diff.render_diff`, `signalforge.diff.render_to_text` (graduated by #9 so the CLI doesn't reach into `_renderers`), `load_diff_config`, `DiffConfig`, `DiffReport`, `DiffEntry`, the typed literal (`Tier`), and the `DiffError` hierarchy (seven classes). Documented in `docs/diff-ops.md`. - `signalforge.cli.main(argv: list[str] | None = None) -> int` (the in-process entry point used by tests; the `signalforge` console-script registered under `[project.scripts]` in `pyproject.toml` wraps it via `sys.exit(main())`), and the CLI-layer `CliError` hierarchy (`CliError`, `CliPathError`, `CliInputError`). Documented in `docs/cli-ops.md`. +- `signalforge.demo.copy_demo(dest: Path | str, *, force: bool = False) -> Path` (issue #47 — the public library entry point that copies the bundled `signalforge._demo/` tree into ``; extracts via `importlib.resources.files("signalforge").joinpath("_demo")` so wheel installs, editable installs, and zipapp/zipimport cases all resolve correctly). Atomic-replace semantics on `force=True` (`shutil.rmtree` then `shutil.copytree`); refuses non-empty `` without `force=True` (`DemoDestExistsError`); refuses `force=True` against `/`, `Path.home()`, or `Path.cwd()` as a blast-radius guard (`DemoDestUnsafeError`). The `DemoError` hierarchy (`DemoError`, `DemoPathError`, `DemoDestExistsError`, `DemoDestUnsafeError`, `DemoFixtureMissingError`) is the lower-level typed-error surface; the CLI `init-demo` handler wraps each subclass into a `CliInitDemo*Error` at the handler boundary so the CLI exit-code taxonomy stays homogeneous (DEC-012). The four new CLI wrappers (`CliInitDemoDestExistsError`, `CliInitDemoDestUnsafeError`, `CliInitDemoFixtureMissingError`, `CliInitDemoCopyError`) all register in `_EXCEPTION_TO_EXIT_CODE` (tier 2 / tier 2 / tier 1 / tier 1) and are auto-gated by the 7th AST scan. Documented in `docs/cli-ops.md` § `signalforge init-demo`. v0.2 additions (issue #22 — temp-table-materialised sample): diff --git a/README.md b/README.md index 79d6b48c..ffe4c85a 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ > LLM-drafted dbt schema.yml, tests, and docs — pruned against real warehouse data so only signal-bearing tests ship. -**Status:** v0.1 alpha. Nine issues shipped — single-model draft + warehouse prune, BigQuery adapter, `signalforge` CLI. Designing in the open on the `dev` branch. +**Status:** v0.1 alpha. Eleven issues shipped — single-model draft + warehouse prune, BigQuery adapter, `signalforge` CLI, `signalforge init-demo` for first-run UX. Designing in the open on the `dev` branch. ## Why this exists @@ -43,15 +43,16 @@ The grading layer reuses [clauditor](https://github.com/wjduenow/clauditor)'s LL ## Quick start -The repo ships a minimal dbt fixture under -`tests/fixtures/dbt_project_austin/` pointing at the public -`bigquery-public-data.austin_bikeshare.bikeshare_trips` dataset, so -you can run `signalforge` end-to-end against a real warehouse with no -infrastructure beyond a Google Cloud billing project and an Anthropic -API key. A run scans ~200–500 MB of BigQuery (well under $0.01 at -on-demand pricing) plus ~$0.13 of Anthropic spend (one draft call + -~84 grade calls on Sonnet 4.6); end-to-end wall-clock is roughly -5–6 minutes. +The wheel ships a minimal dbt demo project (Austin bikeshare staging +model against the public +`bigquery-public-data.austin_bikeshare.bikeshare_trips` dataset), +copied out of the install via `signalforge init-demo`, so you can +run `signalforge` end-to-end against a real warehouse with no +infrastructure beyond a Google Cloud billing project and an +Anthropic API key. A run scans ~200–500 MB of BigQuery (well under +$0.01 at on-demand pricing) plus ~$0.13 of Anthropic spend (one +draft call + ~84 grade calls on Sonnet 4.6); end-to-end wall-clock +is roughly 5–6 minutes. ### 1. Install @@ -104,26 +105,11 @@ Full reference: [docs/safety-ops.md](docs/safety-ops.md), ### 4. Prepare the fixture -Copy the fixture to a writable tmp dir and rewrite the profile to bill -queries against your project (the committed `profiles.yml` is oriented -at `dbt parse` and pins `project: bigquery-public-data`, which you -can't bill to itself): +Copy the bundled demo project to a writable directory and run +`signalforge` against it: ```bash -mkdir -p /tmp/sf-austin -cp -r tests/fixtures/dbt_project_austin/. /tmp/sf-austin/ -cat > /tmp/sf-austin/profiles.yml <` prints the projected USD + warehouse bytes without making any billable Anthropic or warehouse call (one `count_tokens` round-trip per prompt @@ -208,10 +201,11 @@ of the same flow as a gated test (`pytest -m e2e --no-cov`): ## CLI -Three subcommands ship in v0.1: +Four subcommands ship in v0.1: ```bash signalforge generate # full pipeline; --mode, --min-score, --write/--dry-run, --format +signalforge init-demo [] # copy the bundled Austin demo project into ; --force signalforge lint # validate signalforge.yml config blocks signalforge version # print the SignalForge version ``` diff --git a/docs/cli-ops.md b/docs/cli-ops.md index 70eacf47..8080ec40 100644 --- a/docs/cli-ops.md +++ b/docs/cli-ops.md @@ -54,7 +54,7 @@ After install, the `signalforge` console script is registered via ## Subcommands -The CLI exposes three subcommands. `signalforge --help` prints the +The CLI exposes four subcommands. `signalforge --help` prints the top-level help; each subcommand has its own `--help` page (e.g. `signalforge generate --help`). @@ -241,6 +241,82 @@ The flag → config precedence chain is uniform across knobs: **CLI flag > `signalforge.yml` block > library default**. Library defaults are documented per-stage in each layer's ops doc. +### `signalforge init-demo []` + +Copy the bundled Austin bikeshare demo project (a minimal dbt +project pointing at the public +`bigquery-public-data.austin_bikeshare.bikeshare_trips` dataset) +out of the installed wheel into `` so a first-run PyPI user +can `cd` into a working project and exercise the full pipeline +without authoring their own dbt setup. The bundled `profiles.yml` +reads `GOOGLE_CLOUD_PROJECT` from the operator's environment, so +no profile editing is required. + +Wraps the public library entry point +`signalforge.demo.copy_demo(dest, *, force=False) -> Path`; the +CLI re-raises the lower-level `DemoError` subclasses as +`CliInitDemo*Error` wrappers at the handler boundary so the +four-tier exit-code taxonomy stays homogeneous (DEC-012 of +[`plans/super/47-init-demo.md`](../plans/super/47-init-demo.md)). + +Positional argument: + +- `` — Destination directory. Optional; default + `./signalforge-demo/`. Relative paths resolve against the + current working directory; `~` expands; + `Path(dest).expanduser().resolve(strict=False)` follows + symlinks and raises `CliPathError` on a cycle. **No + `--project-dir` containment gate applies** — + `init-demo` is the one subcommand that *creates* a project + rather than operating *inside* one, so the + `canonicalise_user_path(...)` containment helper used by + every other CLI flag is deliberately bypassed (DEC-004). + +Flags: + +- `--force` — Atomically replace `` if it exists and is + non-empty (`shutil.rmtree` then `shutil.copytree`). Without + `--force`, a non-empty `` raises + `CliInitDemoDestExistsError` (tier 2); empty existing + directories proceed without `--force`. As a blast-radius + guard (DEC-001), `--force` refuses `/`, `$HOME`, and the + current working directory and raises + `CliInitDemoDestUnsafeError` (tier 2) on any of those. + +Exit codes (four-tier taxonomy; see § Four-tier exit-code +taxonomy for the full table): + +- `0` — copy succeeded; next-steps message printed to stdout. +- `1` — broken install (`CliInitDemoFixtureMissingError`: + the wheel didn't ship `signalforge/_demo/`), symlink-cycle + resolve failure (`CliPathError`), or generic filesystem + failure such as `ENOSPC` / `EACCES` / `EROFS` + (`CliInitDemoCopyError`). +- `2` — operator-side dest mistakes: + `CliInitDemoDestExistsError` (non-empty dest without + `--force`) or `CliInitDemoDestUnsafeError` (`--force` + against `/`, `$HOME`, or cwd). + +Output: on success, a plain-text "next steps" message lands on +stdout (DEC-014) — no ANSI colour codes, no Markdown, no env-var +*values*, just the env-var *names* (`GOOGLE_CLOUD_PROJECT` and +`ANTHROPIC_API_KEY`) and the three first-run commands an operator +needs to run (`cd `, `signalforge lint`, then +`signalforge generate models/staging/stg_bikeshare_trips.sql --dry-run`). +The message survives `--no-color` because it carries no colour +codes. + +Example: + +```bash +signalforge init-demo /tmp/sf-austin +cd /tmp/sf-austin +export GOOGLE_CLOUD_PROJECT= +export ANTHROPIC_API_KEY=sk-ant-... +signalforge lint +signalforge generate models/staging/stg_bikeshare_trips.sql --dry-run +``` + ### `signalforge lint` Validate the five existing `signalforge.yml` config blocks (`safety:`, diff --git a/plans/super/47-init-demo.md b/plans/super/47-init-demo.md new file mode 100644 index 00000000..fff9cd09 --- /dev/null +++ b/plans/super/47-init-demo.md @@ -0,0 +1,458 @@ +# 47: `signalforge init-demo` — ship Austin demo to PyPI users + +## Meta + +- **Ticket:** [GH #47](https://github.com/wjduenow/SignalForge/issues/47) +- **Branch:** `feature/47-init-demo` +- **Worktree:** `../worktrees/SignalForge/47-init-demo` +- **Phase:** devolved +- **PR:** [#78](https://github.com/wjduenow/SignalForge/pull/78) +- **Epic:** `bd_1-scaffolding-t1o` +- **Sessions:** + - 2026-05-11 — Phase 1 discovery (parallel research, scoping decisions locked) + - 2026-05-11 — Phase 2 architecture review (3 blockers + 4 concerns surfaced) + - 2026-05-11 — Phase 3 refinement (15 DECs locked), Phase 4 detailing (9 stories) + - 2026-05-11 — Phase 5 published as draft PR #78 + - 2026-05-11 — Phase 6 approved, Phase 7 devolved to beads + +## Beads manifest + +- **Epic:** `bd_1-scaffolding-t1o` — "47: signalforge init-demo subcommand" +- **Tasks:** + - `bd_1-scaffolding-t1o.1` — US-001 — Bootstrap `src/signalforge/_demo/` tree + parity test (no deps) + - `bd_1-scaffolding-t1o.2` — US-002 — Wire wheel packaging + `wheel_smoke` maintainer gate (depends on .1) + - `bd_1-scaffolding-t1o.3` — US-003 — Public `signalforge.demo.copy_demo` module (depends on .1) + - `bd_1-scaffolding-t1o.4` — US-004 — CLI `init-demo` subcommand + typed CLI errors (depends on .3) + - `bd_1-scaffolding-t1o.5` — US-005 — 5-surface parity test (depends on .4, .7) + - `bd_1-scaffolding-t1o.6` — US-006 — Subprocess `--help` smoke (depends on .4) + - `bd_1-scaffolding-t1o.7` — US-007 — Docs: README + `cli-ops.md` + `CLAUDE.md` (depends on .4) + - `bd_1-scaffolding-t1o.8` — US-008 — Quality Gate (depends on .1..7) + - `bd_1-scaffolding-t1o.9` — US-009 — Patterns & Memory (depends on .8) +- **Worktree:** `/home/wesd/Projects/worktrees/SignalForge/47-init-demo` +- **Branch:** `feature/47-init-demo` + +## Ticket summary + +Add `signalforge init-demo []` subcommand that copies a packaged Austin dbt-bikeshare demo out of the installed wheel into `` (default `./signalforge-demo/`). Refuses non-empty `` unless `--force` is set. Prints a one-screen "next steps" message naming the env vars + commands a PyPI user needs to actually run the demo. Replaces the broken `cp -r tests/fixtures/...` snippet in README Quick Start (which assumes a clone but is presented under `pip install signalforge-dbt`). + +**AC-1** `signalforge init-demo` works post-`pip install signalforge-dbt` with no repo files present. +**AC-2** Demo fixture lives in the wheel — verifiable via `unzip -l | grep _demo`. +**AC-3** `--force` semantics: refuses non-empty `` unless set. +**AC-4** README Quick Start + `docs/cli-ops.md` § Subcommands both updated. +**AC-5** Subprocess smoke test extends to cover `signalforge init-demo --help`. + +## Discovery + +### Codebase findings (key seams) + +- **CLI subcommand template** — `src/signalforge/cli/version.py` (40 lines) and `src/signalforge/cli/lint.py` (~240 lines) are the canonical shape. Each exports two public symbols: `add_parser(subparsers) -> None` and `cmd_(args) -> int`. Registered from `src/signalforge/cli/__init__.py:79-81`. `init-demo` follows this verbatim — new module `src/signalforge/cli/init_demo.py`. +- **CLI error layer** — `src/signalforge/cli/errors.py` ships `CliError` / `CliPathError` / `CliInputError`. New typed errors for this ticket subclass one of those bases and land in `_EXCEPTION_TO_EXIT_CODE` (7th AST scan auto-gates). +- **Path canonicalisation** — `src/signalforge/cli/_helpers.py::canonicalise_user_path(raw, project_dir)` is the project-wide gate. `init-demo`'s `` is not strictly inside a project_dir (the operator runs it before they have a project), so the canonicalisation contract needs a small adaptation — see refinement. +- **No existing `importlib.resources` consumer** — SignalForge does not currently load any package data at runtime. Greps for `importlib.resources`, `pkg_resources`, and `__file__`-relative resource lookups returned no hits under `src/signalforge/`. This ticket adds the project's first. +- **Hatch wheel target** — `pyproject.toml:36-37`: + ```toml + [tool.hatch.build.targets.wheel] + packages = ["src/signalforge"] + ``` + No `MANIFEST.in`, no `shared-data`, no `force-include`, no `include-package-data`. To ship the demo, files MUST live under `src/signalforge/` (current `packages` declaration includes everything under that tree); files outside it are not packaged. +- **Austin fixture inventory** — `tests/fixtures/dbt_project_austin/` (8 files, ~22 KB content + 13 KB manifest = ~64 KB total on disk): + - `dbt_project.yml` (670 B) + - `signalforge.yml` (547 B) + - `models/staging/sources.yml` (1.3 KB) + - `models/staging/stg_bikeshare_trips.sql` (662 B) + - `target/manifest.json` (13 KB) — locked dbt v1.8 manifest seed + - `profiles.yml` (1.4 KB) — current copy has a "DO NOT use signalforge against this" header and a `bigquery-public-data` placeholder that is billing-broken; demo copy needs a rewrite with `` placeholder + - `.gitignore` (761 B) — references issue #10 / DEC-021; can be slimmed for demo audience + - `regenerate.sh` (4.3 KB) — maintainer-only; uses `uvx dbt parse`; MUST NOT ship in demo +- **README Quick Start snippet to replace** — `README.md:112-129` (`mkdir -p /tmp/sf-austin && cp -r tests/fixtures/dbt_project_austin/. /tmp/sf-austin/` heredoc through `signalforge generate ...`). Also `README.md:47` (intro mention of `tests/fixtures/dbt_project_austin/`) and `README.md:161` (`/tmp/sf-austin/.signalforge/` reference) for cross-surface consistency. +- **`docs/cli-ops.md` § Subcommands template** — Lines 55-271 contain three existing entries (`generate`, `lint`, `version`). `lint` (244-263) is the closest shape precedent for `init-demo` (short, two-paragraph blurb + flags list). The line `"The CLI exposes three subcommands"` (line 57) must bump to four. +- **CLAUDE.md public API surface** — Line 33 documents `signalforge.cli.main(...)` + `CliError` hierarchy. If `init-demo` exposes a new `signalforge.demo` module (e.g., a `copy_demo(dest: Path, *, force: bool) -> None` library entry point), it lands in that bullet. Defaults assume CLI-only — see refinement. +- **Subprocess smoke test** — `tests/cli/test_subprocess_smoke.py::test_signalforge_version_via_subprocess` is the precedent. Marker `@pytest.mark.cli_subprocess`, excluded by default `addopts`, run via `pytest -m cli_subprocess --no-cov`. The new test follows the same shape but invokes `signalforge init-demo --help`. + +### Rules constraints (informing detailing) + +1. **`cli-layer.md` DEC-009** — Flat subcommand layout: one new module `src/signalforge/cli/init_demo.py` exporting `add_parser` + `cmd_init_demo`. Registered in `__init__.py`. No nested dirs. +2. **`cli-layer.md` DEC-008 / DEC-019 / DEC-024 (7th AST scan)** — Every new concrete `*Error` subclass under `cli/errors.py` lands in `_EXCEPTION_TO_EXIT_CODE` at an explicit tier. Likely additions: + - `CliInitDemoDestExistsError(CliInputError)` — tier 2 (non-empty dest, no `--force`); ticket-suggested behaviour reads as input-validation, not load-time-state-not-ready + - `CliInitDemoFixtureMissingError(CliError)` — tier 1 (the wheel didn't ship `_demo/`; broken install) + - Possibly `CliInitDemoCopyError(CliError)` — tier 1 (filesystem write failure) +3. **`cli-layer.md` DEC-016** — `cmd_init_demo` wraps the whole pipeline in one `try/except Exception`, returns `map_exception_to_exit_code(exc)`, never leaks a traceback. Floor-of-every-test assertion: `"Traceback" not in capsys.readouterr().err`. +4. **`cli-layer.md` DEC-017** — Stderr shape: `ERROR: ` plus optional `↳ Remediation:` footer. Multi-violation bullets only if a multi-error case lands (unlikely here). +5. **`cli-layer.md` DEC-019 (logger grep gate)** — AST-based gate scans `src/signalforge/cli/init_demo.py`. Any `_LOGGER` call uses lazy-format JSON (`_LOGGER.info("event: %s", json.dumps({...}))`); never f-string. Likely v0.1 init-demo has zero `_LOGGER` calls — stdout prints are the operator channel. +6. **`cli-layer.md` 5-surface parity** — argparse help string + handler docstring + `docs/cli-ops.md` § Subcommands entry + test name + DEC in this plan, all updated in the implementation PR. +7. **`cli-layer.md` DEC-027** — Path canonicalisation pattern. `init-demo`'s `` is NOT a project_dir assertion (no `dbt_project.yml` required); it's an output directory chosen by the operator. Adapt: resolve via `Path(dest).resolve()` for symlink safety, but don't require containment in a project tree. New typed error covers the "resolve failed" case. +8. **`python-build.md` DEC-011** — Wheel target packaging is non-negotiable: `[tool.hatch.build.targets.wheel] packages = ["src/signalforge"]` already covers everything under that tree, so placing `_demo/` files under `src/signalforge/_demo/` gets them shipped automatically. Hatch will NOT auto-discover files outside the declared `packages` — `tests/fixtures/...` is invisible to the wheel. +9. **`testing-signal.md` (no `assert True`)** — Every new test must be capable of failing on regression. In-process test calls `main(["init-demo", str(tmp_path)])`, asserts return == 0, asserts specific demo files exist at `tmp_path` (e.g., `(tmp_path / "models" / "staging" / "stg_bikeshare_trips.sql").is_file()`). +10. **`testing-signal.md` (subprocess-gated pattern)** — New subprocess test inherits the `@pytest.mark.cli_subprocess` marker; CI ignores it by default; maintainers run `pytest -m cli_subprocess --no-cov`. Single subprocess test is the source of truth for "the wheel actually exposes the script" — extend the existing test file rather than adding a new one. +11. **No new audit-event class, no new AST scan** — `init-demo` writes no JSONL audit, owns no fail-closed writer. The 7th AST scan auto-covers the new error classes. No 8th scan needed. +12. **CLAUDE.md "Public API surface"** — Only update if `init-demo` exposes a new public Python entry point (e.g., `signalforge.demo.copy_demo(...)`). If CLI-internal, no entry needed. + +### Scoping decisions (locked Phase 1) + +- **SD-1 — Two copies + parity test.** `src/signalforge/_demo/` is the shipped tree (lands in the wheel via existing `packages = ["src/signalforge"]`). `tests/fixtures/dbt_project_austin/` stays as the test fixture for issue #10's e2e smoke. A new parity test reads both trees and asserts byte-equality except for two documented rewrites: `profiles.yml` (shipped copy uses `env_var('GOOGLE_CLOUD_PROJECT')`; test copy keeps the maintainer-only header) and `.gitignore` (shipped copy slimmed for demo audience). `tests/fixtures/dbt_project_austin/regenerate.sh` is amended to update BOTH trees in lockstep. +- **SD-2 — `env_var('GOOGLE_CLOUD_PROJECT')` placeholder.** The shipped `profiles.yml` uses dbt's native `env_var(...)` lookup so an operator with `GOOGLE_CLOUD_PROJECT` set (already the README's recommendation) runs the demo with zero file edits. Aligns with `README.md:119-123` precedent. +- **SD-3 — In-process end-to-end + subprocess `--help`.** In-process test calls `main(["init-demo", str(tmp_path)])`, asserts return code 0, asserts representative files exist at `tmp_path` (covers AC-2 in default CI). Subprocess test invokes `signalforge init-demo --help` under `@pytest.mark.cli_subprocess` (covers `[project.scripts]` wiring per AC-5). +- **SD-4 — Ship locked `target/manifest.json`.** `_demo/target/manifest.json` ships the existing 13 KB dbt-v1.8-locked manifest seed so `signalforge generate --dry-run` works out of the box. Next-steps message references `dbt parse` for refresh; no extra friction on first run. +- **SD-5 — Public `signalforge.demo.copy_demo(...)`.** New public module `signalforge.demo` exposes `copy_demo(dest: Path, *, force: bool = False) -> None`. CLI calls into it. CLAUDE.md "Public API surface" gets a new bullet alongside `signalforge.cli.main(...)`. Library callers (notebooks, scripts) get a clean programmatic entry. +- **SD-6 — Tier 2 exit code for dest-exists-without-force.** `CliInitDemoDestExistsError(CliInputError)` → exit 2 (input validation; mirrors `ModelNotFoundError` precedent — operator supplied a path the CLI rejects pre-action). Lands in `_EXCEPTION_TO_EXIT_CODE`; 7th AST scan auto-gates. + +--- + +## Architecture Review + +### Security + +| # | Area | Rating | Finding | +|---|------|--------|---------| +| S-1 | Path/symlink defence on `` | concern | `init-demo` has no `project_dir` context (operator is *creating* one), so the project's `canonicalise_path(input, project_dir)` containment helper doesn't apply directly. Need a standalone resolve + symlink-cycle guard. Open question: refuse `` that resolves outside `Path.home()` / contains marker files (`.git/`, `.bashrc`), or trust the operator? | +| S-2 | `--force` blast radius | **BLOCKER** | The ticket says "`--force` allows overwrite of non-empty dest" but doesn't specify semantics. Three options: (a) `rmtree(dest) && copy` — atomic, simple, catastrophic if `dest=~`; (b) merge-overwrite individual files — polluting `~` is recoverable; (c) refuse per-file clobber even with `--force` — safest, may leave a partial tree. Must pick one. | +| S-3 | `importlib.resources` extraction | pass | `importlib.resources.files("signalforge") / "_demo"` is safe by construction — the API rejects `../` escapes in resource names. Source tree is in our repo + gated by parity test. Use `importlib.resources.as_file(...)` context manager for the temp-extract pattern. | +| S-4 | Symlinks inside the shipped `_demo/` tree | concern | Source tree has no symlinks today, but `shutil.copytree(..., symlinks=False)` plus a parity-test assertion "_demo/ contains no symlinks" codifies the policy and gates future drift. | +| S-5 | TOCTOU between `dest.exists()` check and `copytree` | pass | Theoretical only; matches project-wide stance (loader / profiles layer doesn't defend either). Document in handler docstring. | +| S-6 | Next-steps message naming `ANTHROPIC_API_KEY` | pass | The message names env-var names, not values. Same content the README already prints. No new disclosure surface. | +| S-7 | dbt `env_var('GOOGLE_CLOUD_PROJECT')` in shipped `profiles.yml` | pass | Lookup-time substitution; not a template-injection vector. dbt-native pattern. | + +### Packaging / installation + +| # | Area | Rating | Finding | +|---|------|--------|---------| +| P-1 | Will `_demo/` data files actually ship in the wheel? | **BLOCKER** | The plan's earlier assumption that `packages = ["src/signalforge"]` auto-ships non-`.py` files was **wrong**. Hatchling's default `packages` glob only picks up `.py`. Verified empirically: a current `python -m build --wheel` produces a 324 KB wheel with zero data files. Must add an explicit `include` directive: `[tool.hatch.build.targets.wheel] include = ["src/signalforge/_demo"]` (alongside the existing `packages` line). Verify post-fix with `unzip -l dist/*.whl \| grep _demo/`. | +| P-2 | `importlib.resources` discoverability without `__init__.py` in `_demo/` | pass | Python 3.11+ `Traversable` resolves subdirs without requiring `__init__.py`. `_demo/` stays a plain data dir; don't add a marker. | +| P-3 | Editable install (`pip install -e .`) | pass | Hatchling editable reads directly from `src/signalforge/` on disk; `_demo/` is discoverable immediately, `include` directive is not consulted in editable mode. CI is unaffected. | +| P-4 | CI gate for AC-2 ("demo fixture lives in the wheel") | **BLOCKER** | Manual `unzip -l` at release-time is brittle. AC-2 needs a CI gate. Two options: (i) maintainer-only `pytest -m wheel_smoke` that runs `hatch build` and asserts `_demo/` files in the artifact; (ii) one-shot subprocess test that asserts `importlib.resources.files("signalforge") / "_demo"` has the expected file count when run against the installed editable build. Option (ii) is cheaper and runs in default CI; option (i) is the only one that catches an `include` typo. | +| P-5 | Wheel size: 64 KB demo on top of 324 KB current (~20%) | concern | Acceptable for a CLI tool. Document in CHANGELOG. | +| P-6 | Will Hatchling pick up `_demo/.gitignore`? | concern | Hatchling's `include` glob behaviour on dotfiles is not guaranteed. Verify with a built-wheel inspection; if `.gitignore` is missing, rename to `gitignore.demo` in the shipped tree and have `copy_demo` rewrite to `.gitignore` at copy time. The shipped name matters less than the on-disk name post-copy. | +| P-7 | `target/manifest.json` shipped under `_demo/target/` | pass | Hatchling does not treat `target/` specially. Ships like any other data file under `include`. | +| P-8 | 5-surface parity for `--force` | pass | Precedent at `tests/cli/test_5_surface_parity_select.py` (issue #37). Copy the shape into a new `tests/cli/test_5_surface_parity_init_demo.py`. | + +### Testing strategy + +| # | Area | Rating | Finding | +|---|------|--------|---------| +| T-1 | In-process end-to-end test | pass | Existing pattern: `main(["init-demo", str(tmp_path)])` → assert exit 0, assert representative files appeared. Gates AC-2 in default CI (no marker). | +| T-2 | Subprocess `--help` smoke | pass | Extend `tests/cli/test_subprocess_smoke.py` with a second test invoking `signalforge init-demo --help` (same `@pytest.mark.cli_subprocess` marker). | +| T-3 | Parity test between `src/signalforge/_demo/` and `tests/fixtures/dbt_project_austin/` | pass | New `tests/test_demo_fixture_parity.py` (or similar) reads both trees, asserts byte-equality except for two named files (`profiles.yml`, `.gitignore`). Closes the SD-1 drift hole. | +| T-4 | Wheel-build CI gate (P-4 above) | **BLOCKER** | See P-4. Picking option (i) or (ii) is the refinement decision. | + +### Blockers to resolve in refinement + +- **B-1 (S-2):** `--force` semantics — pick atomic replace, merge-overwrite, or refuse-per-file-clobber. +- **B-2 (P-1):** Confirm `include = ["src/signalforge/_demo"]` directive lands in `pyproject.toml`. (Implementation-level; no design alternative.) +- **B-3 (P-4 / T-4):** CI gate for AC-2 — wheel_smoke marker that runs `hatch build` and inspects the artifact, OR in-process resource-existence check. + +### Concerns to address + +- **C-1 (S-1):** Path-resolve strategy for `` — standalone `Path(dest).resolve()` + symlink-cycle catch; optional marker-file refusal (`.git/`, `.bashrc`). +- **C-2 (S-4):** `shutil.copytree(symlinks=False)` + parity-test assertion that `_demo/` ships no symlinks. +- **C-3 (P-6):** Dotfile inclusion — verify `.gitignore` in built wheel; fall back to `gitignore.demo` rename + copy-time rewrite if Hatchling drops it. +- **C-4 (P-5):** Document the +64 KB wheel size in CHANGELOG / next-steps. + +--- + +## Refinement log (DECs) + +- **DEC-001 (B-1) — `--force` = atomic replace.** `copy_demo` with `force=True` runs `shutil.rmtree(dest)` (after confirming dest is not `/`, not `Path.home()`, not the cwd) then `shutil.copytree(_demo_src, dest)`. Matches `cp -rf` user expectations and produces a verbatim demo tree. The catastrophic-`~` footgun is partially mitigated by the symlink-cycle guard in DEC-004 plus the runtime sanity check that `dest.resolve() not in (Path("/"), Path.home(), Path.cwd())` — refuse with `CliInitDemoUnsafeDestError(CliInputError)` (tier 2). Without `--force`, `dest.exists() and any(dest.iterdir())` raises `CliInitDemoDestExistsError`. Empty existing dirs proceed without `--force`. +- **DEC-002 (B-2) — Hatch wheel `include` directive.** `pyproject.toml` `[tool.hatch.build.targets.wheel]` grows an explicit `include = ["src/signalforge/_demo"]` line alongside the existing `packages = ["src/signalforge"]`. Verified empirically that without this the data files are silently dropped. Pure implementation; no design alternative. +- **DEC-003 (B-3 / T-4) — `@pytest.mark.wheel_smoke` maintainer gate.** New marker registered in `pyproject.toml` `[tool.pytest.ini_options].markers` and added to default `addopts` exclusion (`-m 'not bigquery and not anthropic and not cli_subprocess and not e2e and not wheel_smoke'`). Test at `tests/test_wheel_packaging.py::test_wheel_includes_demo_fixture` shells out `python -m build --wheel --outdir `, opens the artifact via `zipfile.ZipFile`, asserts the canonical demo file set + `.gitignore` appear under `signalforge/_demo/`. Maintainers run `pytest -m wheel_smoke --no-cov` before declaring an init-demo PR ready. Mirrors `cli_subprocess` precedent (`testing-signal.md` § subprocess-gated smoke pattern). +- **DEC-004 (C-1) — Standalone path resolve, no containment boundary.** `` flows through `Path(dest).expanduser().resolve()` with `try/except RuntimeError` for symlink cycles → `CliPathError(cause=...)`. **No** marker-file refusal, **no** `Path.home()` containment. Matches `mkdir` / `cp` / `tar` UX (these don't refuse home-dir writes either). The `--force` blast-radius guard from DEC-001 catches the most catastrophic case (`signalforge init-demo --force ~`) explicitly. Note: `init-demo` does NOT route through `canonicalise_user_path(...)` — that helper enforces a `project_dir` containment boundary that doesn't apply here (operator is *creating* the project). Documented in handler docstring. +- **DEC-005 (C-2) — `shutil.copytree(symlinks=False)` + zero-symlinks parity assertion.** `copy_demo` follows symlinks (i.e. `symlinks=False` means "copy contents, not the link itself"). The parity test from SD-1 also asserts `_demo/` contains zero symlinks via `Path.rglob('*')` + `is_symlink()`. Codifies the "shipped demo is symlink-free" policy. Future drift breaks the parity test loudly. +- **DEC-006 (C-3) — Ship `.gitignore` as-is; wheel_smoke gates inclusion.** Keep the filename. The DEC-003 wheel_smoke test asserts `signalforge/_demo/.gitignore` appears in the built wheel. If Hatchling silently drops it under the directory glob, add an explicit `include = ["src/signalforge/_demo", "src/signalforge/_demo/.gitignore"]` extension — discovered at test-run time, not pre-emptively. Fallback (`gitignore.demo` rename + copy-time rewrite) is documented but not implemented in v0.1. +- **DEC-007 (C-4) — Document +64 KB wheel size in PR body.** No CHANGELOG file exists in v0.1; the PR description carries the size delta. Future v0.x ticket may introduce CHANGELOG; this DEC is a no-op until then. +- **DEC-008 (SD-1) — Parity test for `_demo/` ↔ `tests/fixtures/dbt_project_austin/`.** New `tests/test_demo_fixture_parity.py`: walks both trees, asserts byte-equality EXCEPT for two named files (`profiles.yml`, `.gitignore`). The two exceptions are documented in the test with a clear comment naming the rewrite (the shipped `profiles.yml` uses `env_var('GOOGLE_CLOUD_PROJECT')`; the test-fixture `profiles.yml` has the maintainer-only "DO NOT signalforge against this" header). Drift in any other file fails the test. +- **DEC-009 (SD-2) — Shipped `profiles.yml` uses `env_var('GOOGLE_CLOUD_PROJECT')`.** dbt-native lookup; operator with the env var set runs the demo with zero file edits. Aligns with `README.md:119-123` precedent. +- **DEC-010 (SD-3) — In-process e2e + subprocess `--help`.** `tests/cli/test_init_demo.py` covers in-process happy path + every error tier via `main([...])`. `tests/cli/test_subprocess_smoke.py` extends with one `signalforge init-demo --help` invocation under `@pytest.mark.cli_subprocess`. +- **DEC-011 (SD-4) — Ship locked `target/manifest.json`.** The existing 13 KB dbt-v1.8-locked manifest seed ships at `src/signalforge/_demo/target/manifest.json`. Next-steps message references `dbt parse` for refresh (no in-PR dbt invocation). +- **DEC-012 (SD-5) — Public `signalforge.demo.copy_demo(dest, *, force=False) -> None`.** New module `src/signalforge/demo.py`. CLI calls into it. `CLAUDE.md` "Public API surface" gets a new bullet alongside `signalforge.cli.main(...)`. Library callers (notebooks, scripts) get a programmatic entry point. The function raises `DemoDestExistsError`, `DemoDestUnsafeError`, `DemoFixtureMissingError` (lower-level typed errors); CLI wraps them into `Cli*Error` subclasses at the handler boundary so the CLI exit-code taxonomy stays homogeneous. +- **DEC-013 (SD-6) — Tier 2 exit code for dest-exists-without-force.** `CliInitDemoDestExistsError(CliInputError)` → exit 2. Mirrors `ModelNotFoundError` precedent. Lands in `_EXCEPTION_TO_EXIT_CODE`; 7th AST scan auto-gates. +- **DEC-014 — Next-steps message: plain text, stdout.** The post-copy message prints to stdout (operator channel), uses no ANSI colour codes, no markdown. Names `GOOGLE_CLOUD_PROJECT` and `ANTHROPIC_API_KEY` env vars + the exact commands to run (`signalforge lint`, `signalforge generate models/staging/stg_bikeshare_trips.sql --dry-run`). Single canonical wording lives in `signalforge.cli.init_demo._NEXT_STEPS_MESSAGE`. The message survives `--no-color` because it carries no colour codes. +- **DEC-015 — `regenerate.sh` updates BOTH trees in lockstep.** `tests/fixtures/dbt_project_austin/regenerate.sh` gains a final phase: after the live `dbt parse` lands the test-fixture manifest, the script `cp`s every file into `src/signalforge/_demo/`, then applies the demo-only rewrites (`profiles.yml` swap to `env_var(...)`; `.gitignore` slimmed). The parity test (DEC-008) gates future drift. Maintainers running the script keep both trees aligned by construction. + +### Decision dependencies + +- DEC-001 → DEC-013 (force semantics define the dest-exists error semantics). +- DEC-002 → DEC-003 (Hatch include directive is the surface DEC-003 verifies). +- DEC-008 → DEC-009 + DEC-015 (parity test gates the only two allowed deltas, regen script maintains both sides). +- DEC-012 → DEC-013 (CLI errors wrap the lower-level demo-module errors). + +--- + +## Detailed Breakdown (stories) + +Order follows SignalForge's typical layering: data → library → CLI surface → tests → docs → quality. Each story fits one Ralph context window. Every story's acceptance criteria includes the canonical validation command: + +```bash +ruff check . && ruff format --check . && pyright && pytest +``` + +### US-001 — Bootstrap `src/signalforge/_demo/` tree + parity test + +**Description:** Create the shipped demo tree as a copy of the test fixture with two named rewrites. Add a parity test that gates future drift. Amend the existing regen script to maintain both sides in lockstep. + +**Traces to:** SD-1, DEC-008, DEC-009, DEC-015. + +**Files:** +- NEW `src/signalforge/_demo/dbt_project.yml` — copy of `tests/fixtures/dbt_project_austin/dbt_project.yml` +- NEW `src/signalforge/_demo/signalforge.yml` — copy +- NEW `src/signalforge/_demo/profiles.yml` — copy with the project target rewritten to use `env_var('GOOGLE_CLOUD_PROJECT')` per DEC-009; strip the maintainer-only "DO NOT signalforge against this" header +- NEW `src/signalforge/_demo/.gitignore` — copy, slimmed of issue-#10 / DEC-021 internal references +- NEW `src/signalforge/_demo/models/staging/sources.yml` — copy +- NEW `src/signalforge/_demo/models/staging/stg_bikeshare_trips.sql` — copy +- NEW `src/signalforge/_demo/target/manifest.json` — copy +- NEW `tests/test_demo_fixture_parity.py` — walks both trees, asserts byte-equality EXCEPT for the two named files; asserts `_demo/` contains zero symlinks (`is_symlink()` over `rglob('*')`) +- MOD `tests/fixtures/dbt_project_austin/regenerate.sh` — final phase: copy each file into `src/signalforge/_demo/`, apply demo-only rewrites to `profiles.yml` and `.gitignore` + +**TDD:** +- `test_demo_fixture_parity_holds_byte_for_byte_except_documented_files` — fails on uncommanded drift +- `test_demo_fixture_contains_no_symlinks` — codifies DEC-005 +- `test_demo_profiles_yml_uses_env_var_macro` — pins DEC-009 specifically +- `test_test_fixture_profiles_yml_retains_maintainer_header` — confirms the rewrite is one-way + +**Done when:** Both fixture trees exist; parity test passes; the regen script runs end-to-end (manual maintainer check, not gated in CI). + +**Depends on:** none. + +--- + +### US-002 — Wire wheel packaging + wheel_smoke maintainer gate + +**Description:** Add the Hatch `include` directive so `_demo/` ships in the wheel. Register the `wheel_smoke` pytest marker. Write the maintainer-only test that builds the wheel and asserts demo files appear. + +**Traces to:** DEC-002, DEC-003, DEC-006. + +**Files:** +- MOD `pyproject.toml`: + - `[tool.hatch.build.targets.wheel]` → add `include = ["src/signalforge/_demo"]` alongside existing `packages` + - `[tool.pytest.ini_options].markers` → register `wheel_smoke` + - `[tool.pytest.ini_options].addopts` → extend the existing marker-exclusion expression to `... and not wheel_smoke` +- NEW `tests/test_wheel_packaging.py`: + - `@pytest.mark.wheel_smoke` + - subprocess `python -m build --wheel --outdir ` + - open the artifact via `zipfile.ZipFile`, list members + - assert the canonical demo file set appears under `signalforge/_demo/` (8 files including `.gitignore` per DEC-006) + - assert the test sets a 60-second timeout (mirrors `cli_subprocess` precedent) + +**TDD:** +- `test_wheel_includes_all_demo_files` (under `@pytest.mark.wheel_smoke`) +- `test_wheel_includes_demo_gitignore_dotfile` — DEC-006 specifically + +**Done when:** `pytest -m wheel_smoke --no-cov` passes; default `pytest` excludes the marker; `unzip -l dist/*.whl | grep _demo/` shows 8 files. + +**Depends on:** US-001. + +--- + +### US-003 — Public `signalforge.demo.copy_demo(...)` module + +**Description:** New public library entry point that locates the bundled `_demo/` tree via `importlib.resources`, validates the destination, and copies the tree. Owns the lower-level typed-error surface that the CLI wraps. + +**Traces to:** DEC-001, DEC-004, DEC-005, DEC-011, DEC-012. + +**Files:** +- NEW `src/signalforge/demo.py`: + - Module docstring explains the public contract. + - `copy_demo(dest: Path | str, *, force: bool = False) -> Path` — returns the resolved dest path. + - Path handling: `dest = Path(raw_dest).expanduser().resolve()` with `RuntimeError` catch → `DemoPathError(cause=...)`. + - Sanity gate: refuse `dest in (Path("/"), Path.home(), Path.cwd())` with `force=True` → `DemoDestUnsafeError`. + - Existence gate: `dest.exists() and any(dest.iterdir())` with `force=False` → `DemoDestExistsError`. Empty existing dirs proceed. + - Force branch: `force=True` with non-empty `dest` → `shutil.rmtree(dest)` then `shutil.copytree(...)`. + - Source lookup: `importlib.resources.files("signalforge").joinpath("_demo")`; wrap with `importlib.resources.as_file(...)` for the temp-extract pattern; raise `DemoFixtureMissingError` if the path doesn't traverse. + - `shutil.copytree(src, dest, symlinks=False)` per DEC-005. +- NEW `src/signalforge/_demo_errors.py` (or inline in `demo.py`): `DemoError(Exception)` base + `DemoPathError`, `DemoDestExistsError`, `DemoDestUnsafeError`, `DemoFixtureMissingError`. + +**TDD:** +- `test_copy_demo_to_empty_dir_copies_all_files` +- `test_copy_demo_to_nonexistent_dir_creates_and_copies` +- `test_copy_demo_to_nonempty_dir_without_force_raises_dest_exists_error` +- `test_copy_demo_to_nonempty_dir_with_force_replaces_atomically` +- `test_copy_demo_force_against_home_raises_dest_unsafe_error` +- `test_copy_demo_force_against_root_raises_dest_unsafe_error` +- `test_copy_demo_force_against_cwd_raises_dest_unsafe_error` +- `test_copy_demo_with_symlink_dest_resolves_target` — symlink dest is followed, not preserved +- `test_copy_demo_with_cyclic_symlink_dest_raises_demo_path_error` +- `test_copy_demo_returns_resolved_dest_path` +- `test_copy_demo_copies_target_manifest_json` — DEC-011 specifically +- `test_copy_demo_copies_dotfile_gitignore` — DEC-006 specifically +- `test_copy_demo_with_relative_dest_resolves_against_cwd` + +**Done when:** All TDD tests pass; `pyright src/signalforge/demo.py` clean; module is importable as `from signalforge.demo import copy_demo`. + +**Depends on:** US-001 (needs the source tree). + +--- + +### US-004 — CLI subcommand `init-demo` + typed CLI errors + +**Description:** Add the CLI subcommand following the `version.py` / `lint.py` template. Wrap `signalforge.demo`'s lower-level typed errors at the handler boundary into CLI-tier errors. Register all new errors in `_EXCEPTION_TO_EXIT_CODE`. Emit the next-steps message to stdout on success. + +**Traces to:** DEC-001, DEC-012, DEC-013, DEC-014. + +**Files:** +- NEW `src/signalforge/cli/init_demo.py`: + - `add_parser(subparsers)` registers positional `dest` (optional, default `./signalforge-demo/`) + `--force` + - `cmd_init_demo(args) -> int` wraps the pipeline in a single `try/except Exception`, calls `signalforge.demo.copy_demo(args.dest, force=args.force)`, prints `_NEXT_STEPS_MESSAGE.format(dest=...)` to stdout, returns 0 + - `_NEXT_STEPS_MESSAGE` constant — plain text, names `GOOGLE_CLOUD_PROJECT` + `ANTHROPIC_API_KEY` + the three commands per DEC-014 + - On any exception: `format_error_to_stderr(exc)`, `return map_exception_to_exit_code(exc)` +- MOD `src/signalforge/cli/__init__.py` — register the subcommand (mirror existing `lint`/`version` add_parser calls) +- MOD `src/signalforge/cli/errors.py`: + - `CliInitDemoDestExistsError(CliInputError)` — tier 2, wraps `DemoDestExistsError` + - `CliInitDemoDestUnsafeError(CliInputError)` — tier 2, wraps `DemoDestUnsafeError` + - `CliInitDemoFixtureMissingError(CliError)` — tier 1 (broken install), wraps `DemoFixtureMissingError` + - `CliInitDemoCopyError(CliError)` — tier 1, generic copy failure (`OSError` / `shutil` errors) + - Each carries a `default_remediation` +- MOD `src/signalforge/cli/_helpers.py` (`_EXCEPTION_TO_EXIT_CODE`) — register all four new error classes (`CliPathError` already mapped; reused for DEC-004 symlink-cycle case) + +**TDD:** +- `test_cmd_init_demo_to_fresh_path_returns_0_and_prints_next_steps` +- `test_cmd_init_demo_emits_next_steps_naming_env_vars` — DEC-014 specifically (asserts `"GOOGLE_CLOUD_PROJECT"` and `"ANTHROPIC_API_KEY"` in stdout) +- `test_cmd_init_demo_against_existing_nonempty_dir_returns_exit_2_with_remediation` +- `test_cmd_init_demo_force_against_existing_nonempty_dir_returns_0` +- `test_cmd_init_demo_force_against_home_returns_exit_2_dest_unsafe` +- `test_cmd_init_demo_never_leaks_traceback` — DEC-016 of `cli-layer.md` floor-of-every-CLI-test assertion +- `test_init_demo_help_lists_force_flag` — argparse help surface (precursor to 5-surface parity) +- `test_init_demo_help_lists_dest_positional` +- `test_init_demo_default_dest_is_signalforge_demo` — pins the default +- `test_cli_init_demo_dest_exists_error_in_exit_code_table` — 7th AST scan auto-covers, but pin explicit tier 2 assertion + +**Done when:** `signalforge init-demo --help` works; in-process e2e tests pass; all four new errors land in the exit-code mapping table; 7th AST scan passes. + +**Depends on:** US-003. + +--- + +### US-005 — 5-surface parity test for `init-demo` + `--force` + +**Description:** New parity test mirroring `tests/cli/test_5_surface_parity_select.py` (issue #37). Asserts the `init-demo` subcommand name and the `--force` flag appear with consistent semantics across argparse help, handler docstring, `docs/cli-ops.md`, and this plan's DEC list. + +**Traces to:** `cli-layer.md` 5-surface parity rule; DEC-001. + +**Files:** +- NEW `tests/cli/test_5_surface_parity_init_demo.py`: + - Asserts `"--force"` and the destination-positional name appear in: argparse help output (via `main(["init-demo", "--help"])` capturing stdout), `signalforge.cli.init_demo.__doc__` / handler docstring, `docs/cli-ops.md` § Subcommands, `plans/super/47-init-demo.md` DEC list + - Copy the shape from `test_5_surface_parity_select.py` verbatim where applicable + +**TDD:** the test itself is the artefact; no separate unit tests. + +**Done when:** `pytest tests/cli/test_5_surface_parity_init_demo.py` passes; the four surfaces are aligned in the same commit. + +**Depends on:** US-004, US-007 (needs `docs/cli-ops.md` updated first OR the test is written to fail until US-007 lands; recommend gating the test with `@pytest.mark.xfail(strict=True, reason="enabled by US-007")` and removing the marker in US-007). + +--- + +### US-006 — Subprocess smoke test for `init-demo --help` + +**Description:** Extend the existing subprocess smoke test file with one `signalforge init-demo --help` invocation. Catches `[project.scripts]` regressions specifically against the new subcommand. + +**Traces to:** DEC-010; ticket AC-5. + +**Files:** +- MOD `tests/cli/test_subprocess_smoke.py`: + - New test `test_signalforge_init_demo_help_via_subprocess` under `@pytest.mark.cli_subprocess` + - `subprocess.run(["signalforge", "init-demo", "--help"], capture_output=True, text=True, timeout=10)` + - assert returncode == 0; assert stdout contains `"init-demo"`; assert `"Traceback" not in result.stderr` + +**TDD:** the test itself. + +**Done when:** `pytest -m cli_subprocess --no-cov` passes (maintainer-only); the new test is excluded by default `addopts`. + +**Depends on:** US-004. + +--- + +### US-007 — Docs: README + `docs/cli-ops.md` + CLAUDE.md + +**Description:** Update README Quick Start to use `signalforge init-demo` instead of the `cp -r` snippet. Add `init-demo` entry to `docs/cli-ops.md` § Subcommands following the `lint` precedent. Extend CLAUDE.md "Public API surface" with the new `signalforge.demo.copy_demo` symbol. + +**Traces to:** AC-4; DEC-012; `cli-layer.md` 5-surface parity rule. + +**Files:** +- MOD `README.md`: + - Lines 47, 112-129, 161 — replace the `cp -r tests/fixtures/dbt_project_austin/...` snippet with `signalforge init-demo /tmp/sf-austin` (or similar); strip the inline heredoc for `profiles.yml` since the shipped one now uses `env_var('GOOGLE_CLOUD_PROJECT')` +- MOD `docs/cli-ops.md`: + - Line 57: bump "three subcommands" → "four subcommands" + - § Subcommands: new `### \`signalforge init-demo []\`` entry between `generate` and `lint` (or after `version` — pick by alphabetical or feature-grouping convention; check existing order) + - Document: positional ``, `--force` flag, error tiers, the next-steps message +- MOD `CLAUDE.md`: + - Public API surface bullet for `signalforge.cli.main(...)` — extend to mention `signalforge.demo.copy_demo(dest: Path | str, *, force: bool = False) -> Path` and the `DemoError` hierarchy (`DemoError`, `DemoPathError`, `DemoDestExistsError`, `DemoDestUnsafeError`, `DemoFixtureMissingError`) + - Repository-status block: add `#47 (init-demo)` entry summarising the ticket + +**TDD:** none (docs); the 5-surface parity test (US-005) is the implicit gate. + +**Done when:** the three doc surfaces describe `init-demo` consistently; the README Quick Start reads cleanly for a PyPI user who has never cloned the repo. + +**Depends on:** US-004. + +--- + +### US-008 — Quality Gate + +**Description:** Run the code-reviewer agent four times across the full changeset, fixing all real bugs found each pass. Run CodeRabbit review if available. Project validation (`ruff check . && ruff format --check . && pyright && pytest`) must pass after all fixes. Additionally run the gated marker suites once locally: `pytest -m wheel_smoke --no-cov`, `pytest -m cli_subprocess --no-cov`. + +**Traces to:** All DECs (verification pass). + +**Done when:** Four review passes complete; no real bugs remaining; all gated suites pass. + +**Depends on:** US-001, US-002, US-003, US-004, US-005, US-006, US-007 (everything). + +--- + +### US-009 — Patterns & Memory + +**Description:** Update `.claude/rules/` with new patterns learned from this ticket and refresh CLAUDE.md. + +**Traces to:** Repo convention maintenance. + +**Likely additions:** +- `.claude/rules/python-build.md` — new section on shipping non-`.py` package-data via Hatch's `include` directive (DEC-002), and the `wheel_smoke` maintainer-gate pattern (DEC-003). Document the empirical-verification step (`python -m build --wheel && unzip -l ...`). +- `.claude/rules/cli-layer.md` — extend the subcommand layout section with `init-demo` as a fourth precedent; note the new `signalforge.demo` library-surface pattern (CLI calls into a public lib module with its own typed error hierarchy, wrapped at the handler boundary). +- `.claude/rules/testing-signal.md` — extend the gated-marker section with `wheel_smoke` as a fourth marker (alongside `bigquery`, `anthropic`, `cli_subprocess`, `e2e`). +- `CLAUDE.md` — repo-status block already updated in US-007; verify nothing else needs sync. + +**Done when:** Rules reflect the new patterns; CLAUDE.md is consistent. + +**Depends on:** US-008. + +--- + +## Rules compliance gate + +Validated each story against the 12 rules-constraints from Phase 1: + +| Rule | Story coverage | +|------|----------------| +| cli-layer.md DEC-009 flat layout | US-004 | +| cli-layer.md DEC-008/024 four-tier exits + 7th AST scan | US-004 | +| cli-layer.md DEC-016 no traceback | US-004 (TDD includes traceback assertion) | +| cli-layer.md DEC-017 stderr shape | US-004 (typed errors carry remediation) | +| cli-layer.md DEC-019 logger grep gate | US-003, US-004 (no `_LOGGER` calls; stdout prints) | +| cli-layer.md DEC-027 path canonicalisation | US-003 (standalone resolve per DEC-004) | +| cli-layer.md 5-surface parity | US-005 (dedicated test) | +| python-build.md DEC-011 wheel packaging | US-002 | +| testing-signal.md no `assert True` | All test stories (specific assertions) | +| testing-signal.md src layout | already in place; no change | +| testing-signal.md subprocess-gated smoke | US-006 | +| testing-signal.md coverage gate | US-002 adds wheel_smoke exclusion to `addopts`; default `pytest` still passes coverage floor | + +--- + +## Quality-gate addendum (US-008, 4 review passes) + +Four code-review passes refined the implementation. Original DECs above are preserved verbatim as the ADR-style record; pass-by-pass adjustments below. + +- **Pass 1 — DEC-012 refactored: `signalforge.demo` is a subpackage, not a flat module.** The original DEC specified `src/signalforge/demo.py`. Pass 1 promoted it to `src/signalforge/demo/{__init__.py, errors.py}` so the existing `src/signalforge/*/errors.py` AST-scan glob auto-covers the demo layer's typed errors. Without the refactor, a v0.2 contributor adding a `Demo*Error` subclass would not get a test failure if they forgot to wire the CLI wrapper. `DemoError` (abstract base) joins `_EXCEPTION_MAPPING_EXCLUDED_BASES`; the four concrete subclasses (`DemoPathError`, `DemoDestExistsError`, `DemoDestUnsafeError`, `DemoFixtureMissingError`) land in `_EXCEPTION_TO_EXIT_CODE` at the same tiers as their CLI wrappers (defence-in-depth — a future raise outside the CLI handler boundary still gets a sensible exit code via the MRO walk). Scan-7 sanity test bumps from 9 to 10 expected paths. + +- **Pass 1 — DEC-014 polished: dest-exists remediation names the blast-radius guard.** `CliInitDemoDestExistsError`'s default remediation now reads "Remove the existing directory or run 'signalforge init-demo --force' to replace it (refuses '/', $HOME, or the current working directory as a blast-radius guard)." so an operator following the suggestion against `~` understands the `--force` outcome before they type it. + +- **Pass 1 — `_NEXT_STEPS_MESSAGE` shell-quotes the cd line.** Destinations containing spaces (`/Users/Wes Duenow/...`) now produce a copy-pasteable `cd ''` line via `shlex.quote` instead of an ambiguous bare path. + +- **Pass 2 — DEC-001 extended: dest-is-file shape gate.** The original force-semantics decision (atomic `rmtree+copytree`) didn't address `dest=` (regular file, not directory). Pass 2 adds a shape pre-check before the existence-gate: a non-directory dest raises `DemoDestExistsError` ("exists but is not a directory") with `force=False`, or is `unlink()`-ed and replaced with the demo tree under `force=True`. Without the pre-check, `iterdir()` raised raw `NotADirectoryError` mid-flow which the CLI wrapped as a misleading `CliInitDemoCopyError` (tier 1, "failed to copy demo tree") — the operator's mental model expected tier 2 + a "you supplied bad input" message. + +- **Pass 3 — DEC-009 graduated into the warehouse layer.** The original DEC said the shipped `profiles.yml` uses `env_var('GOOGLE_CLOUD_PROJECT')` so the operator runs the demo with zero file edits. The original implementation shipped the literal jinja string in the profile, but `signalforge.warehouse.profiles` only did `yaml.safe_load` — no jinja rendering — so the literal jinja text was sent to BigQuery as the project ID, breaking the "zero file edits" happy path. Pass 3 added a minimal dbt-compatible `env_var('NAME')` / `env_var('NAME', 'default')` substitution to `signalforge.warehouse.profiles._load_profiles_yaml` (pre-yaml-parse so YAML quoting is preserved). New typed error `ProfileEnvVarUnsetError(ProfileNotFoundError)` — tier 1. Verified end-to-end: `signalforge init-demo` + `load_profile` resolves `GOOGLE_CLOUD_PROJECT` correctly; unset env surfaces a clear typed error pointing at the missing var instead of a downstream BigQuery rejection. Also extended `tests/llm/test_logger_grep_gate.py::_SCAN_SUBPACKAGES` to cover `signalforge.demo` (pass-1 refactor promoted the layer; the gate needed to follow) and updated `.claude/rules/cli-layer.md` stale "nine modules" / "nine per-stage abstract bases" prose to "ten" (with `LLMHelperError` explicitly noted as deliberately NOT excluded). + +- **Pass 4 — Re-pinned `__all__` count.** `ProfileEnvVarUnsetError` was correctly registered in the exit-code mapping and re-exported from `signalforge.warehouse`, but missing from `signalforge.warehouse.errors.__all__`. The warehouse errors module's self-tests iterate `__all__` to assert alphabetical sort + count + every class is smoke-constructible + every class carries a non-empty `default_remediation`. Without the `__all__` entry, the new class bypassed all four invariants. Pass 4 added the entry in alphabetical position, bumped the count constant from 19 to 20, and registered the constructor kwargs in `_CONSTRUCT_KWARGS`. + +All four passes preserved the load-bearing acceptance criteria (ticket AC-1..AC-5) without introducing any new DEC. The four error-class additions (`CliInitDemoDestExistsError`, `CliInitDemoDestUnsafeError`, `CliInitDemoFixtureMissingError`, `CliInitDemoCopyError`) plus the pass-3 addition (`ProfileEnvVarUnsetError`) are the complete typed-error surface this ticket ships. + +Final validation at the close of QG: pytest 1730 passed (default suite) + 4 passed (gated `wheel_smoke` + `cli_subprocess`), ruff/pyright clean, end-to-end happy path (`init-demo` → `cd` → `lint` → `generate --dry-run`) verified manually. Six pre-existing WSL2 symlink-loop test failures are environmental; confirmed on `dev` HEAD; not regressions. + +--- + diff --git a/pyproject.toml b/pyproject.toml index 938df0e2..a2f4fa86 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -35,6 +35,13 @@ path = "src/signalforge/__init__.py" [tool.hatch.build.targets.wheel] packages = ["src/signalforge"] +# `include` is defence-in-depth for the demo tree under `src/signalforge/_demo/` +# — Hatchling's default `packages` glob picks `.py` reliably but its behaviour +# on non-`.py` data files (and dotfiles like `.gitignore` per DEC-006 of +# `plans/super/47-init-demo.md`) is not contractually guaranteed across releases. +# The wheel_smoke marker (`tests/test_wheel_packaging.py`) gates the demo file +# set in the built artifact; this directive is the production-side guarantee. +include = ["src/signalforge/_demo"] [tool.ruff] line-length = 100 @@ -57,7 +64,7 @@ testpaths = ["tests"] # `--import-mode=importlib` lets us share basenames across test dirs (e.g. # tests/manifest/test_errors.py and tests/warehouse/test_errors.py) without # adding tests/__init__.py — keeps `testing-signal.md`'s no-init rule intact. -addopts = "-ra --strict-markers --import-mode=importlib -m 'not bigquery and not anthropic and not cli_subprocess and not e2e' --cov=signalforge --cov-report=xml --cov-report=term-missing --cov-fail-under=80" +addopts = "-ra --strict-markers --import-mode=importlib -m 'not bigquery and not anthropic and not cli_subprocess and not e2e and not wheel_smoke' --cov=signalforge --cov-report=xml --cov-report=term-missing --cov-fail-under=80" minversion = "7.0" strict_markers = true markers = [ @@ -71,4 +78,5 @@ markers = [ "anthropic: real-API smoke test (requires ANTHROPIC_API_KEY; excluded from default CI)", "cli_subprocess: belt-and-braces subprocess-driven CLI smoke (skipped by default; run with -m cli_subprocess)", "e2e: end-to-end smoke test against real Anthropic + real BigQuery (gated by SF_RUN_BQ=1, ANTHROPIC_API_KEY, GOOGLE_CLOUD_PROJECT; skipped by default)", + "wheel_smoke: maintainer-only; builds the wheel via `python -m build --wheel` and inspects the artifact for the demo file set (run with --no-cov)", ] diff --git a/src/signalforge/_demo/.gitignore b/src/signalforge/_demo/.gitignore new file mode 100644 index 00000000..b1d46e19 --- /dev/null +++ b/src/signalforge/_demo/.gitignore @@ -0,0 +1,3 @@ +# SignalForge writes per-run audit logs and sidecar artefacts under .signalforge/. +# These are reproducible from a re-run; no value in committing them. +.signalforge/ diff --git a/src/signalforge/_demo/dbt_project.yml b/src/signalforge/_demo/dbt_project.yml new file mode 100644 index 00000000..5af0880a --- /dev/null +++ b/src/signalforge/_demo/dbt_project.yml @@ -0,0 +1,26 @@ +# Fixture project for issue #10 e2e BigQuery smoke test. +# Targets dbt-bigquery 1.8.x — no v1.9-specific keys (e.g. no `unit_tests:` block). +# Profile `austin` is defined in the sibling profiles.yml; the regen script +# (US-002) sets DBT_PROFILES_DIR to this directory before invoking `dbt parse`. +name: signalforge_test_austin +version: 1.0.0 +config-version: 2 + +profile: austin + +model-paths: ["models"] +analysis-paths: ["analyses"] +test-paths: ["tests"] +seed-paths: ["seeds"] +macro-paths: ["macros"] +snapshot-paths: ["snapshots"] + +target-path: target +clean-targets: + - target + - dbt_packages + +models: + signalforge_test_austin: + staging: + +materialized: view diff --git a/src/signalforge/_demo/models/staging/sources.yml b/src/signalforge/_demo/models/staging/sources.yml new file mode 100644 index 00000000..69e30ded --- /dev/null +++ b/src/signalforge/_demo/models/staging/sources.yml @@ -0,0 +1,31 @@ +version: 2 + +sources: + - name: austin_bikeshare + description: "Austin Bikeshare public dataset (bigquery-public-data)." + database: bigquery-public-data + schema: austin_bikeshare + tables: + - name: bikeshare_trips + description: "One row per bikeshare trip in Austin, TX." + columns: + - name: trip_id + description: "Unique identifier for each trip." + - name: subscriber_type + description: "Membership type of the rider (e.g. local monthly, walk-up)." + - name: bike_id + description: "Identifier of the bike used for the trip." + - name: bike_type + description: "Type/model of the bike." + - name: start_time + description: "Trip start timestamp (UTC)." + - name: start_station_id + description: "Identifier of the trip's origin station." + - name: start_station_name + description: "Human-readable name of the origin station." + - name: end_station_id + description: "Identifier of the trip's destination station." + - name: end_station_name + description: "Human-readable name of the destination station." + - name: duration_minutes + description: "Trip duration in minutes." diff --git a/src/signalforge/_demo/models/staging/stg_bikeshare_trips.sql b/src/signalforge/_demo/models/staging/stg_bikeshare_trips.sql new file mode 100644 index 00000000..ee4b41ee --- /dev/null +++ b/src/signalforge/_demo/models/staging/stg_bikeshare_trips.sql @@ -0,0 +1,16 @@ +-- Source-as-model: the manifest aliases this model to `bikeshare_trips` +-- so its relation_name resolves directly to the public source table. +-- SignalForge runs queries against the materialised relation; without +-- `dbt run` against a writable billing project this keeps the smoke +-- test a single command (issue #10 Path A). The `always-passes` AC then +-- relies on natural NOT NULL columns (`trip_id`, `start_time`) rather +-- than engineered literal/COALESCE columns. +SELECT + trip_id, + subscriber_type, + bike_id, + start_time, + start_station_id, + end_station_id, + duration_minutes +FROM {{ source('austin_bikeshare', 'bikeshare_trips') }} diff --git a/src/signalforge/_demo/profiles.yml b/src/signalforge/_demo/profiles.yml new file mode 100644 index 00000000..5cdac61f --- /dev/null +++ b/src/signalforge/_demo/profiles.yml @@ -0,0 +1,23 @@ +# Demo dbt profile shipped by `signalforge init-demo`. +# +# `method: oauth` falls through to Application Default Credentials — run +# `gcloud auth application-default login` once before invoking +# `signalforge generate` against this project. +# +# `project: "{{ env_var('GOOGLE_CLOUD_PROJECT') }}"` resolves at runtime from +# your billing project; export it before running the demo: +# +# export GOOGLE_CLOUD_PROJECT= +# +# The Austin bikeshare data is served from the public +# `bigquery-public-data.austin_bikeshare` dataset; your `GOOGLE_CLOUD_PROJECT` +# is the BILLING project the BigQuery SDK uses to issue the read. +austin: + target: dev + outputs: + dev: + type: bigquery + method: oauth + project: "{{ env_var('GOOGLE_CLOUD_PROJECT') }}" + dataset: austin_bikeshare + location: US diff --git a/src/signalforge/_demo/signalforge.yml b/src/signalforge/_demo/signalforge.yml new file mode 100644 index 00000000..5f260842 --- /dev/null +++ b/src/signalforge/_demo/signalforge.yml @@ -0,0 +1,14 @@ +# Issue #10 e2e fixture config. Locked values are load-bearing for the smoke test +# — see plans/super/10-e2e-bigquery-smoke.md DEC-005..DEC-018. +# Pinned to match DraftConfig.model default; bump in lockstep when Sonnet 4.6 sunsets. +llm: + model: claude-sonnet-4-6 +safety: + mode: aggregate-only +prune: + sample_strategy: materialised # v0.2 default; exercises BQ session-state. +grade: + min_pass_rate: 0.95 + min_mean_score: 0.95 + fail_on_below_threshold: false + total_budget_seconds: 600 # default 300 is tight at p99 latency × ~12 calls. diff --git a/src/signalforge/_demo/target/manifest.json b/src/signalforge/_demo/target/manifest.json new file mode 100644 index 00000000..51aad295 --- /dev/null +++ b/src/signalforge/_demo/target/manifest.json @@ -0,0 +1,340 @@ +{ + "metadata": { + "dbt_schema_version": "https://schemas.getdbt.com/dbt/manifest/v12.json", + "dbt_version": "1.8.9", + "generated_at": null, + "invocation_id": null, + "env": {}, + "project_name": "signalforge_test_austin", + "project_id": "signalforgeaustin0000000000000000", + "user_id": null, + "send_anonymous_usage_stats": null, + "adapter_type": null + }, + "nodes": { + "model.signalforge_test_austin.stg_bikeshare_trips": { + "database": "bigquery-public-data", + "schema": "austin_bikeshare", + "name": "stg_bikeshare_trips", + "resource_type": "model", + "package_name": "signalforge_test_austin", + "path": "staging/stg_bikeshare_trips.sql", + "original_file_path": "models/staging/stg_bikeshare_trips.sql", + "unique_id": "model.signalforge_test_austin.stg_bikeshare_trips", + "fqn": [ + "signalforge_test_austin", + "staging", + "stg_bikeshare_trips" + ], + "alias": "bikeshare_trips", + "checksum": { + "name": "sha256", + "checksum": "0000000000000000000000000000000000000000000000000000000000000000" + }, + "config": { + "enabled": true, + "alias": null, + "schema": null, + "database": null, + "tags": [], + "meta": {}, + "group": null, + "materialized": "view", + "incremental_strategy": null, + "persist_docs": {}, + "post-hook": [], + "pre-hook": [], + "quoting": {}, + "column_types": {}, + "full_refresh": null, + "unique_key": null, + "on_schema_change": "ignore", + "on_configuration_change": "apply", + "grants": {}, + "packages": [], + "docs": { + "show": true, + "node_color": null + }, + "contract": { + "enforced": false, + "alias_types": true + }, + "access": "protected" + }, + "tags": [], + "description": "Source-as-model passthrough for the Austin Bikeshare public dataset's `bikeshare_trips` table. Each row represents one bike-share trip in Austin, TX, captured by the city's bikeshare programme. The model exposes a curated subset of seven columns (trip identifier, subscriber type, bike identifier, start timestamp, start/end station identifiers, duration in minutes) for the LLM drafter to reason about; the full source schema (10 columns including bike type and station names) is documented in `models/staging/sources.yml`. The model's `alias` is overridden to `bikeshare_trips` so its `relation_name` resolves directly to `bigquery-public-data.austin_bikeshare.bikeshare_trips`, sidestepping the need for a `dbt run` materialisation step in the v0.1 smoke-test path. The dataset is authoritative for Austin bikeshare ridership and updated regularly by the city; downstream models can rely on `trip_id` being unique per ride and on `start_time` ordering trips chronologically.", + "columns": { + "trip_id": { + "name": "trip_id", + "description": "Unique identifier assigned to each bikeshare trip by the city's bikeshare system. Acts as the natural primary key for this table; no two rows in the source share a `trip_id`. Stored as a STRING because the underlying identifier is alphanumeric in some city installations even though it looks numeric in this dataset.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "subscriber_type": { + "name": "subscriber_type", + "description": "Membership classification of the rider who took the trip — typical values include `local monthly`, `walk-up`, `single trip`, `student membership`, `weekender`, etc. Useful for segmenting demand by user category. Free-form STRING (not enumerated in the source schema), so downstream consumers should expect long-tail values.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "bike_id": { + "name": "bike_id", + "description": "Identifier of the physical bike used for this trip. Maps a single bike across many trips so utilisation per bike can be computed. Note the underscore in `bike_id` (the source column is `bike_id`, NOT `bikeid`); the underscore matters for joins and for any downstream model that references this column by name.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "start_time": { + "name": "start_time", + "description": "Timestamp marking when the trip began (the rider docked-out the bike). Stored as TIMESTAMP in UTC. This is the primary time dimension for the model and is reliably non-null in the source data — every trip has a recorded start time even when other fields are sparse.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "start_station_id": { + "name": "start_station_id", + "description": "Numeric identifier of the bikeshare station where the trip began. Joins to `austin_bikeshare.bikeshare_stations.station_id` to resolve station name, latitude, longitude, council district. Some legacy trips have NULL here when the station was deleted from the registry but the trip record was preserved.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "end_station_id": { + "name": "end_station_id", + "description": "Numeric identifier of the station where the trip ended (rider docked-in the bike). Same join semantics as `start_station_id`. NULL is possible for trips that ended outside the station network or whose end-station record was later deleted from the registry.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "duration_minutes": { + "name": "duration_minutes", + "description": "Trip length in whole minutes, computed by the source system as `end_time - start_time`. INTEGER. Most trips fall under 60 minutes; the long tail above 1440 (24 hours) typically indicates abandoned bikes or system glitches rather than real ridership. Downstream analytics often filter to `duration_minutes BETWEEN 1 AND 240` to focus on legitimate trips.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + } + }, + "meta": {}, + "group": null, + "docs": { + "show": true, + "node_color": null + }, + "patch_path": null, + "build_path": null, + "unrendered_config": { + "materialized": "view" + }, + "created_at": 0, + "relation_name": "`bigquery-public-data`.`austin_bikeshare`.`bikeshare_trips`", + "raw_code": "-- Source-as-model: alias overridden to `bikeshare_trips` so the model's\n-- relation_name resolves directly to the public source table. SignalForge\n-- runs against materialised tables; without `dbt run` against a writable\n-- billing project this is the v0.1 path that keeps the smoke test a single\n-- command (issue #10 Path A).\nSELECT\n trip_id,\n subscriber_type,\n bike_id,\n start_time,\n start_station_id,\n end_station_id,\n duration_minutes\nFROM {{ source('austin_bikeshare', 'bikeshare_trips') }}\n", + "language": "sql", + "refs": [], + "sources": [ + [ + "austin_bikeshare", + "bikeshare_trips" + ] + ], + "metrics": [], + "depends_on": { + "macros": [], + "nodes": [ + "source.signalforge_test_austin.austin_bikeshare.bikeshare_trips" + ] + }, + "compiled_path": null, + "contract": { + "enforced": false, + "alias_types": true, + "checksum": null + }, + "access": "protected", + "constraints": [], + "version": null, + "latest_version": null, + "deprecation_date": null + } + }, + "sources": { + "source.signalforge_test_austin.austin_bikeshare.bikeshare_trips": { + "database": "bigquery-public-data", + "schema": "austin_bikeshare", + "name": "bikeshare_trips", + "resource_type": "source", + "package_name": "signalforge_test_austin", + "path": "models/staging/sources.yml", + "original_file_path": "models/staging/sources.yml", + "unique_id": "source.signalforge_test_austin.austin_bikeshare.bikeshare_trips", + "fqn": [ + "signalforge_test_austin", + "austin_bikeshare", + "bikeshare_trips" + ], + "source_name": "austin_bikeshare", + "source_description": "Austin Bikeshare public dataset (bigquery-public-data).", + "loader": "", + "identifier": "bikeshare_trips", + "quoting": { + "database": null, + "schema": null, + "identifier": null, + "column": null + }, + "loaded_at_field": null, + "freshness": { + "warn_after": { + "count": null, + "period": null + }, + "error_after": { + "count": null, + "period": null + }, + "filter": null + }, + "external": null, + "description": "One row per bikeshare trip in Austin, TX.", + "columns": { + "trip_id": { + "name": "trip_id", + "description": "Unique identifier for each trip.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "subscriber_type": { + "name": "subscriber_type", + "description": "Membership type of the rider (e.g. local monthly, walk-up).", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "bike_id": { + "name": "bike_id", + "description": "Identifier of the bike used for the trip.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "bike_type": { + "name": "bike_type", + "description": "Type/model of the bike.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "start_time": { + "name": "start_time", + "description": "Trip start timestamp (UTC).", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "start_station_id": { + "name": "start_station_id", + "description": "Identifier of the trip's origin station.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "start_station_name": { + "name": "start_station_name", + "description": "Human-readable name of the origin station.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "end_station_id": { + "name": "end_station_id", + "description": "Identifier of the trip's destination station.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "end_station_name": { + "name": "end_station_name", + "description": "Human-readable name of the destination station.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + }, + "duration_minutes": { + "name": "duration_minutes", + "description": "Trip duration in minutes.", + "meta": {}, + "data_type": null, + "constraints": [], + "quote": null, + "tags": [] + } + }, + "meta": {}, + "source_meta": {}, + "tags": [], + "config": { + "enabled": true + }, + "patch_path": null, + "unrendered_config": {}, + "relation_name": "`bigquery-public-data`.`austin_bikeshare`.`bikeshare_trips`", + "created_at": 0 + } + }, + "macros": {}, + "docs": {}, + "exposures": {}, + "metrics": {}, + "groups": {}, + "selectors": {}, + "disabled": {}, + "parent_map": { + "model.signalforge_test_austin.stg_bikeshare_trips": [ + "source.signalforge_test_austin.austin_bikeshare.bikeshare_trips" + ], + "source.signalforge_test_austin.austin_bikeshare.bikeshare_trips": [] + }, + "child_map": { + "model.signalforge_test_austin.stg_bikeshare_trips": [], + "source.signalforge_test_austin.austin_bikeshare.bikeshare_trips": [ + "model.signalforge_test_austin.stg_bikeshare_trips" + ] + }, + "group_map": {}, + "saved_queries": {}, + "semantic_models": {}, + "unit_tests": {} +} diff --git a/src/signalforge/cli/__init__.py b/src/signalforge/cli/__init__.py index d664c2fc..219f112b 100644 --- a/src/signalforge/cli/__init__.py +++ b/src/signalforge/cli/__init__.py @@ -19,6 +19,7 @@ import signalforge from signalforge.cli import generate as generate_cmd +from signalforge.cli import init_demo as init_demo_cmd from signalforge.cli import lint as lint_cmd from signalforge.cli import version as version_cmd from signalforge.cli._helpers import ( @@ -79,6 +80,7 @@ def _build_parser() -> argparse.ArgumentParser: version_cmd.add_parser(subparsers) lint_cmd.add_parser(subparsers) generate_cmd.add_parser(subparsers) + init_demo_cmd.add_parser(subparsers) return parser diff --git a/src/signalforge/cli/_helpers.py b/src/signalforge/cli/_helpers.py index d3631aa5..e9401358 100644 --- a/src/signalforge/cli/_helpers.py +++ b/src/signalforge/cli/_helpers.py @@ -41,11 +41,21 @@ from signalforge._common.path_safety import PathContainmentError, canonicalise_path from signalforge.cli.errors import ( CliError, + CliInitDemoCopyError, + CliInitDemoDestExistsError, + CliInitDemoDestUnsafeError, + CliInitDemoFixtureMissingError, CliInputError, CliPathError, CliSelectorNoMatchError, CliSelectorParseError, ) +from signalforge.demo import ( + DemoDestExistsError, + DemoDestUnsafeError, + DemoFixtureMissingError, + DemoPathError, +) # --- per-stage public-surface imports for the exit-code table --------------- # Importing from each ``signalforge.`` package mirrors how the rest of @@ -135,6 +145,7 @@ ManifestSchemaNotFoundError, MaterialisationFailedError, MaterialisationNotSupportedError, + ProfileEnvVarUnsetError, ProfileNotFoundError, ProfileTargetNotFoundError, QuerySyntaxError, @@ -202,6 +213,7 @@ # Warehouse profile / connection-shape config (auth lives in tier 3 # because it's an external-dep state rather than a config-shape issue). ProfileNotFoundError: 1, + ProfileEnvVarUnsetError: 1, ProfileTargetNotFoundError: 1, UnsupportedProfileTypeError: 1, UnsupportedAuthMethodError: 1, @@ -222,6 +234,24 @@ # CLI-layer load-shape errors. CliError: 1, CliPathError: 1, + # init-demo broken-install / filesystem-failure wrappers (issue #47 / + # DEC-012 of plans/super/47-init-demo.md). Tier 1 because both fire + # before any user-content work has happened and represent state we + # couldn't get into a coherent shape (missing wheel resource, generic + # OSError during the copytree / rmtree). + CliInitDemoFixtureMissingError: 1, + CliInitDemoCopyError: 1, + # Lower-level signalforge.demo typed errors (issue #47). The CLI + # wraps these into the Cli* wrappers above, so under normal CLI + # operation they never reach this mapping directly. They land in + # the table anyway as defence-in-depth: the 7th AST scan + # (tests/test_audit_completeness.py) gates every concrete *Error + # under src/signalforge/*/errors.py; mapping them here means a + # v0.2 contributor who adds a new Demo*Error and forgets to wire + # the CLI wrapper still gets a sensible exit code via the MRO + # walk in :func:`map_exception_to_exit_code`. + DemoPathError: 1, + DemoFixtureMissingError: 1, # ---- Tier 2: input ---------------------------------------------------- # Manifest selection (the operator picked a model that doesn't exist or # is disabled — caller's fault, not load). @@ -274,6 +304,17 @@ # zero-match mirrors ``ModelNotFoundError``'s tier. CliSelectorParseError: 2, CliSelectorNoMatchError: 2, + # init-demo input-validation wrappers (issue #47 / DEC-013 of + # plans/super/47-init-demo.md). Tier 2 because both fire on + # operator-supplied dest values that conflict with project state — + # mirrors the precedent set by ModelNotFoundError (tier 2 for "the + # operator named something the project rejects"). + CliInitDemoDestExistsError: 2, + CliInitDemoDestUnsafeError: 2, + # Lower-level demo-layer counterparts — see the tier-1 demo block + # above for the defence-in-depth rationale. + DemoDestExistsError: 2, + DemoDestUnsafeError: 2, # ---- Tier 3: API / external dep --------------------------------------- # LLM connectivity / quota / SDK issues. LLMError: 3, diff --git a/src/signalforge/cli/errors.py b/src/signalforge/cli/errors.py index 0dd258d4..8395ad1c 100644 --- a/src/signalforge/cli/errors.py +++ b/src/signalforge/cli/errors.py @@ -158,3 +158,193 @@ def __init__(self, *, expr: str, remediation: str | None = None) -> None: ), ) self.expr = expr + + +# --------------------------------------------------------------------------- +# init-demo wrappers (issue #47 — US-004, DEC-012 / DEC-013) +# --------------------------------------------------------------------------- +# +# The CLI subcommand ``signalforge init-demo`` calls into the public +# :func:`signalforge.demo.copy_demo` helper. The helper raises four typed +# :class:`signalforge.demo.DemoError` subclasses; the CLI handler wraps each +# at the boundary into one of the four ``CliInitDemo*Error`` classes below so +# the four-tier exit-code taxonomy stays homogeneous (DEC-012). DEC-013 locks +# the dest-exists / dest-unsafe cases at tier 2 (input-validation — the +# operator chose a destination that already has content or that would clobber +# a system / user directory); the fixture-missing / generic copy-failure cases +# land at tier 1 (load — broken install or filesystem state we couldn't get +# into a usable shape before doing work). +# +# Each class carries a ``default_remediation`` so the layer-base ``__str__`` +# renders the canonical ``ERROR: \n ↳ Remediation: `` shape +# without subclasses having to redefine rendering. +# +# The :class:`signalforge.demo.DemoPathError` case (symlink-cycle resolve +# failure) re-uses the existing :class:`CliPathError` rather than getting its +# own wrapper — every CLI-originated path-safety failure produces one error +# type, regardless of which underlying helper detected the cycle. + + +_CLI_INIT_DEMO_DEST_EXISTS_DEFAULT_REMEDIATION: str = ( + "Remove the existing directory or run 'signalforge init-demo --force' to replace it " + "(refuses '/', $HOME, or the current working directory as a blast-radius guard)." +) + +_CLI_INIT_DEMO_DEST_UNSAFE_DEFAULT_REMEDIATION: str = ( + "Refusing to clobber a system or home directory. Choose a different " + "(a fresh subdirectory, not '/', $HOME, or the current working directory)." +) + +_CLI_INIT_DEMO_FIXTURE_MISSING_DEFAULT_REMEDIATION: str = ( + "Reinstall signalforge-dbt — the bundled demo fixture is missing from your install." +) + +_CLI_INIT_DEMO_COPY_DEFAULT_REMEDIATION: str = ( + "Check disk space, permissions, and that the parent directory of is writable." +) + + +class CliInitDemoDestExistsError(CliInputError): + """Raised by ``cmd_init_demo`` when the destination directory exists + and is non-empty and ``--force`` was not supplied. + + Wraps :class:`signalforge.demo.DemoDestExistsError`. Tier 2 (input + validation — the operator chose a non-empty destination without + opting in to clobbering it). Mirrors the precedent set by + :class:`signalforge.manifest.errors.ModelNotFoundError` (tier 2 for + "the operator named something that conflicts with project state"). + + Message shape: + ``f"destination {dest!r} exists and is not empty: {cause}"`` when a + ``cause`` is provided; otherwise just the dest path. The trailing + ``cause`` rendering reflects the underlying ``DemoDestExistsError``'s + own message body for diagnostic continuity. + """ + + def __init__( + self, + *, + dest: str, + cause: Exception | None = None, + remediation: str | None = None, + ) -> None: + if cause is None: + message = f"destination {dest!r} exists and is not empty" + else: + message = f"destination {dest!r} exists and is not empty: {cause}" + super().__init__( + message, + remediation=( + remediation + if remediation is not None + else _CLI_INIT_DEMO_DEST_EXISTS_DEFAULT_REMEDIATION + ), + ) + self.dest = dest + self.cause = cause + + +class CliInitDemoDestUnsafeError(CliInputError): + """Raised by ``cmd_init_demo`` when ``--force`` would target a + catastrophic path (``/``, ``Path.home()``, or the current working + directory). + + Wraps :class:`signalforge.demo.DemoDestUnsafeError`. Tier 2 (input + validation — the operator named a target the blast-radius guard + refuses to clobber even under ``--force``). DEC-001 of + ``plans/super/47-init-demo.md``. + """ + + def __init__( + self, + *, + dest: str, + cause: Exception | None = None, + remediation: str | None = None, + ) -> None: + if cause is None: + message = ( + f"refusing to --force-replace {dest!r}: would clobber a " + "top-level system or user directory" + ) + else: + message = ( + f"refusing to --force-replace {dest!r}: would clobber a " + f"top-level system or user directory: {cause}" + ) + super().__init__( + message, + remediation=( + remediation + if remediation is not None + else _CLI_INIT_DEMO_DEST_UNSAFE_DEFAULT_REMEDIATION + ), + ) + self.dest = dest + self.cause = cause + + +class CliInitDemoFixtureMissingError(CliError): + """Raised by ``cmd_init_demo`` when the bundled + ``signalforge._demo/`` tree is missing from the installed package. + + Wraps :class:`signalforge.demo.DemoFixtureMissingError`. Tier 1 + (load — the wheel install is broken and there is no work that can + proceed). The wheel-packaging convention in + ``.claude/rules/python-build.md`` makes this practically unreachable + on a clean ``pip install signalforge-dbt`` run, but a corrupted + install (partial wheel extract, hand-edited site-packages) would + surface here. DEC-011 of ``plans/super/47-init-demo.md``. + """ + + def __init__( + self, + *, + cause: Exception | None = None, + remediation: str | None = None, + ) -> None: + if cause is None: + message = "bundled demo fixture is missing from the signalforge-dbt install" + else: + message = f"bundled demo fixture is missing from the signalforge-dbt install: {cause}" + super().__init__( + message, + remediation=( + remediation + if remediation is not None + else _CLI_INIT_DEMO_FIXTURE_MISSING_DEFAULT_REMEDIATION + ), + ) + self.cause = cause + + +class CliInitDemoCopyError(CliError): + """Raised by ``cmd_init_demo`` when the ``shutil.copytree`` / + ``shutil.rmtree`` operation fails with a generic ``OSError``. + + Catch-all for filesystem failures the more-specific + ``CliInitDemo*`` wrappers above do not cover: ENOSPC, EACCES on the + parent directory, EROFS, etc. Tier 1 (load — the filesystem isn't + in a state where work can proceed) per DEC-012 of + ``plans/super/47-init-demo.md``. + """ + + def __init__( + self, + *, + dest: str, + cause: Exception | None = None, + remediation: str | None = None, + ) -> None: + if cause is None: + message = f"failed to copy demo tree to {dest!r}" + else: + message = f"failed to copy demo tree to {dest!r}: {cause}" + super().__init__( + message, + remediation=( + remediation if remediation is not None else _CLI_INIT_DEMO_COPY_DEFAULT_REMEDIATION + ), + ) + self.dest = dest + self.cause = cause diff --git a/src/signalforge/cli/init_demo.py b/src/signalforge/cli/init_demo.py new file mode 100644 index 00000000..802f3a1d --- /dev/null +++ b/src/signalforge/cli/init_demo.py @@ -0,0 +1,213 @@ +"""``signalforge init-demo`` subcommand (US-004 — issue #47). + +Copies the bundled ``signalforge._demo/`` tree into a destination directory +so a first-run operator can ``cd`` into a working dbt project and run +``signalforge lint`` / ``signalforge generate --dry-run`` against +``bigquery-public-data.austin_bikeshare.bikeshare_trips`` without first +authoring their own project. Wraps :func:`signalforge.demo.copy_demo` (the +public library entry point, US-003) and re-raises the four +:class:`signalforge.demo.DemoError` subclasses at the handler boundary as +``CliInitDemo*Error`` wrappers so the CLI's four-tier exit-code taxonomy +stays homogeneous (DEC-012). + +Path-handling note +================== + +``init-demo`` is the one subcommand that *creates* a project rather than +operating *inside* one, so it deliberately does **not** route ``dest`` +through :func:`signalforge.cli._helpers.canonicalise_user_path` — that +helper enforces a ``project_dir`` containment boundary appropriate for +paths consumed inside an existing project (DEC-004 of +``plans/super/47-init-demo.md``). Symlink-cycle defence still applies: +:func:`signalforge.demo.copy_demo` resolves ``dest`` via +``Path(dest).expanduser().resolve(strict=False)`` and raises +:class:`signalforge.demo.DemoPathError` on a cycle; the handler wraps +that into :class:`signalforge.cli.errors.CliPathError`. + +Next-steps message +================== + +On success, the handler prints :data:`_NEXT_STEPS_MESSAGE` to stdout per +DEC-014: plain text (no ANSI, no markdown), names the two env vars +operators must export (``GOOGLE_CLOUD_PROJECT`` and +``ANTHROPIC_API_KEY``), and lists the three first-run commands so an +operator can copy-paste their way to a working pipeline. The message +survives ``--no-color`` because it carries no colour codes. +""" + +from __future__ import annotations + +import argparse +import shlex +import sys + +from signalforge.cli._helpers import ( + format_error_to_stderr, + map_exception_to_exit_code, +) +from signalforge.cli.errors import ( + CliInitDemoCopyError, + CliInitDemoDestExistsError, + CliInitDemoDestUnsafeError, + CliInitDemoFixtureMissingError, + CliPathError, +) +from signalforge.demo import ( + DemoDestExistsError, + DemoDestUnsafeError, + DemoFixtureMissingError, + DemoPathError, + copy_demo, +) + +__all__ = ["add_parser", "cmd_init_demo"] + + +# DEC-014: plain text, stdout, names both env vars + the three first-run +# commands. ``{dest}`` is the resolved-on-disk path returned by +# :func:`copy_demo` so the operator's copy-paste ``cd`` command lands on +# the actual directory rather than the (possibly relative) string they +# typed. ``{dest_quoted}`` shell-quotes the path so an operator whose home +# directory contains spaces (``/Users/Wes Duenow/...``) still gets a valid +# copy-pasteable ``cd`` line. +_NEXT_STEPS_MESSAGE: str = """\ +Demo copied to {dest} + +Next steps: + 1. export GOOGLE_CLOUD_PROJECT= + 2. export ANTHROPIC_API_KEY= + 3. cd {dest_quoted} + 4. signalforge lint + 5. signalforge generate models/staging/stg_bikeshare_trips.sql --dry-run + +The demo uses bigquery-public-data.austin_bikeshare.bikeshare_trips. The +bundled profiles.yml reads GOOGLE_CLOUD_PROJECT from your environment, so +no profile editing is required. +""" + + +def add_parser(subparsers: argparse._SubParsersAction) -> None: # type: ignore[type-arg] + """Register the ``init-demo`` subcommand on the top-level parser. + + Mirrors the registration shape of :mod:`signalforge.cli.lint` and + :mod:`signalforge.cli.version` (DEC-009 of ``.claude/rules/cli-layer.md`` + — one flat module per subcommand). Two surfaces: + + * Positional ``dest`` — optional (``nargs="?"``) with a string + default of ``./signalforge-demo/``. String (not :class:`pathlib.Path`) + so argparse's default stringification is predictable across + Python versions and platforms; :func:`copy_demo` itself runs + ``Path(dest)`` so callers can pass either form. + * ``--force`` — boolean flag (default ``False``). Triggers + :func:`copy_demo`'s atomic-replace path (``rmtree`` then + ``copytree``); refuses non-empty dest unless ``--force`` is + supplied (DEC-001). + """ + parser = subparsers.add_parser( + "init-demo", + help="Copy the bundled demo dbt project into a fresh directory.", + description=( + "Copy the bundled signalforge demo project (Austin " + "bikeshare staging model against bigquery-public-data) into " + " so you can run 'signalforge lint' and 'signalforge " + "generate --dry-run' against a known-good fixture. The " + "subcommand refuses non-empty dest unless --force is " + "supplied; --force will not clobber '/', $HOME, or the " + "current working directory." + ), + ) + parser.add_argument( + "dest", + nargs="?", + default="./signalforge-demo/", + metavar="DEST", + help=( + "Destination directory for the demo project. Default: " + "./signalforge-demo/. Refuses non-empty dest unless --force " + "is supplied." + ), + ) + parser.add_argument( + "--force", + action="store_true", + default=False, + help=( + "Atomically replace dest if it exists and is non-empty. " + "Refuses '/', $HOME, and the current working directory as " + "a blast-radius guard." + ), + ) + parser.set_defaults(func=cmd_init_demo) + + +def cmd_init_demo(args: argparse.Namespace) -> int: + """Copy the bundled demo project to ``args.dest`` and print next steps. + + Returns the integer exit code per the four-tier CLI taxonomy + (DEC-008 of ``.claude/rules/cli-layer.md``): + + * ``0`` — copy succeeded; next-steps message printed to stdout. + * ``1`` — broken install (:class:`CliInitDemoFixtureMissingError`), + symlink cycle (:class:`CliPathError`), or generic filesystem + failure (:class:`CliInitDemoCopyError`). + * ``2`` — operator-side dest mistakes: + :class:`CliInitDemoDestExistsError` (non-empty dest without + ``--force``) or :class:`CliInitDemoDestUnsafeError` (``--force`` + against ``/``, ``Path.home()``, or cwd). + + The single ``try / except Exception`` boundary matches DEC-016 (no + traceback ever leaks); failures route through + :func:`format_error_to_stderr` so the canonical ``ERROR: `` + + ``↳ Remediation: `` shape applies uniformly with the rest + of the CLI. + """ + try: + resolved_dest = copy_demo(args.dest, force=args.force) + except DemoDestExistsError as exc: + wrapped: Exception = CliInitDemoDestExistsError(dest=str(args.dest), cause=exc) + print(format_error_to_stderr(wrapped), file=sys.stderr) + return map_exception_to_exit_code(wrapped) + except DemoDestUnsafeError as exc: + wrapped = CliInitDemoDestUnsafeError(dest=str(args.dest), cause=exc) + print(format_error_to_stderr(wrapped), file=sys.stderr) + return map_exception_to_exit_code(wrapped) + except DemoFixtureMissingError as exc: + wrapped = CliInitDemoFixtureMissingError(cause=exc) + print(format_error_to_stderr(wrapped), file=sys.stderr) + return map_exception_to_exit_code(wrapped) + except DemoPathError as exc: + # Symlink-cycle resolve failure — re-use the existing CliPathError + # so every CLI-originated path-safety failure produces one error + # type (DEC-012 — re-use rather than add a new wrapper for the + # path case). + wrapped = CliPathError( + f"failed to resolve dest path {str(args.dest)!r}: {exc}", + remediation=("Remove the symlink cycle at the destination or pick a different path."), + ) + print(format_error_to_stderr(wrapped), file=sys.stderr) + return map_exception_to_exit_code(wrapped) + except (KeyboardInterrupt, SystemExit): + # Preserve Python's default semantics for operator Ctrl-C and + # any clean SystemExit raised from within copy_demo (none today, + # but defensive parity with the rest of the CLI). + raise + except OSError as exc: + # Generic filesystem failure from shutil.copytree / rmtree: + # ENOSPC, EACCES on the parent, EROFS, etc. Tier 1 per DEC-012. + wrapped = CliInitDemoCopyError(dest=str(args.dest), cause=exc) + print(format_error_to_stderr(wrapped), file=sys.stderr) + return map_exception_to_exit_code(wrapped) + except Exception as exc: # noqa: BLE001 — uniform CLI boundary catch (DEC-016) + # Belt-and-braces — any forward-compat exception added to the + # demo helper's raise surface routes through the canonical + # formatter + mapper rather than leaking a traceback. + print(format_error_to_stderr(exc), file=sys.stderr) + return map_exception_to_exit_code(exc) + + print( + _NEXT_STEPS_MESSAGE.format( + dest=resolved_dest, + dest_quoted=shlex.quote(str(resolved_dest)), + ) + ) + return 0 diff --git a/src/signalforge/demo/__init__.py b/src/signalforge/demo/__init__.py new file mode 100644 index 00000000..b8263f20 --- /dev/null +++ b/src/signalforge/demo/__init__.py @@ -0,0 +1,184 @@ +"""Public ``signalforge.demo`` subpackage — programmatic access to the +bundled demo project. + +Library callers (notebooks, scripts, CI bootstrap) can copy the bundled +``signalforge._demo/`` tree into a fresh directory via +:func:`copy_demo`. The CLI subcommand ``signalforge init-demo`` +wraps this function and re-raises the lower-level :class:`DemoError` +subclasses into ``Cli*Error`` wrappers so the CLI exit-code taxonomy +stays homogeneous (DEC-012). + +Path-handling note +================== + +``copy_demo`` does **not** route ``dest`` through the project-wide +``canonicalise_user_path`` helper — that helper enforces a ``project_dir`` +containment boundary appropriate for paths the CLI consumes *inside* an +existing project. ``init-demo`` is the one entry point in the toolchain +that *creates* a project, so the containment gate doesn't apply. + +The function still defends against symlink cycles (``.resolve()`` raises +``RuntimeError`` on cycles regardless of ``strict=``) and refuses, when +``force=True``, to nuke any of ``/``, ``Path.home()``, or ``Path.cwd()`` +(DEC-001 — the ``--force`` blast-radius guard). + +See ``plans/super/47-init-demo.md`` § US-003 + DEC-001 / DEC-004 / DEC-005 +/ DEC-011 / DEC-012 for the full contract. +""" + +from __future__ import annotations + +import shutil +from importlib.resources import as_file, files +from pathlib import Path + +from signalforge.demo.errors import ( + DemoDestExistsError, + DemoDestUnsafeError, + DemoError, + DemoFixtureMissingError, + DemoPathError, +) + +__all__ = [ + "DemoDestExistsError", + "DemoDestUnsafeError", + "DemoError", + "DemoFixtureMissingError", + "DemoPathError", + "copy_demo", +] + + +def copy_demo(dest: Path | str, *, force: bool = False) -> Path: + """Copy the bundled ``signalforge._demo/`` tree to ``dest``. + + Parameters + ---------- + dest: + Destination directory. Resolved via + ``Path(dest).expanduser().resolve(strict=False)`` — a relative + path resolves against the current working directory; ``~`` + expands; symlinks are followed; cycles raise + :class:`DemoPathError`. + force: + If ``True``, a non-empty existing destination is replaced + atomically (``shutil.rmtree`` then ``shutil.copytree``). The + sanity gate refuses ``force=True`` against ``/``, + ``Path.home()``, or ``Path.cwd()`` (DEC-001). + + Returns + ------- + Path + The resolved destination path (post ``.expanduser().resolve()``). + Library callers and the CLI's next-steps message both consume + this for downstream messaging. + + Raises + ------ + DemoPathError + Symlink cycle at ``dest``. + DemoDestUnsafeError + ``force=True`` against ``/``, ``Path.home()``, or ``Path.cwd()``. + DemoDestExistsError + ``dest`` exists, is non-empty, and ``force=False``. + DemoFixtureMissingError + The bundled ``_demo/`` tree is missing from the installed + package (broken install). + """ + + raw = Path(dest) + expanded_dest = raw.expanduser() + try: + resolved_dest = expanded_dest.resolve(strict=False) + except RuntimeError as exc: # symlink cycle + raise DemoPathError( + f"failed to resolve destination path {str(raw)!r}: {exc}", + cause=exc, + ) from exc + + # Symlink + --force blast-radius guard. `resolve()` above followed the + # link, so if we proceeded with `force=True` the existence gate's + # `shutil.rmtree(resolved_dest)` would delete the symlink TARGET, not + # the link itself — an unintended external location. Refuse loudly. + # Without --force the existing behaviour is preserved: a symlink-dest + # copy follows the link and writes into the target directory (pinned + # by ``test_copy_demo_with_symlink_dest_resolves_target``). DEC-001 + # extended by QG follow-up to cover the symlink case. + if expanded_dest.is_symlink() and force: + raise DemoDestUnsafeError( + f"refusing to --force-replace symlink destination {str(expanded_dest)!r}: " + "would follow the link and clobber the resolved target. Remove the " + "symlink first or pick a different destination." + ) + + # DEC-001 blast-radius guard — only fires under force=True; without + # force the existence gate below handles the same paths benignly + # (a non-empty home / cwd / "/" raises DemoDestExistsError instead). + if force: + unsafe_targets = { + Path("/").resolve(), + Path.home().resolve(), + Path.cwd().resolve(), + } + if resolved_dest in unsafe_targets: + raise DemoDestUnsafeError( + f"refusing to --force-replace {str(resolved_dest)!r}: " + "would clobber a top-level system or user directory" + ) + + # Shape gate — if dest exists but isn't a directory (file, symlink to + # a file, etc.), treat it as non-empty content. Without this + # pre-check, the iterdir() below would raise NotADirectoryError + # which surfaces through the CLI as a misleading + # "failed to copy demo tree" wrap instead of the clear + # DemoDestExistsError the operator should see. + if resolved_dest.exists() and not resolved_dest.is_dir(): + if not force: + raise DemoDestExistsError( + f"destination {str(resolved_dest)!r} exists but is not a directory" + ) + # force=True against a non-directory → remove the file/symlink + # and fall through to the fresh-dest branch. + resolved_dest.unlink() + + # Existence gate — non-empty + no force → loud refusal. Empty dirs + # and non-existent dests both fall through to the copy. We track + # whether the dest was already a (empty) directory so we can pass + # ``dirs_exist_ok=True`` to ``shutil.copytree`` for that case. + dest_existed_empty = False + if resolved_dest.exists() and any(resolved_dest.iterdir()): + if not force: + raise DemoDestExistsError(f"destination {str(resolved_dest)!r} exists and is not empty") + # force=True with non-empty dest → atomic replace. + shutil.rmtree(resolved_dest) + elif resolved_dest.is_dir(): + # Empty existing directory — copytree refuses to clobber the dir + # by default, so we opt into ``dirs_exist_ok`` for this branch. + dest_existed_empty = True + + # Source lookup via importlib.resources — handles editable installs, + # wheel installs, and zipapp/zipimport cases. ``as_file`` materialises + # zip-extracted resources to a real Path; for filesystem installs it's + # an effective no-op. All file I/O is performed inside the ``with`` + # block so the materialised path is valid for the duration of the copy. + source_ref = files("signalforge").joinpath("_demo") + if not source_ref.is_dir(): + raise DemoFixtureMissingError( + "bundled signalforge._demo/ tree not found in the installed package" + ) + + with as_file(source_ref) as source_path: + # symlinks=False: follow symlinks during the copy (copy contents, + # not the link itself). DEC-005 + the parity test pins zero + # symlinks in the shipped tree, so this codifies the no-symlink + # policy: if a symlink ever sneaks in, the consumer gets a real + # file at the other end. + shutil.copytree( + source_path, + resolved_dest, + symlinks=False, + dirs_exist_ok=dest_existed_empty, + ) + + return resolved_dest diff --git a/src/signalforge/demo/errors.py b/src/signalforge/demo/errors.py new file mode 100644 index 00000000..9fbf54ce --- /dev/null +++ b/src/signalforge/demo/errors.py @@ -0,0 +1,119 @@ +"""Typed error hierarchy for ``signalforge.demo``. + +Mirrors the layer-base pattern in every other ``signalforge.*.errors`` +module (manifest, warehouse, safety, llm, draft, prune, grade, diff, +cli). The ``DemoError`` base carries an optional ``remediation`` field; +``__str__`` renders ``message`` plus a ``↳ Remediation: `` line +when remediation is set. Subclasses define a ``default_remediation`` +class attribute used when no explicit ``remediation`` is provided. + +The CLI subcommand ``signalforge init-demo`` (see +``signalforge.cli.init_demo``) catches each concrete subclass and +re-raises it as the matching ``CliInitDemo*Error`` so the CLI +exit-code taxonomy stays homogeneous (DEC-012 of +``plans/super/47-init-demo.md``). The 7th AST scan in +``tests/test_audit_completeness.py`` walks every ``errors.py`` under +``src/signalforge/*/`` (including this one) and gates that every +concrete leaf appears in +``signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE``. The four +concretes below are mapped there at the same tiers as their CLI +wrappers (defence-in-depth — a future ``Demo*Error`` subclass that +escapes the CLI's try/except ladder will still get a sensible exit +code via ``map_exception_to_exit_code``'s MRO walk). +""" + +from __future__ import annotations + +__all__ = [ + "DemoDestExistsError", + "DemoDestUnsafeError", + "DemoError", + "DemoFixtureMissingError", + "DemoPathError", +] + + +class DemoError(Exception): + """Abstract base for ``signalforge.demo`` errors. + + Listed in ``_EXCEPTION_MAPPING_EXCLUDED_BASES`` — every concrete + leaf below must appear in the exit-code mapping, but the base is + excluded (the MRO walk in ``map_exception_to_exit_code`` resolves + forward-compat subclasses to their parent's tier). + """ + + default_remediation: str | None = None + + def __init__( + self, + message: str, + *, + remediation: str | None = None, + cause: Exception | None = None, + ) -> None: + super().__init__(message) + self.message = message + self.remediation = remediation if remediation is not None else self.default_remediation + self.cause = cause + + def __str__(self) -> str: + if self.remediation is None: + return self.message + return f"{self.message}\n ↳ Remediation: {self.remediation}" + + +class DemoPathError(DemoError): + """Raised when the destination path cannot be canonicalised. + + Currently fires on symlink-cycle detection (``Path.resolve()`` + raises ``RuntimeError`` regardless of ``strict=`` on a cyclic + chain). The triggering ``RuntimeError`` rides on the ``cause`` + kwarg. The CLI wraps this as ``CliPathError`` (tier 1). + """ + + default_remediation = "Remove the symlink cycle at the destination or pick a different path." + + +class DemoDestExistsError(DemoError): + """Raised when ``dest`` exists and is non-empty and ``force=False``. + + Empty existing directories proceed without ``--force`` — the gate + only fires when there is content that ``copy_demo`` would + otherwise have to either merge into (unsafe) or replace (requires + explicit opt-in). The CLI wraps this as + ``CliInitDemoDestExistsError`` (tier 2). + """ + + default_remediation = ( + "Remove the destination or pass force=True (CLI: --force) to replace it " + "(refuses '/', $HOME, or the current working directory)." + ) + + +class DemoDestUnsafeError(DemoError): + """Raised when ``force=True`` would target a catastrophic path. + + Refuses ``/``, ``Path.home()``, and ``Path.cwd()`` per DEC-001 — + the blast-radius guard for ``signalforge init-demo --force``. + Without the refusal, ``--force ~`` would ``rmtree($HOME)``. The + CLI wraps this as ``CliInitDemoDestUnsafeError`` (tier 2). + """ + + default_remediation = ( + "Pick a fresh subdirectory rather than '/', $HOME, or the current working directory." + ) + + +class DemoFixtureMissingError(DemoError): + """Raised when ``importlib.resources`` cannot locate the bundled + ``_demo/`` tree. + + Indicates a broken install — the wheel target packaging should + always ship ``src/signalforge/_demo/`` (``python-build.md`` DEC-011 + + plan DEC-002). The CLI wraps this as + ``CliInitDemoFixtureMissingError`` (tier 1). + """ + + default_remediation = ( + "Reinstall signalforge-dbt — the bundled demo tree is missing from your install." + ) diff --git a/src/signalforge/warehouse/__init__.py b/src/signalforge/warehouse/__init__.py index 402bb0ab..9e7b62c6 100644 --- a/src/signalforge/warehouse/__init__.py +++ b/src/signalforge/warehouse/__init__.py @@ -34,6 +34,7 @@ ManifestSchemaNotFoundError, MaterialisationFailedError, MaterialisationNotSupportedError, + ProfileEnvVarUnsetError, ProfileNotFoundError, ProfileTargetNotFoundError, QuerySyntaxError, @@ -75,6 +76,7 @@ "MaterialisationFailedError", "MaterialisationNotSupportedError", "PartitionFilter", + "ProfileEnvVarUnsetError", "ProfileNotFoundError", "ProfileTargetNotFoundError", "QuerySyntaxError", diff --git a/src/signalforge/warehouse/errors.py b/src/signalforge/warehouse/errors.py index c5a4fc34..9c17b953 100644 --- a/src/signalforge/warehouse/errors.py +++ b/src/signalforge/warehouse/errors.py @@ -128,6 +128,52 @@ def __init__( super().__init__(message, remediation=remediation) +class ProfileEnvVarUnsetError(ProfileNotFoundError): + """The profile references ``env_var('NAME')`` for a variable that is + not set in the environment and has no default supplied. + + Inherits from :class:`ProfileNotFoundError` so existing callers + catching "profile won't load" with one except keep working + (init-demo's documented happy path falls back to ``profiles.yml`` + edits when this fires, so the operator sees one error type for + "profile broken"). + + The dbt convention ``env_var('NAME', 'default')`` resolves to the + default and never raises; only the no-default form trips this. + Added by issue #47 to support the demo's + ``{{ env_var('GOOGLE_CLOUD_PROJECT') }}`` profile. + """ + + default_remediation: ClassVar[str] = ( + "Set the environment variable named in the message, or supply a default " + "to the env_var(...) call (e.g. env_var('NAME', 'fallback'))." + ) + + def __init__( + self, + var_name: str, + profiles_path: Path, + *, + remediation: str | None = None, + ) -> None: + self.var_name = var_name + self.profiles_path = profiles_path + message = ( + f"profiles.yml at {_format_value(str(profiles_path))} references " + f"env_var({_format_value(var_name)}) but environment variable " + f"{_format_value(var_name)} is not set and no default was supplied." + ) + if remediation is None: + remediation = ( + f"Set the environment variable: `export {var_name}=`, " + f"or edit {_format_value(str(profiles_path))} to supply a default: " + f"`env_var('{var_name}', '')`." + ) + # Track searched_paths so parent contract holds. + self.searched_paths: list[Path] = [profiles_path] + WarehouseError.__init__(self, message, remediation=remediation) + + class ProfileTargetNotFoundError(ProfileNotFoundError): """The profile resolved but the requested ``target`` field is missing. @@ -455,6 +501,7 @@ def __init__(self, adapter_name: str, *, remediation: str | None = None) -> None "ManifestSchemaNotFoundError", "MaterialisationFailedError", "MaterialisationNotSupportedError", + "ProfileEnvVarUnsetError", "ProfileNotFoundError", "ProfileTargetNotFoundError", "QuerySyntaxError", diff --git a/src/signalforge/warehouse/profiles.py b/src/signalforge/warehouse/profiles.py index 6460fa6d..58f9f898 100644 --- a/src/signalforge/warehouse/profiles.py +++ b/src/signalforge/warehouse/profiles.py @@ -44,6 +44,7 @@ import logging import os +import re from pathlib import Path from typing import Any @@ -52,6 +53,7 @@ from signalforge.warehouse._path_safety import canonicalise_path from signalforge.warehouse.errors import ( + ProfileEnvVarUnsetError, ProfileNotFoundError, ProfileTargetNotFoundError, UnsupportedAuthMethodError, @@ -161,11 +163,61 @@ def _maybe_warn_large_profile(path: Path) -> None: ) +_ENV_VAR_RE = re.compile( + # dbt-compatible: ``{{ env_var('NAME') }}`` or ``{{ env_var("NAME") }}`` + # with an optional second positional arg used as the default. + # Whitespace between args is tolerated. The outer ``{{ ... }}`` jinja + # brackets are REQUIRED — this matches dbt's own jinja-rendering + # semantics (a bare ``env_var('NAME')`` is just a string in dbt's + # YAML, not an env-var reference). If the bundled init-demo profile + # ever needs the bare form, extend the regex (and document the + # divergence from dbt) — for v0.1 we follow dbt's contract. + r"""\{\{\s*env_var\(\s*['"]([A-Za-z_][A-Za-z0-9_]*)['"]""" + r"""(?:\s*,\s*['"]([^'"]*)['"])?\s*\)\s*\}\}""", + re.VERBOSE, +) + + +def _render_env_vars(text: str, *, path: Path) -> str: + """Render dbt-style ``{{ env_var('NAME') }}`` macros in ``text``. + + Supports two forms — ``env_var('NAME')`` (raises + :class:`ProfileEnvVarUnsetError` if NAME is unset) and + ``env_var('NAME', 'default')`` (falls back to the literal default). + + This is a deliberately minimal jinja-compat shim — full jinja + rendering (loops, conditionals, other macros) is out of scope. Just + enough to make the bundled ``signalforge init-demo`` profile's + ``project: "{{ env_var('GOOGLE_CLOUD_PROJECT') }}"`` line work + without an extra rendering step. Issue #47. + """ + + def _replace(match: re.Match[str]) -> str: + var_name = match.group(1) + default = match.group(2) + value = os.environ.get(var_name) + if value is not None: + return value + if default is not None: + return default + raise ProfileEnvVarUnsetError(var_name=var_name, profiles_path=path) + + return _ENV_VAR_RE.sub(_replace, text) + + def _load_profiles_yaml(path: Path) -> dict[str, Any]: - """Read and parse ``path`` as YAML, returning the top-level mapping.""" + """Read and parse ``path`` as YAML, returning the top-level mapping. + + Applies a dbt-compatible ``env_var('NAME')`` substitution pass over + the raw text before YAML parsing — see :func:`_render_env_vars`. The + substitution runs against the YAML text (not the parsed structure) + so quoted ``"{{ env_var('NAME') }}"`` strings cleanly become quoted + ``""`` strings; YAML quoting rules are preserved. + """ _maybe_warn_large_profile(path) - with path.open("r", encoding="utf-8") as fh: - raw = yaml.safe_load(fh) + raw_text = path.read_text(encoding="utf-8") + rendered_text = _render_env_vars(raw_text, path=path) + raw = yaml.safe_load(rendered_text) if not isinstance(raw, dict): raise ProfileNotFoundError( searched_paths=[path], diff --git a/tests/cli/test_5_surface_parity_init_demo.py b/tests/cli/test_5_surface_parity_init_demo.py new file mode 100644 index 00000000..a30da530 --- /dev/null +++ b/tests/cli/test_5_surface_parity_init_demo.py @@ -0,0 +1,220 @@ +"""5-surface parity test for the issue #47 / US-005 ``init-demo`` subcommand. + +This bead's DEC-001 (``--force`` semantics) plus the ``cli-layer.md`` +5-surface parity rule require that the ``init-demo`` subcommand name and +the ``--force`` flag appear consistently across: + +1. **argparse help** — the ``init-demo`` subparser's ``--help`` output. + Source of truth lives in :func:`signalforge.cli.init_demo.add_parser` + (US-004 wired it). +2. **Handler docstring** — :mod:`signalforge.cli.init_demo`'s module + docstring plus :func:`signalforge.cli.init_demo.cmd_init_demo`'s + docstring. Both reference the flag and the subcommand by name. +3. **docs/cli-ops.md § Subcommands** — the ``signalforge init-demo`` + subsection ships with US-007. +4. **plans/super/47-init-demo.md** — DEC-001 names the ``--force`` + semantics; the user-story section also names the subcommand. +5. **The test file itself** — implicitly satisfied (this file). + +The test reads bytes from each external surface at runtime and asserts +the canonical tokens (``"init-demo"`` and ``"--force"``) appear in each. +Bespoke per ``cli-layer.md`` 5-surface parity rule — future flags get +their own parity test (or extend this one). +""" + +from __future__ import annotations + +import argparse +from pathlib import Path + +import pytest + +import signalforge.cli.init_demo as init_demo_module +from signalforge.cli import main +from signalforge.cli.init_demo import add_parser, cmd_init_demo + +# --------------------------------------------------------------------------- +# Surface locations +# --------------------------------------------------------------------------- + +# The plan + ops doc live at the repository root; ``__file__`` is at +# ``tests/cli/test_5_surface_parity_init_demo.py``. +_REPO_ROOT = Path(__file__).resolve().parent.parent.parent +_PLAN_FILE = _REPO_ROOT / "plans" / "super" / "47-init-demo.md" +_OPS_DOC = _REPO_ROOT / "docs" / "cli-ops.md" + +# Canonical tokens the four external surfaces must all carry. Sourced from +# DEC-001 (``--force`` semantics) and the subcommand name itself. +_CANONICAL_TOKENS = ( + "init-demo", + "--force", +) + + +def _init_demo_help_text() -> str: + """Render the full ``signalforge init-demo --help`` output. + + Invokes :func:`signalforge.cli.main` with ``["init-demo", "--help"]`` + so the test exercises the same argparse surface an operator sees on + the command line. argparse's ``--help`` action raises + :class:`SystemExit` after printing; we capture stdout via + :class:`pytest.CaptureFixture` upstream. + """ + parser = argparse.ArgumentParser(prog="signalforge") + subparsers = parser.add_subparsers(dest="command") + add_parser(subparsers) + sub = subparsers.choices["init-demo"] + return sub.format_help() + + +# --------------------------------------------------------------------------- +# Surface 1: argparse help +# --------------------------------------------------------------------------- + + +def test_init_demo_in_argparse_help() -> None: + """Each canonical token appears in the rendered ``init-demo --help`` + output (surface 1 of 5). + """ + help_text = _init_demo_help_text() + for token in _CANONICAL_TOKENS: + assert token in help_text, ( + f"init-demo --help missing canonical token {token!r}; got:\n{help_text}" + ) + + +def test_init_demo_help_via_main_entrypoint(capsys: pytest.CaptureFixture[str]) -> None: + """End-to-end variant of surface 1: drive ``main(["init-demo", "--help"])`` + so argparse's ``--help`` action prints to stdout. + + Belt-and-braces against a refactor that moves the subparser + registration out of :func:`add_parser` — only the full ``main`` + dispatch path catches that drift. argparse's ``--help`` action + raises :class:`SystemExit(0)`; :func:`signalforge.cli.main` catches + it and returns ``0`` so the ``-> int`` contract holds (see + ``.claude/rules/cli-layer.md`` § "No traceback ever leaks"). + """ + rc = main(["init-demo", "--help"]) + # argparse --help exits 0; main() returns it as an int. + assert rc == 0 + captured = capsys.readouterr() + for token in _CANONICAL_TOKENS: + assert token in captured.out, ( + f"main(['init-demo', '--help']) stdout missing token {token!r}; got:\n{captured.out}" + ) + + +# --------------------------------------------------------------------------- +# Surface 2: handler docstring +# --------------------------------------------------------------------------- + + +def test_init_demo_in_handler_docstring() -> None: + """Each canonical token appears in either the module docstring or the + handler docstring (surface 2 of 5). + + The handler ships two docstring surfaces — the module-level one + (describing what ``init-demo`` does and how ``--force`` interacts with + :func:`signalforge.demo.copy_demo`) and the per-function one on + :func:`cmd_init_demo`. The parity check accepts a hit in either so a + future refactor that consolidates the prose into one surface doesn't + break the contract. + """ + module_doc = init_demo_module.__doc__ or "" + handler_doc = cmd_init_demo.__doc__ or "" + add_parser_doc = add_parser.__doc__ or "" + combined = "\n".join((module_doc, handler_doc, add_parser_doc)) + for token in _CANONICAL_TOKENS: + assert token in combined, ( + f"signalforge.cli.init_demo docstrings missing canonical token " + f"{token!r}; got module:\n{module_doc}\n\nhandler:\n{handler_doc}\n\n" + f"add_parser:\n{add_parser_doc}" + ) + + +# --------------------------------------------------------------------------- +# Surface 3: docs/cli-ops.md § Subcommands +# --------------------------------------------------------------------------- + + +def test_init_demo_in_cli_ops_doc() -> None: + """Each canonical token appears in ``docs/cli-ops.md`` (surface 3 of 5). + + The check is intentionally whole-file rather than scoped to the + ``init-demo`` subsection — restricting to the subsection would + couple the test to the doc's heading structure (brittle on a + refactor that splits or merges sections). + """ + assert _OPS_DOC.exists(), f"docs/cli-ops.md not found at {_OPS_DOC}" + ops_text = _OPS_DOC.read_text(encoding="utf-8") + for token in _CANONICAL_TOKENS: + assert token in ops_text, ( + f"docs/cli-ops.md missing canonical token {token!r} — " + "5-surface parity break (US-007 ships this surface)" + ) + + +# --------------------------------------------------------------------------- +# Surface 4: plans/super/47-init-demo.md DEC list +# --------------------------------------------------------------------------- + + +def test_init_demo_in_plan_dec_list() -> None: + """Each canonical token appears in ``plans/super/47-init-demo.md`` + (surface 4 of 5). + + The plan's DEC-001 names the ``--force`` semantics; the + user-story section names the subcommand. Whole-file check rather + than DEC-scoped for the same reason as surface 3 — the contract is + "the tokens appear somewhere in the plan," not "in a specific + section." + """ + assert _PLAN_FILE.exists(), f"plan file not found at {_PLAN_FILE}" + plan_text = _PLAN_FILE.read_text(encoding="utf-8") + for token in _CANONICAL_TOKENS: + assert token in plan_text, ( + f"plans/super/47-init-demo.md missing canonical token {token!r} " + "— 5-surface parity break" + ) + + +# --------------------------------------------------------------------------- +# Aggregate parity summary +# --------------------------------------------------------------------------- + + +def test_force_flag_consistent_across_surfaces( + capsys: pytest.CaptureFixture[str], +) -> None: + """Aggregate check: every canonical token appears in every external + surface (1, 2, 3, 4 — the 5th is this test file). + + This is the single test a reviewer reads first to verify the + contract; the per-surface tests above pinpoint exactly which + surface drifted on failure. + """ + surfaces: dict[str, str] = { + "argparse_help": _init_demo_help_text(), + "handler_docstring": "\n".join( + ( + init_demo_module.__doc__ or "", + cmd_init_demo.__doc__ or "", + add_parser.__doc__ or "", + ) + ), + "cli_ops_doc": _OPS_DOC.read_text(encoding="utf-8"), + "plan_dec_list": _PLAN_FILE.read_text(encoding="utf-8"), + } + missing: list[tuple[str, str]] = [] + for surface_name, surface_text in surfaces.items(): + for token in _CANONICAL_TOKENS: + if token not in surface_text: + missing.append((surface_name, token)) + assert not missing, ( + f"5-surface parity break — canonical tokens missing from one or more surfaces: {missing!r}" + ) + # Drain any stdout the help-rendering helpers produced (argparse's + # ``--help`` action prints when exercised through ``main(...)``; here + # we used ``format_help`` so no stdout, but capsys is part of the + # signature for parity with the surface-1 test). + capsys.readouterr() diff --git a/tests/cli/test_exit_codes.py b/tests/cli/test_exit_codes.py index 2c89665f..e329eac7 100644 --- a/tests/cli/test_exit_codes.py +++ b/tests/cli/test_exit_codes.py @@ -242,6 +242,26 @@ class _Probe(BaseModel): if name == "CliSelectorNoMatchError": return cls(expr="tag:nonexistent") + # init-demo CLI wrappers (issue #47 / DEC-012, DEC-013 — US-004). + # Each wrapper takes keyword-only kwargs (``dest=`` / ``cause=`` / + # ``remediation=``); the dest-exists and dest-unsafe variants are + # tier 2 (input-validation), fixture-missing and copy-error are + # tier 1 (broken install / generic filesystem failure). + if name in {"CliInitDemoDestExistsError", "CliInitDemoDestUnsafeError"}: + return cls(dest="/tmp/synthetic", cause=_SENTINEL_CAUSE) + if name == "CliInitDemoFixtureMissingError": + return cls(cause=_SENTINEL_CAUSE) + if name == "CliInitDemoCopyError": + return cls(dest="/tmp/synthetic", cause=_SENTINEL_CAUSE) + + # Warehouse profile env_var failure (issue #47 — supports init-demo's + # bundled `{{ env_var('GOOGLE_CLOUD_PROJECT') }}` profile). Requires + # (var_name, profiles_path) as positional args. + if name == "ProfileEnvVarUnsetError": + from pathlib import Path + + return cls(var_name="SYNTHETIC_VAR", profiles_path=Path("/tmp/synthetic/profiles.yml")) + # Catch-all: layer-base default ``Cls(message, *, remediation=None)``. try: return cls(_SENTINEL_MESSAGE) diff --git a/tests/cli/test_init_demo.py b/tests/cli/test_init_demo.py new file mode 100644 index 00000000..5c9dd97f --- /dev/null +++ b/tests/cli/test_init_demo.py @@ -0,0 +1,506 @@ +"""Tests for ``signalforge init-demo`` (US-004 — issue #47). + +In-process e2e via :func:`signalforge.cli.main` + ``capsys``. Covers the +US-004 acceptance criteria: + +* happy path (fresh dest) → exit 0, next-steps printed to stdout +* DEC-014 next-steps message names both env vars + the three first-run + commands +* non-empty dest without ``--force`` → exit 2 with remediation +* non-empty dest with ``--force`` → exit 0 (atomic replace) +* ``--force`` against ``Path.home()`` → exit 2 (dest-unsafe) +* every test asserts no traceback leaks (DEC-016 floor) +* ``--help`` lists ``DEST`` positional + ``--force`` flag +* default dest is ``./signalforge-demo/`` +* the four CLI wrapper errors are registered in the exit-code table at + the right tiers (paired with the 7th AST scan in + ``tests/test_audit_completeness.py``) +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from signalforge.cli import main +from signalforge.cli._helpers import _EXCEPTION_TO_EXIT_CODE +from signalforge.cli.errors import ( + CliInitDemoCopyError, + CliInitDemoDestExistsError, + CliInitDemoDestUnsafeError, + CliInitDemoFixtureMissingError, + CliInputError, +) + + +def _capture(capsys: pytest.CaptureFixture[str]) -> tuple[str, str]: + captured = capsys.readouterr() + return captured.out, captured.err + + +_EXPECTED_TOP_LEVEL = frozenset( + { + ".gitignore", + "dbt_project.yml", + "models", + "profiles.yml", + "signalforge.yml", + "target", + } +) + + +# --------------------------------------------------------------------------- +# Happy path +# --------------------------------------------------------------------------- + + +def test_cmd_init_demo_to_fresh_path_returns_0_and_prints_next_steps( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], +) -> None: + dest = tmp_path / "demo" + ret = main(["init-demo", str(dest)]) + out, err = _capture(capsys) + assert ret == 0, f"stdout: {out}\nstderr: {err}" + # Demo tree landed. + assert dest.is_dir() + entries = {p.name for p in dest.iterdir()} + assert _EXPECTED_TOP_LEVEL.issubset(entries), entries + # Next-steps message printed to stdout. + assert "Demo copied to" in out + assert str(dest.resolve()) in out + # Floor: no traceback. + assert "Traceback" not in err + + +def test_cmd_init_demo_emits_next_steps_naming_env_vars( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], +) -> None: + """DEC-014: the next-steps stdout names both env vars + the three commands.""" + dest = tmp_path / "demo" + ret = main(["init-demo", str(dest)]) + out, err = _capture(capsys) + assert ret == 0 + # Both env vars must appear in stdout (DEC-014 contract). + assert "GOOGLE_CLOUD_PROJECT" in out + assert "ANTHROPIC_API_KEY" in out + # The three first-run commands must appear (DEC-014). + assert "signalforge lint" in out + assert "signalforge generate" in out + assert "--dry-run" in out + # Floor: no traceback. + assert "Traceback" not in err + + +def test_cmd_init_demo_to_empty_existing_dir_returns_0( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], +) -> None: + """Empty existing dir proceeds without --force per US-003's contract.""" + dest = tmp_path / "demo" + dest.mkdir() + ret = main(["init-demo", str(dest)]) + out, err = _capture(capsys) + assert ret == 0, f"stdout: {out}\nstderr: {err}" + assert dest.is_dir() + assert (dest / "dbt_project.yml").is_file() + assert "Traceback" not in err + + +# --------------------------------------------------------------------------- +# Existence-gate +# --------------------------------------------------------------------------- + + +def test_cmd_init_demo_against_existing_nonempty_dir_returns_exit_2_with_remediation( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], +) -> None: + dest = tmp_path / "demo" + dest.mkdir() + (dest / "stale.txt").write_text("don't clobber me") + ret = main(["init-demo", str(dest)]) + out, err = _capture(capsys) + assert ret == 2, f"stdout: {out}\nstderr: {err}" + # Canonical CLI error shape + remediation footer. + assert err.startswith("ERROR: ") + assert "exists" in err + assert "↳ Remediation:" in err + assert "--force" in err + # The preexisting file is untouched. + assert (dest / "stale.txt").read_text() == "don't clobber me" + # No traceback floor. + assert "Traceback" not in err + + +def test_cmd_init_demo_force_against_existing_nonempty_dir_returns_0( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], +) -> None: + dest = tmp_path / "demo" + dest.mkdir() + (dest / "stale.txt").write_text("stale") + (dest / "old_sub").mkdir() + (dest / "old_sub" / "x.txt").write_text("also stale") + ret = main(["init-demo", str(dest), "--force"]) + out, err = _capture(capsys) + assert ret == 0, f"stdout: {out}\nstderr: {err}" + # Stale content gone, demo content present. + assert not (dest / "stale.txt").exists() + assert not (dest / "old_sub").exists() + assert (dest / "dbt_project.yml").is_file() + assert "Traceback" not in err + + +def test_cmd_init_demo_dest_is_file_returns_exit_2_with_clear_message( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], +) -> None: + """A ``dest`` that exists as a regular file (not a directory) routes + to tier 2 ``CliInitDemoDestExistsError`` with a "not a directory" + message — NOT a tier-1 ``CliInitDemoCopyError`` wrap of a raw + ``NotADirectoryError`` from ``iterdir()``. Pass-2 QG defence. + """ + dest = tmp_path / "demo" + dest.write_text("i am a regular file, not a directory") + ret = main(["init-demo", str(dest)]) + out, err = _capture(capsys) + assert ret == 2, f"expected tier 2; got {ret}\nstderr: {err}" + assert err.startswith("ERROR: ") + assert "not a directory" in err + # The original file is untouched. + assert dest.read_text() == "i am a regular file, not a directory" + assert "Traceback" not in err + + +# --------------------------------------------------------------------------- +# Force blast-radius guard +# --------------------------------------------------------------------------- + + +def test_cmd_init_demo_force_against_home_returns_exit_2_dest_unsafe( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], +) -> None: + """``--force`` against ``Path.home()`` is refused — DEC-001.""" + fake_home = tmp_path / "fake_home" + fake_home.mkdir() + # Put something inside so the existence-gate would trigger if force + # weren't refused first. + (fake_home / "important_dotfile").write_text("don't nuke me") + monkeypatch.setattr(Path, "home", lambda: fake_home) + ret = main(["init-demo", str(fake_home), "--force"]) + out, err = _capture(capsys) + assert ret == 2, f"stdout: {out}\nstderr: {err}" + assert err.startswith("ERROR: ") + # Message names the catastrophic-path rationale. + assert "system or user" in err or "clobber" in err + # Home directory + its contents survived. + assert fake_home.is_dir() + assert (fake_home / "important_dotfile").read_text() == "don't nuke me" + assert "Traceback" not in err + + +def test_cmd_init_demo_force_against_cwd_returns_exit_2_dest_unsafe( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], +) -> None: + """``--force`` against ``Path.cwd()`` is refused — DEC-001.""" + monkeypatch.chdir(tmp_path) + (tmp_path / "important.txt").write_text("preserved") + ret = main(["init-demo", str(tmp_path), "--force"]) + out, err = _capture(capsys) + assert ret == 2, f"stdout: {out}\nstderr: {err}" + assert err.startswith("ERROR: ") + # cwd survived. + assert (tmp_path / "important.txt").read_text() == "preserved" + assert "Traceback" not in err + + +# --------------------------------------------------------------------------- +# Floor: no traceback +# --------------------------------------------------------------------------- + + +def test_cmd_init_demo_never_leaks_traceback( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], +) -> None: + """DEC-016 of cli-layer.md — floor-of-every-CLI-test assertion. + + Force the path-error branch by passing a destination whose parent is + a regular file (creating it inside is impossible — ``copytree`` + raises ``OSError``). The CLI must wrap into ``CliInitDemoCopyError`` + and print ``ERROR: ...`` without leaking a traceback. + """ + blocker = tmp_path / "blocker" + blocker.write_text("i am a file, not a dir") + dest = blocker / "sub" + ret = main(["init-demo", str(dest)]) + out, err = _capture(capsys) + assert "Traceback" not in err + # Exit code is tier 1 — the copytree OSError is wrapped as + # CliInitDemoCopyError (CliError → tier 1). A tier 2 outcome here + # would mean the OSError was misrouted to an input-validation + # wrapper; assert exact tier so a regression surfaces loudly. + assert ret == 1, f"expected tier 1 (CliInitDemoCopyError); got {ret}\nstderr: {err}" + assert err.startswith("ERROR: ") + + +# --------------------------------------------------------------------------- +# argparse help surface +# --------------------------------------------------------------------------- + + +def test_init_demo_help_lists_force_flag( + capsys: pytest.CaptureFixture[str], +) -> None: + """``signalforge init-demo --help`` mentions ``--force``.""" + ret = main(["init-demo", "--help"]) + out, err = _capture(capsys) + # argparse --help exits 0. + assert ret == 0 + assert "--force" in out + # The help text must explicitly note the non-empty-dest refusal so + # the 5-surface parity test (US-005) can hard-assert this phrase. + assert "non-empty" in out.lower() or "refuses" in out.lower() + assert "Traceback" not in err + + +def test_init_demo_help_lists_dest_positional( + capsys: pytest.CaptureFixture[str], +) -> None: + """``--help`` shows the positional ``DEST`` arg.""" + ret = main(["init-demo", "--help"]) + out, _err = _capture(capsys) + assert ret == 0 + assert "DEST" in out + + +def test_init_demo_default_dest_is_signalforge_demo( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], +) -> None: + """No positional arg → default dest ``./signalforge-demo/``.""" + monkeypatch.chdir(tmp_path) + ret = main(["init-demo"]) + out, err = _capture(capsys) + assert ret == 0, f"stdout: {out}\nstderr: {err}" + default_dest = tmp_path / "signalforge-demo" + assert default_dest.is_dir() + assert (default_dest / "dbt_project.yml").is_file() + assert "Traceback" not in err + + +def test_init_demo_help_default_string_visible( + capsys: pytest.CaptureFixture[str], +) -> None: + """The help output references the default-dest string ``signalforge-demo``. + + Pins DEC-001's default-name choice and pairs with the 5-surface + parity test (US-005) for the default-dest name across the help / + docstring / docs / DEC list. + """ + ret = main(["init-demo", "--help"]) + out, _err = _capture(capsys) + assert ret == 0 + assert "signalforge-demo" in out + + +# --------------------------------------------------------------------------- +# Exit-code table membership (paired with the 7th AST scan) +# --------------------------------------------------------------------------- + + +def test_cli_init_demo_dest_exists_error_in_exit_code_table() -> None: + """The dest-exists wrapper is tier 2 — DEC-013 / 7th AST scan.""" + assert CliInitDemoDestExistsError in _EXCEPTION_TO_EXIT_CODE + assert _EXCEPTION_TO_EXIT_CODE[CliInitDemoDestExistsError] == 2 + # Subclass relationship — every CLI-input-validation error is a + # CliInputError so callers can pattern-match on the base. + assert issubclass(CliInitDemoDestExistsError, CliInputError) + + +def test_cli_init_demo_dest_unsafe_error_in_exit_code_table() -> None: + assert CliInitDemoDestUnsafeError in _EXCEPTION_TO_EXIT_CODE + assert _EXCEPTION_TO_EXIT_CODE[CliInitDemoDestUnsafeError] == 2 + assert issubclass(CliInitDemoDestUnsafeError, CliInputError) + + +def test_cli_init_demo_fixture_missing_error_in_exit_code_table() -> None: + """Broken-install wrapper is tier 1 — DEC-012.""" + assert CliInitDemoFixtureMissingError in _EXCEPTION_TO_EXIT_CODE + assert _EXCEPTION_TO_EXIT_CODE[CliInitDemoFixtureMissingError] == 1 + + +def test_cli_init_demo_copy_error_in_exit_code_table() -> None: + """Generic copy-failure wrapper is tier 1 — DEC-012.""" + assert CliInitDemoCopyError in _EXCEPTION_TO_EXIT_CODE + assert _EXCEPTION_TO_EXIT_CODE[CliInitDemoCopyError] == 1 + + +# --------------------------------------------------------------------------- +# Fixture-missing path (broken-install simulation) +# --------------------------------------------------------------------------- + + +def test_cmd_init_demo_fixture_missing_returns_exit_1( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], +) -> None: + """Simulated broken install → exit 1 with the broken-install remediation.""" + import signalforge.demo as demo_mod + + class _NotADir: + def joinpath(self, name: str) -> _NotADir: + return self + + def is_dir(self) -> bool: + return False + + monkeypatch.setattr(demo_mod, "files", lambda pkg: _NotADir()) + ret = main(["init-demo", str(tmp_path / "demo")]) + out, err = _capture(capsys) + assert ret == 1, f"stdout: {out}\nstderr: {err}" + assert err.startswith("ERROR: ") + assert "missing" in err.lower() or "fixture" in err.lower() + assert "Reinstall" in err + assert "Traceback" not in err + + +# --------------------------------------------------------------------------- +# Remediation footer surfaces in stderr (DEC-017 stderr shape) +# --------------------------------------------------------------------------- + + +def test_cmd_init_demo_dest_exists_remediation_mentions_force( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], +) -> None: + """The remediation footer points operators at ``--force``.""" + dest = tmp_path / "demo" + dest.mkdir() + (dest / "stale.txt").write_text("stale") + ret = main(["init-demo", str(dest)]) + _out, err = _capture(capsys) + assert ret == 2 + # The remediation footer is part of the canonical CLI error shape. + assert "↳ Remediation:" in err + assert "--force" in err + + +# --------------------------------------------------------------------------- +# Coverage gap fills: defensive except branches + cause=None error constructors +# --------------------------------------------------------------------------- + + +def test_cmd_init_demo_demo_path_error_wraps_to_cli_path_error( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A ``DemoPathError`` raised by ``copy_demo`` (symlink cycle on a + platform where ``Path.resolve()`` triggers ``RuntimeError``) is wrapped + as ``CliPathError`` per DEC-012. + + Exercised here via monkeypatch because WSL2's filesystem does not + raise ``RuntimeError`` on symlink cycles — the natural test path in + ``test_demo.py`` is skipped on WSL2 (see the parity test). The CLI + handler branch still needs coverage, so we synthesise the error. + """ + from signalforge.cli import init_demo as init_demo_mod + from signalforge.demo import DemoPathError + + def _raise(*_args: object, **_kwargs: object) -> Path: + raise DemoPathError( + "failed to resolve destination path 'fake': simulated cycle", + cause=RuntimeError("simulated symlink cycle"), + ) + + monkeypatch.setattr(init_demo_mod, "copy_demo", _raise) + ret = main(["init-demo", str(tmp_path / "demo")]) + out, err = _capture(capsys) + assert ret == 1, f"stdout: {out}\nstderr: {err}" + assert err.startswith("ERROR: ") + assert "resolve" in err.lower() or "symlink" in err.lower() + assert "Traceback" not in err + + +def test_cmd_init_demo_keyboard_interrupt_propagates( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + """``KeyboardInterrupt`` from inside ``copy_demo`` is re-raised — the + CLI handler does NOT swallow it (operator Ctrl-C must propagate to + Python's default handler for sane shell semantics). DEC-016 carve-out. + """ + from signalforge.cli import init_demo as init_demo_mod + + def _raise(*_args: object, **_kwargs: object) -> Path: + raise KeyboardInterrupt + + monkeypatch.setattr(init_demo_mod, "copy_demo", _raise) + with pytest.raises(KeyboardInterrupt): + main(["init-demo", str(tmp_path / "demo")]) + + +def test_cmd_init_demo_forward_compat_exception_belt_and_braces( + tmp_path: Path, + capsys: pytest.CaptureFixture[str], + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A forward-compat exception type that ``copy_demo`` might add in + a future version still routes through the canonical formatter + + mapper without leaking a traceback. The catch-all ``except Exception`` + in ``cmd_init_demo`` is the belt-and-braces seam (DEC-016). + """ + from signalforge.cli import init_demo as init_demo_mod + + class _FutureDemoConcurrencyError(Exception): + """Hypothetical v0.x error type.""" + + def _raise(*_args: object, **_kwargs: object) -> Path: + raise _FutureDemoConcurrencyError("demo busy in another process") + + monkeypatch.setattr(init_demo_mod, "copy_demo", _raise) + ret = main(["init-demo", str(tmp_path / "demo")]) + out, err = _capture(capsys) + # Unmapped → tier 1 (panic-path default). + assert ret == 1, f"stdout: {out}\nstderr: {err}" + assert err.startswith("ERROR: ") + assert "demo busy" in err + assert "Traceback" not in err + + +def test_cli_init_demo_error_constructors_render_without_cause() -> None: + """Every ``CliInitDemo*Error`` constructor accepts ``cause=None`` and + renders a sensible message without it. Exit-code-table tests always + pass a ``cause``, so this exercises the ``cause is None`` branch in + each of the four wrappers. + """ + e1 = CliInitDemoDestExistsError(dest="/tmp/x") + assert "exists" in str(e1) + assert e1.cause is None + assert "↳ Remediation:" in str(e1) + + e2 = CliInitDemoDestUnsafeError(dest="/") + assert "force" in str(e2).lower() + assert e2.cause is None + assert "↳ Remediation:" in str(e2) + + e3 = CliInitDemoFixtureMissingError() + assert "missing" in str(e3).lower() + assert e3.cause is None + assert "↳ Remediation:" in str(e3) + + e4 = CliInitDemoCopyError(dest="/tmp/x") + assert "copy" in str(e4).lower() + assert e4.cause is None + assert "↳ Remediation:" in str(e4) diff --git a/tests/cli/test_subprocess_smoke.py b/tests/cli/test_subprocess_smoke.py index 1f7a12d8..09f725dc 100644 --- a/tests/cli/test_subprocess_smoke.py +++ b/tests/cli/test_subprocess_smoke.py @@ -48,3 +48,36 @@ def test_signalforge_version_via_subprocess() -> None: # exactly that. Mirrors the in-process smoke assertion in # ``tests/cli/test_smoke.py``. assert "Traceback" not in result.stderr + + +@pytest.mark.cli_subprocess +def test_signalforge_init_demo_help_via_subprocess() -> None: + """``signalforge init-demo --help`` exits 0 with the subcommand's help. + + US-006 of ``plans/super/47-init-demo.md`` (DEC-010 / AC-5) — extends + the subprocess-gated smoke to the new ``init-demo`` subcommand so a + ``[project.scripts]`` regression specific to its argparse wiring + (subparser deletion, ``add_parser`` typo, console-script wrapper + losing the dispatch entry) is caught by ``pytest -m cli_subprocess``. + The in-process ``main(argv)`` smoke tests in ``tests/cli/`` cannot + catch this class of regression — they bypass the + ``[project.scripts]`` table entirely. + """ + result = subprocess.run( + ["signalforge", "init-demo", "--help"], + capture_output=True, + text=True, + timeout=10, + ) + + assert result.returncode == 0 + # The presence of the subcommand name, the ``--force`` flag, and the + # ``DEST`` positional (rendered in argparse's help as the metavar + # ``DEST`` AND inside the description prose as ``dest``) jointly + # guarantee argparse is rendering the new subcommand's help, not + # falling back to top-level usage. + assert "init-demo" in result.stdout + assert "--force" in result.stdout + assert "dest" in result.stdout.lower() + # No-traceback floor — see the ``--version`` test above. + assert "Traceback" not in result.stderr diff --git a/tests/fixtures/dbt_project_austin/regenerate.sh b/tests/fixtures/dbt_project_austin/regenerate.sh index 66458972..f12616b1 100755 --- a/tests/fixtures/dbt_project_austin/regenerate.sh +++ b/tests/fixtures/dbt_project_austin/regenerate.sh @@ -24,6 +24,13 @@ # DuckDB-fixture regen). It does NOT modify or replace that script — single # source of truth per fixture, not per repo. # +# Per DEC-015 of plans/super/47-init-demo.md: the final phase mirrors every +# file (except this script) into `src/signalforge/_demo/` then applies two +# demo-only rewrites (`profiles.yml` → env_var() macro per DEC-009; +# `.gitignore` slimmed per DEC-008). Keeping both trees aligned by +# construction means the parity gate at +# `tests/test_demo_fixture_parity.py` fires only on uncommanded drift. +# # Requirements: # * `uvx` (https://docs.astral.sh/uv/) and `jq` on PATH. # * `gcloud auth application-default login` run once (BigQuery uses ADC). @@ -117,3 +124,90 @@ rm -f \ echo "==> Done. Committed manifest:" ls -1 "${TARGET_DIR}" + +# --------------------------------------------------------------------------- +# DEC-015 of plans/super/47-init-demo.md: keep src/signalforge/_demo/ in sync +# with this test fixture so the parity gate at +# tests/test_demo_fixture_parity.py fires only on uncommanded drift. +# +# 1. Mirror every file (except this script) verbatim into the shipped tree. +# 2. Overwrite profiles.yml with the env_var('GOOGLE_CLOUD_PROJECT') variant +# (DEC-009) — the maintainer-only "DO NOT signalforge against this" header +# is dropped because the env_var() lookup makes the shipped copy +# safe-to-run as-is. +# 3. Overwrite .gitignore with the slimmed demo-audience copy (DEC-008) — +# issue-#10 / DEC-021 internal references are removed; only the +# `.signalforge/` exclusion (actually useful to a demo user) remains. +# --------------------------------------------------------------------------- + +REPO_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)" +DEMO_DIR="${REPO_ROOT}/src/signalforge/_demo" + +echo "==> Mirroring fixture tree into ${DEMO_DIR}" +mkdir -p "${DEMO_DIR}" + +# Use rsync if available (cleanly handles the regenerate.sh exclusion); +# fall back to a portable cp -R + explicit prune otherwise. +if command -v rsync >/dev/null 2>&1; then + rsync -a --delete \ + --exclude regenerate.sh \ + "${PROJECT_DIR}/" "${DEMO_DIR}/" +else + rm -rf "${DEMO_DIR}" + mkdir -p "${DEMO_DIR}" + (cd "${PROJECT_DIR}" && find . -mindepth 1 -path ./regenerate.sh -prune -o -print \ + | while IFS= read -r rel; do + rel="${rel#./}" + src_path="${PROJECT_DIR}/${rel}" + dst_path="${DEMO_DIR}/${rel}" + if [[ -d "${src_path}" ]]; then + mkdir -p "${dst_path}" + else + mkdir -p "$(dirname "${dst_path}")" + cp -f "${src_path}" "${dst_path}" + fi + done) +fi + +# Demo-only rewrite #1: profiles.yml uses env_var('GOOGLE_CLOUD_PROJECT') +# (DEC-009). Operator with that env var set runs the demo with zero file +# edits. Written verbatim so the file is fully reproducible from this +# script and the parity test's "rewrite must be different from source" +# clause is preserved. +cat >"${DEMO_DIR}/profiles.yml" <<'PROFILES_YML' +# Demo dbt profile shipped by `signalforge init-demo`. +# +# `method: oauth` falls through to Application Default Credentials — run +# `gcloud auth application-default login` once before invoking +# `signalforge generate` against this project. +# +# `project: "{{ env_var('GOOGLE_CLOUD_PROJECT') }}"` resolves at runtime from +# your billing project; export it before running the demo: +# +# export GOOGLE_CLOUD_PROJECT= +# +# The Austin bikeshare data is served from the public +# `bigquery-public-data.austin_bikeshare` dataset; your `GOOGLE_CLOUD_PROJECT` +# is the BILLING project the BigQuery SDK uses to issue the read. +austin: + target: dev + outputs: + dev: + type: bigquery + method: oauth + project: "{{ env_var('GOOGLE_CLOUD_PROJECT') }}" + dataset: austin_bikeshare + location: US +PROFILES_YML + +# Demo-only rewrite #2: .gitignore slimmed (DEC-008). Drop issue-#10 / +# DEC-021 internal references; keep only the `.signalforge/` exclusion the +# demo audience actually needs. +cat >"${DEMO_DIR}/.gitignore" <<'GITIGNORE' +# SignalForge writes per-run audit logs and sidecar artefacts under .signalforge/. +# These are reproducible from a re-run; no value in committing them. +.signalforge/ +GITIGNORE + +echo "==> Done. Shipped demo files:" +find "${DEMO_DIR}" -type f | sort diff --git a/tests/llm/test_logger_grep_gate.py b/tests/llm/test_logger_grep_gate.py index bf3c56b0..7d0b180d 100644 --- a/tests/llm/test_logger_grep_gate.py +++ b/tests/llm/test_logger_grep_gate.py @@ -44,6 +44,7 @@ # diffing when a future stage extends the list. _SCAN_SUBPACKAGES: tuple[str, ...] = ( "cli", + "demo", "diff", "draft", "grade", diff --git a/tests/test_audit_completeness.py b/tests/test_audit_completeness.py index 1b63ef34..19ec3aa5 100644 --- a/tests/test_audit_completeness.py +++ b/tests/test_audit_completeness.py @@ -584,6 +584,15 @@ def test_grade_event_construction_in_grade_audit_module_is_present() -> None: "GradeError", "DiffError", "CliError", + # ``DemoError`` (issue #47) — abstract base of the + # ``signalforge.demo`` typed-error hierarchy. Its four concrete + # subclasses are wrapped at the CLI handler boundary into + # ``CliInitDemo*Error`` wrappers, but the concretes themselves + # still land in ``_EXCEPTION_TO_EXIT_CODE`` (defence-in-depth so + # a v0.2 ``Demo*Error`` that escapes the ladder gets a sensible + # exit code via the MRO walk). The base is excluded per the + # abstract-base convention. + "DemoError", } ) @@ -704,6 +713,7 @@ def test_scan_7_discovers_every_per_stage_errors_module() -> None: rel_names = sorted(p.relative_to(_SIGNALFORGE_DIR).as_posix() for p in paths) assert rel_names == [ "cli/errors.py", + "demo/errors.py", "diff/errors.py", "draft/errors.py", "grade/errors.py", @@ -713,8 +723,8 @@ def test_scan_7_discovers_every_per_stage_errors_module() -> None: "safety/errors.py", "warehouse/errors.py", ], ( - "Expected exactly nine per-stage errors.py modules (one per " - "stage); got: " + "Expected exactly ten per-stage errors.py modules (one per " + "stage; demo added in #47); got: " f"{rel_names}. If this changes, update Scan 7's expected set." ) diff --git a/tests/test_demo.py b/tests/test_demo.py new file mode 100644 index 00000000..7ac8706a --- /dev/null +++ b/tests/test_demo.py @@ -0,0 +1,404 @@ +"""Tests for the public :mod:`signalforge.demo` module (US-003).""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from signalforge.demo import ( + DemoDestExistsError, + DemoDestUnsafeError, + DemoError, + DemoFixtureMissingError, + DemoPathError, + copy_demo, +) + +# --------------------------------------------------------------------------- +# Helper: expected top-level entries in the shipped demo tree. +# --------------------------------------------------------------------------- + +_EXPECTED_TOP_LEVEL = frozenset( + { + ".gitignore", + "dbt_project.yml", + "models", + "profiles.yml", + "signalforge.yml", + "target", + } +) + + +# --------------------------------------------------------------------------- +# Happy paths +# --------------------------------------------------------------------------- + + +def test_copy_demo_to_empty_dir(tmp_path: Path) -> None: + dest = tmp_path / "demo" + # Pre-create an empty dir — DEC documents that empty dirs proceed. + dest.mkdir() + result = copy_demo(dest) + assert result.is_dir() + entries = {p.name for p in result.iterdir()} + assert _EXPECTED_TOP_LEVEL.issubset(entries) + + +def test_copy_demo_to_nonexistent_dir_creates_and_copies(tmp_path: Path) -> None: + dest = tmp_path / "fresh" + assert not dest.exists() + result = copy_demo(dest) + assert result.is_dir() + entries = {p.name for p in result.iterdir()} + assert _EXPECTED_TOP_LEVEL.issubset(entries) + + +def test_copy_demo_returns_resolved_dest_path(tmp_path: Path) -> None: + dest = tmp_path / "out" + result = copy_demo(dest) + assert isinstance(result, Path) + assert result == dest.expanduser().resolve() + # Resolved path is absolute. + assert result.is_absolute() + + +def test_copy_demo_copies_target_manifest_json(tmp_path: Path) -> None: + """DEC-011: the locked ``target/manifest.json`` ships with the tree.""" + dest = tmp_path / "demo" + result = copy_demo(dest) + manifest = result / "target" / "manifest.json" + assert manifest.is_file() + # Non-empty — pin against an empty-file regression. + assert manifest.stat().st_size > 0 + + +def test_copy_demo_copies_dotfile_gitignore(tmp_path: Path) -> None: + """DEC-006: ``.gitignore`` is a dotfile and must be copied.""" + dest = tmp_path / "demo" + result = copy_demo(dest) + gitignore = result / ".gitignore" + assert gitignore.is_file() + assert gitignore.stat().st_size > 0 + + +def test_copy_demo_with_relative_dest_resolves_against_cwd( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.chdir(tmp_path) + result = copy_demo("demo-rel") + assert result == (tmp_path / "demo-rel").resolve() + assert result.is_dir() + + +def test_copy_demo_with_symlink_dest_resolves_target(tmp_path: Path) -> None: + """A symlink ``dest`` resolves to its target; the symlink itself is + not preserved — the resolved-target directory receives the copy.""" + + real_target = tmp_path / "real" + real_target.mkdir() + symlink = tmp_path / "link" + symlink.symlink_to(real_target) + + result = copy_demo(symlink) + # resolved dest should be the real target, not the symlink itself. + assert result == real_target.resolve() + assert (real_target / "dbt_project.yml").is_file() + + +# --------------------------------------------------------------------------- +# Existence-gate (force=False) +# --------------------------------------------------------------------------- + + +def test_copy_demo_to_nonempty_dir_without_force_raises_dest_exists( + tmp_path: Path, +) -> None: + dest = tmp_path / "demo" + dest.mkdir() + (dest / "preexisting.txt").write_text("hi") + with pytest.raises(DemoDestExistsError) as excinfo: + copy_demo(dest) + # The default remediation footer surfaces in __str__. + rendered = str(excinfo.value) + assert "exists" in rendered + assert "Remediation" in rendered + # The preexisting file is untouched. + assert (dest / "preexisting.txt").read_text() == "hi" + + +def test_copy_demo_dest_is_file_without_force_raises_dest_exists( + tmp_path: Path, +) -> None: + """A ``dest`` that exists as a regular file (not a directory) raises + :class:`DemoDestExistsError` with a "not a directory" message, NOT a + raw ``NotADirectoryError`` from ``iterdir()``. Pinned because the + naive existence-gate would call ``iterdir()`` on a non-directory and + surface a misleading "failed to copy demo tree" wrap through the + CLI. Pass-2 QG defence. + """ + dest = tmp_path / "demo" + dest.write_text("i am a file, not a directory") + with pytest.raises(DemoDestExistsError) as excinfo: + copy_demo(dest) + rendered = str(excinfo.value) + assert "not a directory" in rendered + # The file is untouched. + assert dest.read_text() == "i am a file, not a directory" + + +# --------------------------------------------------------------------------- +# Force semantics +# --------------------------------------------------------------------------- + + +def test_copy_demo_to_nonempty_dir_with_force_replaces_atomically( + tmp_path: Path, +) -> None: + dest = tmp_path / "demo" + dest.mkdir() + (dest / "stale.txt").write_text("stale content") + (dest / "old_subdir").mkdir() + (dest / "old_subdir" / "inside.txt").write_text("also stale") + + result = copy_demo(dest, force=True) + assert result.is_dir() + # Stale content is gone. + assert not (dest / "stale.txt").exists() + assert not (dest / "old_subdir").exists() + # Demo content is present. + assert (dest / "dbt_project.yml").is_file() + assert (dest / "target" / "manifest.json").is_file() + + +def test_copy_demo_dest_is_file_with_force_replaces_with_demo_dir( + tmp_path: Path, +) -> None: + """``force=True`` against a ``dest`` that exists as a regular file + unlinks the file and copies the demo tree in its place. Pinned + because the existence-gate's "remove and replace" semantics applies + to non-directory contents too. Pass-2 QG defence. + """ + dest = tmp_path / "demo" + dest.write_text("i am a file that will be replaced") + result = copy_demo(dest, force=True) + assert result == dest.resolve() + assert result.is_dir() + assert (dest / "dbt_project.yml").is_file() + assert (dest / "target" / "manifest.json").is_file() + + +def test_copy_demo_force_against_home_raises_dest_unsafe( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + fake_home = tmp_path / "fake_home" + fake_home.mkdir() + monkeypatch.setattr(Path, "home", lambda: fake_home) + with pytest.raises(DemoDestUnsafeError) as excinfo: + copy_demo(fake_home, force=True) + assert "force" in str(excinfo.value).lower() or "system or user" in str(excinfo.value) + # The "home" dir was not nuked. + assert fake_home.is_dir() + + +def test_copy_demo_force_against_root_raises_dest_unsafe() -> None: + with pytest.raises(DemoDestUnsafeError): + copy_demo("/", force=True) + + +def test_copy_demo_force_against_cwd_raises_dest_unsafe( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.chdir(tmp_path) + with pytest.raises(DemoDestUnsafeError): + copy_demo(tmp_path, force=True) + # cwd directory wasn't nuked. + assert tmp_path.is_dir() + + +def test_copy_demo_force_against_symlink_dest_raises_dest_unsafe( + tmp_path: Path, +) -> None: + """``force=True`` against a symlink ``dest`` refuses to proceed. + + The naive code path would ``rmtree(resolved_dest)`` after ``resolve()`` + followed the link, which would delete the link's TARGET (an arbitrary + external location) instead of just the link itself. Defence: + ``DemoDestUnsafeError`` before any destructive operation. The + no-force symlink-dest behaviour is preserved by + ``test_copy_demo_with_symlink_dest_resolves_target`` above. + """ + real_target = tmp_path / "target_dir_we_must_not_clobber" + real_target.mkdir() + (real_target / "important.txt").write_text("don't nuke me") + + symlink = tmp_path / "demo_link" + symlink.symlink_to(real_target) + + with pytest.raises(DemoDestUnsafeError) as excinfo: + copy_demo(symlink, force=True) + assert "symlink" in str(excinfo.value).lower() + # Critical: the target directory + its content survived. + assert real_target.is_dir() + assert (real_target / "important.txt").read_text() == "don't nuke me" + # The symlink itself also survived (not unlink()-ed before the raise). + assert symlink.is_symlink() + + +# --------------------------------------------------------------------------- +# Symlink cycle → DemoPathError +# --------------------------------------------------------------------------- + + +def test_copy_demo_with_cyclic_symlink_raises_demo_path_error(tmp_path: Path) -> None: + """A symlink cycle at the destination raises :class:`DemoPathError`.""" + + # Create A -> B and B -> A (mutually pointing symlinks form a resolve cycle). + link_a = tmp_path / "loop_a" + link_b = tmp_path / "loop_b" + link_a.symlink_to(link_b) + link_b.symlink_to(link_a) + + # On some filesystems (notably WSL2), resolve() does NOT raise on this + # pattern — it returns a path with the symlink unresolved. Skip when + # the platform doesn't enforce the cycle guard the way the contract + # expects; the GitHub Actions Linux runner does enforce it. + try: + link_a.resolve(strict=False) + except RuntimeError: + pass + else: + pytest.skip( + "filesystem does not raise RuntimeError on symlink cycles; " + "DemoPathError path is verified on the CI Linux runner" + ) + + with pytest.raises(DemoPathError) as excinfo: + copy_demo(link_a) + # The triggering RuntimeError rides on the cause. + assert isinstance(excinfo.value.cause, RuntimeError) + + +# --------------------------------------------------------------------------- +# DemoFixtureMissingError — broken-install path +# --------------------------------------------------------------------------- + + +def test_copy_demo_fixture_missing_raises(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """If ``importlib.resources`` cannot locate ``_demo``, raise. + + Simulated by monkeypatching the ``files`` lookup to return a + non-directory traversable. + """ + + import signalforge.demo as demo_mod + + class _NotADir: + def joinpath(self, name: str) -> _NotADir: + return self + + def is_dir(self) -> bool: + return False + + monkeypatch.setattr(demo_mod, "files", lambda pkg: _NotADir()) + with pytest.raises(DemoFixtureMissingError) as excinfo: + copy_demo(tmp_path / "demo") + assert "bundled" in str(excinfo.value) or "missing" in str(excinfo.value).lower() + + +# --------------------------------------------------------------------------- +# Error-class shape +# --------------------------------------------------------------------------- + + +def test_demo_errors_share_base_class() -> None: + assert issubclass(DemoDestExistsError, DemoError) + assert issubclass(DemoDestUnsafeError, DemoError) + assert issubclass(DemoPathError, DemoError) + assert issubclass(DemoFixtureMissingError, DemoError) + + +def test_demo_error_str_renders_remediation_footer() -> None: + err = DemoDestExistsError("destination 'x' exists and is not empty") + rendered = str(err) + assert "destination 'x' exists and is not empty" in rendered + assert "↳ Remediation:" in rendered + + +def test_demo_error_str_without_remediation_is_message_only() -> None: + err = DemoError("plain message", remediation=None) + # Base class default_remediation is None; subclasses set defaults. + assert str(err) == "plain message" + + +def test_demo_error_remediation_override_wins() -> None: + err = DemoDestExistsError("dest 'y' exists", remediation="custom hint") + assert "custom hint" in str(err) + assert "Remove the destination" not in str(err) + + +# --------------------------------------------------------------------------- +# Public API import surface +# --------------------------------------------------------------------------- + + +def test_public_import_surface() -> None: + # The README + DEC-012 promises this exact import. + # The signature accepts (dest, *, force). + import inspect + + from signalforge.demo import copy_demo as _copy_demo # noqa: F401 + + sig = inspect.signature(_copy_demo) + params = list(sig.parameters.values()) + assert params[0].name == "dest" + assert sig.parameters["force"].kind == inspect.Parameter.KEYWORD_ONLY + assert sig.parameters["force"].default is False + + +def test_module_all_lists_public_surface() -> None: + import signalforge.demo as demo_mod + + expected = { + "DemoDestExistsError", + "DemoDestUnsafeError", + "DemoError", + "DemoFixtureMissingError", + "DemoPathError", + "copy_demo", + } + assert set(demo_mod.__all__) == expected + + +# --------------------------------------------------------------------------- +# Sanity: the copy preserves a nested file structure +# --------------------------------------------------------------------------- + + +def test_copy_demo_preserves_nested_model_sql(tmp_path: Path) -> None: + dest = tmp_path / "demo" + copy_demo(dest) + nested = dest / "models" / "staging" / "stg_bikeshare_trips.sql" + assert nested.is_file() + assert nested.stat().st_size > 0 + + +def test_copy_demo_does_not_follow_into_unrelated_symlinks_on_destination( + tmp_path: Path, +) -> None: + """A symlink at ``dest`` that points to a target outside ``tmp_path`` + is followed via ``.resolve()`` — the test exists to pin behaviour, + not to catch a regression (since ``init-demo`` deliberately does + not enforce a containment boundary per DEC-004).""" + + real_target = tmp_path / "other" + real_target.mkdir() + symlink = tmp_path / "link" + symlink.symlink_to(real_target) + + result = copy_demo(symlink) + assert result == real_target.resolve() + # Confirms the resolved dest is real_target, not the symlink path. + assert not result.is_symlink() + assert (result / "dbt_project.yml").is_file() diff --git a/tests/test_demo_fixture_parity.py b/tests/test_demo_fixture_parity.py new file mode 100644 index 00000000..02393639 --- /dev/null +++ b/tests/test_demo_fixture_parity.py @@ -0,0 +1,151 @@ +"""Parity test between ``src/signalforge/_demo/`` and ``tests/fixtures/dbt_project_austin/``. + +Implements DEC-008 of ``plans/super/47-init-demo.md``: the shipped demo tree +must stay byte-for-byte equal to the e2e-smoke fixture tree EXCEPT for two +documented rewrites: + +1. ``profiles.yml`` — the shipped copy uses dbt's ``env_var('GOOGLE_CLOUD_PROJECT')`` + macro for the BigQuery project field and drops the maintainer-only + "DO NOT signalforge against this" header (DEC-009). +2. ``.gitignore`` — the shipped copy is slimmed to a single ``.signalforge/`` + exclusion; the test-fixture copy keeps the issue-#10 / DEC-021 maintainer + commentary. + +Additional invariants: + +* ``regenerate.sh`` lives only in the test fixture (maintainer-only; + documented in DEC-015 — the shipped tree does not include it). +* The shipped ``_demo/`` tree contains zero symlinks (DEC-005). + +The maintainer-only ``tests/fixtures/dbt_project_austin/regenerate.sh`` script +updates BOTH trees in lockstep so this parity gate fires only on uncommanded +drift. +""" + +from __future__ import annotations + +from pathlib import Path + +_REPO_ROOT = Path(__file__).resolve().parent.parent +_TEST_FIXTURE_DIR = _REPO_ROOT / "tests" / "fixtures" / "dbt_project_austin" +_DEMO_DIR = _REPO_ROOT / "src" / "signalforge" / "_demo" + +# Files allowed to diverge between the two trees. Every other file in the demo +# tree MUST be byte-equal to its test-fixture counterpart. Adding to this list +# is a deliberate widening of the rewrite surface and MUST be accompanied by an +# updated DEC entry in plans/super/47-init-demo.md. +_ALLOWED_REWRITES = frozenset({"profiles.yml", ".gitignore"}) + +# Files allowed to live only in the test-fixture tree (maintainer-only artefacts +# that DO NOT ship in the wheel). The shipped demo intentionally omits these. +_TEST_FIXTURE_ONLY = frozenset({"regenerate.sh"}) + + +def _relative_files(root: Path) -> set[str]: + return {str(p.relative_to(root)) for p in root.rglob("*") if p.is_file()} + + +def test_demo_fixture_parity_holds_byte_for_byte_except_documented_files() -> None: + """The two trees must agree byte-for-byte except for the two named rewrites. + + Implements DEC-008. The exceptions documented inline are: + - profiles.yml (DEC-009 — env_var('GOOGLE_CLOUD_PROJECT') swap) + - .gitignore (DEC-008 — slim out issue-#10 / DEC-021 references) + """ + test_files = _relative_files(_TEST_FIXTURE_DIR) + demo_files = _relative_files(_DEMO_DIR) + + # The shipped tree must contain every test-fixture file except the + # maintainer-only set. + expected_in_demo = test_files - _TEST_FIXTURE_ONLY + missing_from_demo = expected_in_demo - demo_files + assert not missing_from_demo, ( + f"shipped demo is missing files present in test fixture: {sorted(missing_from_demo)}" + ) + + # The shipped tree must NOT carry unexpected extras (anything not in the + # test fixture). Net-new shipped files require an explicit DEC. + unexpected_extras = demo_files - test_files + assert not unexpected_extras, ( + f"shipped demo has files not in test fixture: {sorted(unexpected_extras)}" + ) + + # Maintainer-only files (regenerate.sh) must NOT ship in the demo tree. + leaked_maintainer_files = demo_files & _TEST_FIXTURE_ONLY + assert not leaked_maintainer_files, ( + f"maintainer-only files leaked into shipped demo: {sorted(leaked_maintainer_files)}" + ) + + # Every shared file must be byte-equal except for the two documented + # rewrites. The rewrites are still asserted to BE DIFFERENT below so a + # silent identical copy doesn't pass the parity gate while violating + # DEC-009. + drift: list[str] = [] + for rel in sorted(expected_in_demo): + test_bytes = (_TEST_FIXTURE_DIR / rel).read_bytes() + demo_bytes = (_DEMO_DIR / rel).read_bytes() + if rel in _ALLOWED_REWRITES: + if test_bytes == demo_bytes: + drift.append( + f"{rel} is identical between trees but must be rewritten per DEC-008/DEC-009" + ) + else: + if test_bytes != demo_bytes: + drift.append(f"{rel} differs between trees (uncommanded drift)") + assert not drift, "parity drift detected:\n - " + "\n - ".join(drift) + + +def test_demo_fixture_contains_no_symlinks() -> None: + """Codifies DEC-005: the shipped demo tree is symlink-free. + + Walks ``src/signalforge/_demo/`` and asserts ``not p.is_symlink()`` for + every entry. Drift here (e.g. an accidental ``ln -s`` during a regen) + breaks the test loudly. + """ + assert _DEMO_DIR.is_dir(), f"demo tree missing at {_DEMO_DIR}" + offending = [str(p.relative_to(_DEMO_DIR)) for p in _DEMO_DIR.rglob("*") if p.is_symlink()] + assert not offending, f"shipped demo contains symlinks (DEC-005 violation): {sorted(offending)}" + + +def test_demo_profiles_yml_uses_env_var_macro() -> None: + """Pins DEC-009: the shipped profile uses ``env_var('GOOGLE_CLOUD_PROJECT')``. + + A PyPI user with the env var set runs the demo with zero file edits. + """ + profile_text = (_DEMO_DIR / "profiles.yml").read_text() + assert "env_var('GOOGLE_CLOUD_PROJECT')" in profile_text, ( + "shipped profiles.yml must use env_var('GOOGLE_CLOUD_PROJECT') per DEC-009; " + f"got:\n{profile_text}" + ) + # The shipped copy must NOT carry the maintainer-only "DO NOT signalforge" + # warning header — that header is specifically about the billing-broken + # bigquery-public-data placeholder which the shipped copy replaces. + assert "DO NOT" not in profile_text, ( + "shipped profiles.yml must not carry the maintainer-only DO NOT header" + ) + # The shipped copy must NOT carry the broken billing placeholder. + assert "project: bigquery-public-data" not in profile_text, ( + "shipped profiles.yml must not pin project: bigquery-public-data " + "(it's the billing-broken placeholder; use env_var(...) instead)" + ) + + +def test_test_fixture_profiles_yml_retains_maintainer_header() -> None: + """Confirms the DEC-009 rewrite is one-way. + + The test-fixture copy keeps the "do not signalforge against this" header + plus the billing-broken ``bigquery-public-data`` placeholder so the e2e + smoke test (issue #10) overwrites it in ``tmp_path`` with the operator's + real billing project. The shipped copy does the opposite swap (DEC-009). + """ + profile_text = (_TEST_FIXTURE_DIR / "profiles.yml").read_text() + assert "project: bigquery-public-data" in profile_text, ( + "test-fixture profiles.yml must retain the bigquery-public-data " + "placeholder so the e2e smoke test exercises the overwrite path; " + f"got:\n{profile_text}" + ) + # The maintainer-only warning header must remain in the test-fixture copy. + assert "WRONG for query-time use" in profile_text, ( + "test-fixture profiles.yml must retain the maintainer-only warning " + "header explaining why running signalforge directly against it fails" + ) diff --git a/tests/test_wheel_packaging.py b/tests/test_wheel_packaging.py new file mode 100644 index 00000000..a9b53f05 --- /dev/null +++ b/tests/test_wheel_packaging.py @@ -0,0 +1,162 @@ +"""Maintainer-only wheel packaging smoke for the demo tree. + +US-002 (issue #47) — gated by ``@pytest.mark.wheel_smoke`` so the default +``pytest`` run skips it. Maintainers run ``pytest -m wheel_smoke --no-cov`` +before declaring an init-demo PR ready. + +The test builds the wheel via ``python -m build --wheel --outdir `` +(falling back to ``uvx --from build pyproject-build`` when ``build`` is +not installed in the active Python — matches the project's ``uvx`` +convention for ephemeral build tooling, see +``tests/fixtures/regenerate.sh``), opens the artifact via :mod:`zipfile`, +and asserts that the canonical ``src/signalforge/_demo/`` file set ships +under ``signalforge/_demo/`` inside the wheel. Mirrors the +``cli_subprocess`` precedent (``tests/cli/test_subprocess_smoke.py``) +for marker-gated subprocess smokes; the ``--no-cov`` flag is required +because the coverage gate in ``addopts`` would fail a marker-specific +run that exercises only this file (see ``testing-signal.md`` § +"Coverage measurement" / "Known gap"). + +Closes the P-1 BLOCKER from ``plans/super/47-init-demo.md``: Hatchling's +default ``packages`` glob behaviour on non-``.py`` data files is not +contractually guaranteed, so we add an explicit ``include`` directive in +``pyproject.toml`` AND gate the result with this wheel-build inspection. + +DEC-002 — ``[tool.hatch.build.targets.wheel] include = ["src/signalforge/_demo"]``. +DEC-003 — ``wheel_smoke`` marker + this test. +DEC-006 — Ship ``.gitignore`` as-is; the dedicated dotfile test below +pins inclusion explicitly. +""" + +from __future__ import annotations + +import importlib.util +import shutil +import subprocess +import sys +import zipfile +from pathlib import Path + +import pytest + +# Repo root resolved from this file's location so the test is invariant to +# pytest's cwd. ``tests/`` lives directly under the repo root. +_REPO_ROOT = Path(__file__).resolve().parent.parent + +# Canonical demo file set under ``signalforge/_demo/`` inside the built +# wheel. Sourced from the on-disk tree at ``src/signalforge/_demo/`` (7 +# files; the Austin test fixture has 8 because it carries ``regenerate.sh`` +# which is maintainer-only and deliberately excluded from the demo per +# ``plans/super/47-init-demo.md`` line 66). Drift between this list and the +# on-disk tree fails the test loudly — that is the point. +_EXPECTED_DEMO_FILES: tuple[str, ...] = ( + "signalforge/_demo/.gitignore", + "signalforge/_demo/dbt_project.yml", + "signalforge/_demo/profiles.yml", + "signalforge/_demo/signalforge.yml", + "signalforge/_demo/models/staging/sources.yml", + "signalforge/_demo/models/staging/stg_bikeshare_trips.sql", + "signalforge/_demo/target/manifest.json", +) + + +def _build_command(outdir: Path) -> list[str]: + """Pick the wheel-build invocation available in the current environment. + + Prefers ``python -m build`` when the active interpreter has ``build`` + importable (this is the ticket's canonical invocation, US-002 of + ``plans/super/47-init-demo.md``). Falls back to ``uvx --from build + pyproject-build`` when ``build`` is not installed — mirrors the + project's ephemeral-tooling convention (``tests/fixtures/regenerate.sh`` + uses ``uvx`` for ephemeral ``dbt`` installs at pinned versions). + + Raises ``RuntimeError`` if neither path is available — the maintainer + needs a loud, actionable failure rather than a confusing + ``CalledProcessError`` for "No module named build". + """ + if importlib.util.find_spec("build") is not None: + return [sys.executable, "-m", "build", "--wheel", "--outdir", str(outdir)] + uvx = shutil.which("uvx") + if uvx is not None: + return [uvx, "--from", "build", "pyproject-build", "--wheel", "--outdir", str(outdir)] + raise RuntimeError( + "wheel_smoke needs `python -m build` available. Install via " + "`pip install build` in the active venv, or install `uvx` " + "(https://docs.astral.sh/uv/) on PATH for the ephemeral fallback." + ) + + +def _build_wheel(outdir: Path) -> Path: + """Build the wheel into ``outdir`` and return the resolved artifact path. + + Returns the resolved path to the freshly built ``.whl`` artifact. + Raises ``subprocess.CalledProcessError`` on build failure (the + maintainer needs to see the error to fix it) or ``AssertionError`` + if no wheel landed in ``outdir``. A 60-second timeout mirrors the + ``cli_subprocess`` precedent — Hatchling on this repo builds in + well under 5 seconds; a 60s timeout means a real regression, not a + slow build. + """ + subprocess.run( + _build_command(outdir), + cwd=str(_REPO_ROOT), + check=True, + capture_output=True, + text=True, + timeout=60, + ) + wheels = sorted(outdir.glob("*.whl")) + assert wheels, f"no wheel artifact landed in {outdir}" + # Exactly one wheel is expected per build invocation. + assert len(wheels) == 1, f"expected one wheel in {outdir}, got {wheels}" + return wheels[0] + + +@pytest.fixture(scope="module") +def _built_wheel_members(tmp_path_factory: pytest.TempPathFactory) -> set[str]: + """Build the wheel ONCE per test-module run and return its member list. + + Module-scoped because ``python -m build --wheel`` is the expensive + operation here (~5-15s including PEP 517 isolation). Running it once + and asserting separate invariants over the resulting member set keeps + ``pytest -m wheel_smoke`` fast without weakening either assertion. + """ + outdir = tmp_path_factory.mktemp("wheel-build") + wheel_path = _build_wheel(outdir) + with zipfile.ZipFile(wheel_path) as zf: + return set(zf.namelist()) + + +@pytest.mark.wheel_smoke +def test_wheel_includes_all_demo_files(_built_wheel_members: set[str]) -> None: + """Every file in ``src/signalforge/_demo/`` ships in the built wheel. + + Gates DEC-002 (``include = ["src/signalforge/_demo"]``) at packaging + time. Without the directive the wheel ships zero demo data files — + Hatchling's default ``packages`` glob is not guaranteed to pick up + non-``.py`` files. + """ + missing = [name for name in _EXPECTED_DEMO_FILES if name not in _built_wheel_members] + assert not missing, ( + f"wheel is missing demo files: {missing}. " + f"Check `[tool.hatch.build.targets.wheel] include` in pyproject.toml." + ) + + +@pytest.mark.wheel_smoke +def test_wheel_includes_demo_gitignore_dotfile(_built_wheel_members: set[str]) -> None: + """``signalforge/_demo/.gitignore`` ships in the wheel (DEC-006). + + Hatchling's ``include`` glob behaviour on dotfiles is not contractually + guaranteed (see ``plans/super/47-init-demo.md`` P-6). If this test + fails, the fallback per DEC-006 is to rename the source-tree file to + ``gitignore.demo`` and have ``copy_demo`` rewrite the on-disk name at + copy time. The wheel_smoke surface is the load-bearing gate for + discovering the regression — manual ``unzip -l`` at release time is + too late. + """ + assert "signalforge/_demo/.gitignore" in _built_wheel_members, ( + "wheel does not ship `signalforge/_demo/.gitignore`. " + "Hatchling may have dropped the dotfile under the directory glob; " + "see DEC-006 of plans/super/47-init-demo.md for the fallback." + ) diff --git a/tests/warehouse/test_errors.py b/tests/warehouse/test_errors.py index faa67f18..f6e14bda 100644 --- a/tests/warehouse/test_errors.py +++ b/tests/warehouse/test_errors.py @@ -40,6 +40,10 @@ "WarehouseAuthError": {"message": "auth failed"}, "UnsupportedProfileTypeError": {"profile_type": "snowflake"}, "UnsupportedAuthMethodError": {"method": "service-account"}, + "ProfileEnvVarUnsetError": { + "var_name": "MY_BILLING_PROJECT", + "profiles_path": Path("/etc/dbt/profiles.yml"), + }, "ProfileNotFoundError": {"searched_paths": [Path("/etc/dbt/profiles.yml")]}, "ProfileTargetNotFoundError": {"profile_name": "myproj", "target": "prod"}, "ManifestProjectNotFoundError": {"model_unique_id": "model.proj.foo"}, @@ -86,12 +90,15 @@ def test_each_subclass_has_default_remediation() -> None: # UnknownTableSizeError) plus the WarehouseError base = 16 classes; # issue #22 (US-001) adds MaterialisationFailedError and # MaterialisationNotSupportedError → 18; issue #36 (US-002) adds - # EstimateNotSupportedError → 19. - assert len(errors_module.__all__) == 19, ( + # EstimateNotSupportedError → 19; issue #47 adds + # ProfileEnvVarUnsetError (supports init-demo profile env_var + # rendering) → 20. + assert len(errors_module.__all__) == 20, ( "DEC-026 enumerates 15 typed subclasses + 1 base; #22 US-001 " "adds 2 more (MaterialisationFailed/NotSupported); #36 US-002 " - "adds EstimateNotSupportedError. Update tests and __all__ " - "together if this changes." + "adds EstimateNotSupportedError; #47 QG pass-3 adds " + "ProfileEnvVarUnsetError. Update tests and __all__ together " + "if this changes." ) for name in errors_module.__all__: cls = getattr(errors_module, name) diff --git a/tests/warehouse/test_profiles.py b/tests/warehouse/test_profiles.py index 760ba0eb..86fac1d7 100644 --- a/tests/warehouse/test_profiles.py +++ b/tests/warehouse/test_profiles.py @@ -21,6 +21,7 @@ from signalforge.warehouse import profiles as profiles_module from signalforge.warehouse.errors import ( + ProfileEnvVarUnsetError, ProfileNotFoundError, ProfileTargetNotFoundError, UnsupportedAuthMethodError, @@ -330,6 +331,117 @@ def test_load_profile_warns_on_large_yaml( ) +# --------------------------------------------------------------------------- +# 6b. env_var() macro rendering (issue #47 — supports init-demo profiles.yml) +# --------------------------------------------------------------------------- + + +def _write_env_var_profile(project_dir: Path, env_var_expr: str) -> None: + """Helper: write a minimal profile that references `env_var(...)`.""" + (project_dir / "profiles.yml").write_text( + "signalforge_test:\n" + " target: dev\n" + " outputs:\n" + " dev:\n" + " type: bigquery\n" + " method: oauth\n" + f' project: "{{{{ {env_var_expr} }}}}"\n' + " dataset: austin_bikeshare\n" + " location: US\n", + encoding="utf-8", + ) + + +def test_load_profile_renders_env_var_macro( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """``env_var('NAME')`` resolves to the environment value at load time. + + Mirrors the dbt convention so the bundled ``init-demo`` profile + (which uses ``{{ env_var('GOOGLE_CLOUD_PROJECT') }}``) works without + profile edits when the operator has the env var set. + """ + _clear_profile_env(monkeypatch) + monkeypatch.setattr(Path, "home", lambda: tmp_path / "fake_home") + monkeypatch.setenv("MY_BILLING_PROJECT", "billing-prod-42") + + project_dir = tmp_path / "project" + project_dir.mkdir() + _write_dbt_project(project_dir) + _write_env_var_profile(project_dir, "env_var('MY_BILLING_PROJECT')") + + target = load_profile(project_dir) + assert target.project == "billing-prod-42" + + +def test_load_profile_env_var_with_default_uses_default( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """``env_var('NAME', 'default')`` falls back to the literal default + when NAME is unset (dbt convention).""" + _clear_profile_env(monkeypatch) + monkeypatch.setattr(Path, "home", lambda: tmp_path / "fake_home") + monkeypatch.delenv("UNSET_BILLING_PROJECT", raising=False) + + project_dir = tmp_path / "project" + project_dir.mkdir() + _write_dbt_project(project_dir) + _write_env_var_profile(project_dir, "env_var('UNSET_BILLING_PROJECT', 'fallback-project')") + + target = load_profile(project_dir) + assert target.project == "fallback-project" + + +def test_load_profile_env_var_unset_no_default_raises( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """``env_var('NAME')`` with no default and NAME unset raises + :class:`ProfileEnvVarUnsetError` — dbt's documented behaviour. + + This is the load-bearing test for init-demo's UX: the first-run + operator who forgets to ``export GOOGLE_CLOUD_PROJECT`` before + ``signalforge lint`` / ``generate`` gets a clear typed error pointing + at the missing env var, not a downstream BigQuery rejection of the + literal jinja string. + """ + _clear_profile_env(monkeypatch) + monkeypatch.setattr(Path, "home", lambda: tmp_path / "fake_home") + monkeypatch.delenv("DEFINITELY_NOT_SET_47", raising=False) + + project_dir = tmp_path / "project" + project_dir.mkdir() + _write_dbt_project(project_dir) + _write_env_var_profile(project_dir, "env_var('DEFINITELY_NOT_SET_47')") + + with pytest.raises(ProfileEnvVarUnsetError) as excinfo: + load_profile(project_dir) + assert excinfo.value.var_name == "DEFINITELY_NOT_SET_47" + rendered = str(excinfo.value) + assert "DEFINITELY_NOT_SET_47" in rendered + assert "↳ Remediation:" in rendered + + +def test_load_profile_env_var_preserves_yaml_quoting( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """A quoted ``"{{ env_var('NAME') }}"`` substitutes the value while + preserving the surrounding YAML string context — the rendered + ``project`` field is a plain string, not a parsed int / bool.""" + _clear_profile_env(monkeypatch) + monkeypatch.setattr(Path, "home", lambda: tmp_path / "fake_home") + # Use a numeric-looking value to verify YAML doesn't coerce to int. + monkeypatch.setenv("NUMERIC_PROJECT", "12345") + + project_dir = tmp_path / "project" + project_dir.mkdir() + _write_dbt_project(project_dir) + _write_env_var_profile(project_dir, "env_var('NUMERIC_PROJECT')") + + target = load_profile(project_dir) + assert target.project == "12345" + assert isinstance(target.project, str) + + # --------------------------------------------------------------------------- # 7. Drift detector (DEC-017) # ---------------------------------------------------------------------------